Papers

PBit1:每个参数用一个可编程 p-bit 表示预训练网络

PBit1: One Programmable P-Bit per Parameter - A Finite-Control Representation of Pretrained Networks

Why we picked it

这篇论文把每个权重换成单个概率型 p-bit,用时间观测重建原网络,GPT-2 和 LFM3 困惑度只涨 0.66% 和 0.26%。对研究概率计算、新型存算硬件的人来说是个很具体的构造方案,做 THRML 硬件实验的团队尤其该看。

arXiv 论文提出 PBit1,一种预训练神经网络的有限控制表示方法:每个学习到的参数对应一个可编程 p-bit,通过 12-bit 命令设定概率,共享仿射范围将概率映射到权重域,再用时间观测重建权重做推理。方法还引入算子感知的范围分组和基于敏感度的采样分配,使表示适配网络结构。

Body

实验中在 32,768 次观测的条件下,选定的配置在 GPT-2、DistilBERT 和 LFM3-1.7B 三个模型上保留了任务质量,全部学习标量都被转换且无需微调。GPT-2 与 LFM3 的困惑度分别上升约 0.66% 和 0.26%,DistilBERT 准确率从 87.51% 变为 87.49%。基于敏感度的策略把 GPT-2 的重建观测数减少 41.7%,同时保持质量标准。完整模型评估在 BF16 推理中复用重建权重,并在 THRML 算子实验中检验原生采样和时空权衡。

Original · arXiv cs.AI

PBit1: One Programmable P-Bit per Parameter - A Finite-Control Representation of Pretrained Networks

We introduce PBit1, a finite-control representation of pretrained neural networks that assigns one programmable p-bit to each learned parameter. A 12-bit command sets its probability, a shared affine range maps that probability to the weight domain, and temporal observations reconstruct the weight for inference. Operator-aware range grouping and sensitivity-based sampling allocation adapt the representation to the network. Across five independent weight draws at 32,768 observations per scalar, selected configurations preserve task quality in GPT-2, DistilBERT, and LFM3-1.7B with every learned scalar converted and no fine-tuning. GPT-2 and LFM3 perplexity increase by approximately 0.66% and 0.26%, respectively, while DistilBERT accuracy changes from 87.51% to 87.49%. A sensitivity-based policy reduces GPT-2 reconstruction observations by 41.7% while retaining the quality criterion. Complete-model evaluations reuse reconstructed weights in BF16 inference, and THRML operator experiments examine native sampling and spatial-temporal tradeoffs. Together, these results establish PBit1 as a concrete construction for studying finite-control weight representation and temporal reconstruction in pretrained networks.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include p-bit, GPT-2, DistilBERT, THRML.

Source attribution: curated from arXiv cs.AI (@Omar Ramadan).