这篇论文用字符串相似度和回溯搜索替代了算术逻辑,让LLM在位操作谜题上验证精度超过96%,比赛第7名,方法很巧妙。
这篇论文介绍了在NVIDIA Nemotron Model Reasoning Challenge中解决Bit Manipulation Puzzles的创新算法。该任务要求发现隐藏的逻辑规则并应用于新输入,但LLMs通常因复杂布尔逻辑模拟而幻觉。作者提出放弃算术逻辑,转而使用字符串相似性、结构化搜索和自主错误恢复。他们将逻辑门推导重构为基选择任务,利用最小比特翻转来隔离基并推导真值表,无需复杂算术。通过回溯DFS和错误恢复,结合比特分词和交互推理SFT,该方法在验证集上达到96%以上的准确率,最终获得比赛第7名。
Teaching LLMs String Matching, Backtracking, and Error Recovery to Deduce Bases and Truth Tables for the Combinatorially Exploding Bit Manipulation Puzzles
This paper presents our algorithmic innovations for the NVIDIA Nemotron Model Reasoning Challenge, focusing on Bit Manipulation Puzzles. In this task, the objective is to discover a hidden logical rule transforming input binary strings to outputs, then apply it to unseen inputs. Large Language Models (LLMs) notoriously struggle here; traditional methods force them to simulate complex boolean logic and arithmetic, leading to hallucinations. Furthermore, the search space of bitwise operations (combinations of shifts, rotations, and logic gates) suffers from a severe combinatorial explosion. To overcome this computational intractability, we present a novel approach that abandons arithmetic logic entirely in favor of string similarity, structured search, and autonomous error recovery. Our core contributions are: 1. Bases and Truth Table Formulation: We reframe logic-gate deduction into a base-selection task, leveraging string similarity (minimal bit flips) to isolate primitive transformations ("bases") and deduce truth tables without complex arithmetic. 2. Backtracking DFS and Error Recovery: We formalize a search process that tests candidate bases, detects logical collisions across examples, and backtracks upon failure to perform robust error recovery. 3. Bit Tokenization and Interactive Reasoning SFT: We force the tokenizer to encode binary strings as individual single-bit tokens. We use dynamic masking to simulate external oracle feedback, training the model to hypothesize, self-evaluate, and backtrack natively. Evaluated on bit manipulation puzzles, our approach achieved over 96% validation accuracy. This represents the highest performance in this category, driving our 7th Place overall finish in the contest.