ProtAgent-8B Matches Frontier Models on Protein Tool Tasks
ProtAgent: Post-Training Compact Protein Tool Agents with Interaction-Level Credit Assignment
Interaction-level credit assignment, not raw model scale, is what closes the gap with frontier systems here — a 671B model loses by roughly 20 points under the same protocol. Researchers building tool-using agents in specialized domains can apply the same step-level reward scheme without frontier API budgets. The authors' caveat that annotation coverage does not prove biological correctness is worth keeping in mind when comparing reported scores.
ProtAgent is a two-stage post-training pipeline that trains a compact 8B model to drive real bioinformatics tools, combining supervised fine-tuning on the tool interface with on-policy REINFORCE for the interaction policy. Step-level returns assign action rewards to individual tool calls, while the final answer receives a terminal reward. Under a fixed closed-loop annotation-coverage protocol, ProtAgent-8B reaches 61.00% success, versus 60.80% for the strongest evaluated frontier system and 41.60% for DeepSeek-V3.2-Thinking 671B. In three-seed comparisons, step-level returns lift mean success to 61.40% (identity normalization) and 58.20% (group-relative), against 42.70% and 36.50% with episode-level baselines. The authors note the metric measures recovery of reference annotations and does not by itself establish biological correctness.
ProtAgent: Post-Training Compact Protein Tool Agents with Interaction-Level Credit Assignment
Protein annotation agents must choose suitable bioinformatics tools, interpret their outputs, and use the resulting evidence in the final answer. Most existing systems rely on frontier-scale models. We ask whether a compact model can instead learn this behavior through post-training with real tool interaction. We present ProtAgent, a two-stage pipeline that uses supervised fine-tuning (SFT) to learn the tool interface and standard on-policy REINFORCE to improve the interaction policy. Intermediate tool calls receive action rewards, and the final response receives a terminal answer reward. Step-level returns assign these rewards to the responses that produced them and to preceding actions. The episode-level baseline instead assigns one episode return to every action. On a fixed closed-loop protocol with real bioinformatics tools, ProtAgent-8B achieves 61.00% success under the annotation-coverage metric, compared with 41.60% for DeepSeek-V3.2-Thinking 671B and 60.80% for the strongest evaluated frontier system. In the three-seed comparison, mean success is 42.70% with episode-level returns and 61.40% with step-level returns under identity return normalization; the corresponding group-relative results are 36.50% and 58.20%. The metric measures recovery of reference annotations under this protocol; it does not by itself establish biological correctness or complete evidence grounding.
