想看看AI在药物发现中到底行不行？这个基准测试用4800条轨迹告诉你，Claude Opus 4.8和GPT-5.5都还差得远，最高才59.3%的通过率。
TxBench-PP是一个用于评估AI agent在小分子临床前药理学中决策能力的基准，包含100个涉及作用机制、药效学等任务的评估。在16个模型配置（涉及11个模型和4800条轨迹）中，最佳配置Claude Opus 4.8 / Pi仅通过59.3%（178/300）的端点尝试，GPT-5.5 / Pi通过55.3%。结果表明，当前AI系统无法可靠复现临床前药理学决策。
TxBench-PP: Analyzing AI Agent Performance on Small-Molecule Preclinical Pharmacology
Artificial intelligence (AI) agents promise to accelerate drug discovery by compressing interpretation and decision-making loops, but practical deployment requires trusted evaluation on realistic program decisions. We introduce TherapeuticsBench Preclinical Pharmacology (TxBench-PP), a verifiable benchmark for small-molecule preclinical pharmacology and the first focused slice of a broader TherapeuticsBench effort across drug-discovery stages and therapeutic modalities. TxBench-PP tests whether agents can recover accurate conclusions from real-world assay data rather than memorized facts from literature. The benchmark contains 100 evaluations indexed by program stage, assay type, and task structure, spanning mechanism-of-action (MoA) and pharmacodynamic (PD) reasoning, compound-target engagement, causal target validation, developability and safety, and translational efficacy. Agents receive realistic workflow snapshots, inspect files in a coding environment, and return structured answers graded deterministically. Across 16 model-harness configurations, comprising 11 models and 4,800 trajectories, no system reliably recovered preclinical pharmacology decisions. The strongest configuration, Claude Opus 4.8 / Pi, passed 59.3\% of endpoint attempts (178/300; 95\% CI, 51.1-67.6), followed by GPT-5.5 / Pi at 55.3\% (166/300; 47.0-63.6).
- @hebbia06-16 05:50原文