PECA：单张消费级 GPU 让 1.7B 小模型医学回答缩短 72.8% 且更准
Parameter-Efficient Conciseness Adaptation: Differentiable Length Control for Concise Clinical Reasoning in Smaller Language Models
临床问答最怕啰嗦，这篇论文用单张消费级 GPU 微调 1.7B 小模型，回答砍掉 72.8% 还更准了。不靠奖励模型和偏好数据，靠提高 EOS 概率这一个技巧，微调党可以直接抄作业。
论文提出 Parameter-Efficient Conciseness Adaptation（PECA），在 QLoRA 监督微调目标中加入一个可微的简洁性项，解决小模型在医学任务上回答过长的问题。由于 token 数不可微，PECA 的做法是在生成超过 token 预算时提高序列结束符（EOS）的概率，以平滑信号在普通训练过程中优化，不需要偏好数据、奖励模型或专有教师模型，只需一张消费级 GPU。
在 1.7B 参数模型上用 medical-o1-reasoning-SFT 语料微调后，PECA 将平均回答长度缩短 72.8%，同时把选择题诊断准确率从 40.8% 提高到 44.0%。对简洁性强度做扫描，可以得到一条可解释的准确率-长度前沿，用于权衡简洁与诊断完整性。
与依赖偏好优化、奖励模型或专有教师蒸馏的现有长度控制方法相比，PECA 把门槛降到单卡 QLoRA 微调即可完成，适合在临床决策支持和床旁场景部署低成本的本地化医学模型。
Parameter-Efficient Conciseness Adaptation: Differentiable Length Control for Concise Clinical Reasoning in Smaller Language Models
Smaller language models (SLMs) make it feasible to deploy clinical natural language processing where large proprietary models are too costly, and parameter-efficient fine-tuning allows them to be specialised on commodity hardware. One obstacle persists after training: on medical tasks these models answer at far greater length than the clinical question requires. In clinical decision support and point-of-care use, where time is short, such verbosity is hard to act on even when the underlying reasoning is sound. Existing length-control methods depend on preference optimisation, reward models, or proprietary teachers that undercut the accessibility making SLMs attractive. We introduce Parameter-Efficient Conciseness Adaptation (PECA), which adds a single differentiable conciseness term to the QLoRA supervised fine-tuning objective. Because the token count is discrete and non-differentiable, PECA instead raises the end-of-sequence probability once generation exceeds a token budget, a smooth signal optimised during ordinary training, with no preference data, no auxiliary model, and a single commodity GPU. On a 1.7-billion-parameter model fine-tuned on the medical-o1-reasoning-SFT corpus, PECA reduces mean response length by 72.8% while improving multiple-choice diagnostic accuracy from 40.8% to 44.0%. A sweep over the conciseness strength traces an interpretable accuracy-to-length frontier for balancing brevity and diagnostic completeness.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include PECA, QLoRA, 医学AI, 微调.
Source attribution: curated from arXiv cs.AI (@Tanzeel Abbas, Ahmad Kamran Malik, Nafees Qamar, Sana Liaqat).