看这篇论文你就知道,不用微调也能高效提取临床症状。Pythia靠多智能体自己写提示词,敏感度0.76、特异度0.95,比传统词典和BERT都稳,还能本地部署。
Pythia是一个多智能体系统,可自主编写并优化临床概念提取提示词,无需人工工程或微调。在本地运行的开源模型上,Pythia利用开发集的敏感度和特异度选择提示词。与基于词典的方法在72个症状和400份临床笔记上比较,Pythia的平均敏感度为0.76,特异度为0.95;词典则分别为0.82和0.76。对于词典将所有笔记均标记为正的14个概念,Pythia通过要求现在时、患者归因的发现,恢复了平均0.97的特异度。在低于2%患病率的概念上,Pythia的敏感度转移出现0.25的平均差距,而BERT分类器在相同开发集上微调后,敏感度仅为0.23,且低于约5%患病率时降为零。
A Multi-Agent System for Autonomous, Fine-Tuning-Free Clinical Symptom Detection: Development and Validation Study
Clinical notes contain many of the signs and symptoms that bring patients to care, yet this information rarely reaches structured fields. Existing extraction approaches either rely on context-insensitive rules that generate false positives or on supervised models that require substantial fine-tuning. We present Pythia, a multi-agent system that autonomously writes and optimizes extraction prompts for clinical concepts without manual prompt engineering or fine-tuning. Running on a locally hosted open-weights model, Pythia keeps clinical notes on local infrastructure and selects prompts using development-set sensitivity and specificity. We compared Pythia with a curated lexicon across 72 signs and symptoms from 400 clinical notes representing 387 patients. Development (n=300) and validation (n=100) sets were partitioned independently for each concept. Pythia achieved mean sensitivity of 0.76 and specificity of 0.95, compared with 0.82 and 0.76 for the lexicon, and matched or exceeded the lexicon on both metrics for 20 of 62 directly comparable concepts. For 14 concepts where the lexicon labeled every note positive, Pythia recovered mean specificity of 0.97 by requiring a present-tense, patient-attributed finding rather than any textual mention of a term. Specificity transferred from development to validation with minimal degradation across prevalences, whereas sensitivity transfer weakened below 5% prevalence, reaching a mean gap of 0.25 below 2% prevalence. A BERT classifier fine-tuned per concept on the same development set achieved mean sensitivity of 0.23 and collapsed to zero sensitivity for concepts below roughly 5% prevalence. These findings suggest that autonomous, fine-tuning-free prompt optimization can produce symptom extraction prompts that generalize effectively from development to validation while remaining deployable on local infrastructure.