这个框架在BioASQ 14b比赛里factoid子任务拿了第一,针对不同问题类型用不同推理策略,很实用。
该研究为BioASQ 14b Task B提出了一个问答类型感知的LLM框架,针对yes/no、factoid和list三类问题分别设计推理流程。yes/no问题采用片段打乱与自反思;factoid问题使用全片段输入+思维链上下文学习;list问题采用多智能体协作架构。在官方评估中,该框架在Batch 4的factoid子任务中取得第一名。结果证明结合类型特定推理、集成预测与智能体验证可提升生物医学问答的鲁棒性。
From Voting to Agent Collaboration: Answer-Type-Aware LLM Pipelines for BioASQ 14b
Biomedical question answering requires not only accurate extraction of information from scientific literature but also reliable integration of evidence across multiple documents. This study presents a question-type-specific large language model (LLM) framework for BioASQ 14b Task B, designed to improve answer robustness and evidence grounding in biomedical question answering. Rather than applying a single prompting strategy to all questions, the framework selects different inference procedures for yes/no, factoid, and list questions according to their distinct reasoning and evaluation requirements. For yes/no questions, snippet shuffling and self-reflection are used to reduce sensitivity to evidence ordering and improve decision stability. For factoid questions, full-snippet input is combined with chain-of-thought-based in-context learning to support accurate biomedical entity identification. For list questions, a multi-agent architecture is employed, in which evidence extraction, candidate generation, answer verification, and final aggregation are handled collaboratively. Preliminary experiments on BioASQ 13b were used to identify effective inference strategies for each question type, and the resulting framework was subsequently evaluated in the official BioASQ 14b Task B challenge. In the official evaluation, our framework showed competitive performance across multiple batches and achieved first place in the factoid subtask of Batch 4. These results demonstrate the effectiveness of combining question-type-specific inference, ensemble prediction, and agent-based verification for reliable biomedical question answering.