医疗AI的幻觉问题直接关系到患者安全，Med-HEAL给出了可落地的缓解方案——做临床NLP或医疗AI部署的团队，可以直接用其公开数据集和代码来评估和优化自己的模型。
医疗大语言模型在临床决策支持中常出现幻觉，带来严重风险。现有基准缺乏真实临床背景，且对缓解幻觉的指导有限。Med-HEAL框架基于EHRNoteQA基准和MIMIC-IV数据，构建了幻觉数据集，通过LLM-as-a-Judge和人工审核双重标注。研究测试了自我批评和检索增强上下文学习两种策略，在五个开源模型上，自我批评策略显著提升了其中三个模型的准确性。该框架提供了可复用的数据集和实用方法，有助于医疗AI的安全部署。
Med-HEAL: Analyzing and Mitigating Hallucinations in Medical LLMs with Hallucination-Aware In-Context Learning
Hallucinations in medical large language models (LLMs) pose serious risks for clinical decision support, particularly when models must reason over complex electronic health records (EHRs). However, existing benchmarks often lack a realistic clinical context and provide limited insight into how hallucinations can be mitigated in practice. We introduce Med-HEAL, a framework for systematically identifying, analyzing, and mitigating hallucinations in medical LLMs using clinically grounded data. Building on the EHRNoteQA benchmark derived from MIMIC-IV discharge summaries, we construct a hallucination dataset by evaluating BioMistral-7B on open-ended clinical question answering tasks. Model outputs are labeled through a dual evaluation pipeline that combines LLM-as-a-Judge assessment (GPT-4o) with human auditing by medical student reviewers, producing correctness judgments and annotations of reasoning errors via a custom web-based evaluation system. We then leverage this dataset to investigate mitigation strategies: a self-critique pipeline, in which the test model reviews its own answers to detect potential errors and regenerates responses for flagged cases, and retrieval-augmented in-context learning (RA-ICL), which exposes the model to hallucinated and corrected examples. Experiments across five open-source LLMs-BioMistral, Llama-3.1, DeepSeek, Qwen2.5, and Qwen3, show that the self-critique strategy improves accuracy for three of five models (p < 0.05) without requiring parameter updates. Med-HEAL provides both a reusable hallucination dataset and a practical framework for studying and mitigating hallucinations in medical LLMs, supporting safer deployment of AI systems in clinical environments. Our code and data are publicly available at https://github.com/yimingliao-blad/med-heal.git.