Claude Sonnet 5.5 and GPT-6 Astra tie on MLCR-AA leaderboard
The split matters more than the tie: Anthropic models dominate completeness, with Claude Sonnet 5.5 at 64.8% versus 42.2% for the best non-Anthropic model, while OpenAI models dominate accuracy, led by GPT-6.1 Sol at 96.7%. Teams deploying models for medical record review should pick based on which failure mode they can tolerate. Even top models still fail more than half of the benchmark's questions, so human review remains necessary.
Artificial Analysis updated MLCR-AA, its leaderboard for Wisedocs' Medical Long Context Reasoning benchmark, changing prompts and grading to reflect real medical record review workflows. Answers are now graded by a three-model judge panel, and all 60 expert reference answers were reviewed and 15 refined. Claude Sonnet 5.5 (max with fallback) and GPT-6 Astra (max) are tied for #1 at 41.7%, with GPT-6.1 Sol (max) at 38.3%. Qwen3.8 2.4T A95B from Alibaba is the top open-weights model at 30.6%, and no model passes more than 42% of questions.
We've updated MLCR-AA, our leaderboard for Wisedocs' Medical Long Context Reasoning benchmark, with Claude Sonnet 5.5 and GPT-6 Astra now tied for the #1 score
We've updated how MLCR-AA prompts and grades models on @wisedocsai's benchmark, to better reflect how medical record review is done in practice. Models are now prompted as medical record reviewers, and asked to answer each part of a question in one paragraph with their conclusion and the supporting evidence from the record. Answers are graded by a new three-model judge panel, and we reviewed all 60 expert reference answers against their source records, refining 15.
Claude Sonnet 5.5 (max with fallback) from @AnthropicAI and GPT-6 Astra (max) from @OpenAI are tied at the top with 41.7%, followed by GPT-6.1 Sol (max) at 38.3%. Qwen3.8 2.4T A95B from @Alibaba_Qwen is the leading open weights model at 30.6%.
Key results from the updated MLCR-AA:
➤ Medical record review remains far from solved: an answer passes only when it is both complete and accurate, and no model passes more than 42% of questions and the median model passes under 15%
➤ Anthropic models lead on completeness, reaching the expert's conclusion with the right medical reasoning: Claude models hold the top 18 completeness scores across every model and effort level we tested, led by Claude Sonnet 5.5 (max with fallback) at 64.8%. The best non-Anthropic model is GPT-6 Astra (max) at 42.2%
➤ OpenAI models lead on accuracy, citing the record without exaggeration or hallucination: OpenAI models hold the top 27 accuracy scores, led by GPT-6.1 Sol (max) at 96.7%, but answer fewer questions completely: GPT-6.1 Sol (max) is complete on 40.0% of questions
We would like to thank Wisedocs for building MLCR and for their collaboration in bringing it to Artificial Analysis
