PapersSelected11月6日 00:55

Max Planck Paper Finds Reasoning Makes LLMs Share Mistakes

rohanpaul_ai@rohanpaul_ai

Post

Why we picked it

The practical implication is that diversity between models shrinks once reasoning is switched on, so ensemble-style switching between LLMs is less reliable for high-stakes, people-facing decisions. Teams building multi-model pipelines should audit whether any model in their set answers correctly, rather than assuming errors are independent. The census-based task design also offers a reusable template for measuring correlated failure across model sets.

A Max Planck paper examines whether using a different LLM can fix another model's errors when judging people. The authors tested 9 Qwen and GPT models on 3 prediction tasks built from US census surveys, with reasoning on and off. With reasoning enabled, models agreed more, and the share of people all 9 models got wrong rose by 2.0 to 2.4 points. For individuals who triggered the longest reasoning traces, adjusted agreement fell to about 0.31 to 0.39.

Original · rohanpaul_ai

New Max Planck paper finds that turning on reasoning makes different LLMs agree more, even on mistakes, while long traces mark where they disagree.

When several LLMs judge the same person, a wrong answer from one model could be fixed by switching to another. That only works if the models do not all make the same mistakes.

The authors tested 9 Qwen and GPT models on 3 prediction tasks built from US census surveys, with reasoning on and off. They measured agreement beyond what each model's accuracy alone would predict.

With reasoning on, models agreed more, and the share of people all 9 models got wrong rose by 2.0 to 2.4 points. For people who drew the longest traces, adjusted agreement fell to about 0.31 to 0.39.

For decisions about people, check whether anyone gets a correct answer from your model set, since more agreement can also mean more shared errors.

#Qwen #GPT #reasoning models #AI safety

Keep reading

Recent stories with the same tag or section

Papers BuildersSelected23:47

Recursa Paper Shows RSI Lifts Model Performance Without Fine-Tuning

The non-obvious takeaway is that domain adaptation can be achieved through a validated skill repository rather than weight updates, making improvement cycles cheaper and reversible. Teams adapting agents to specialised domains such as law can try the repository approach before investing in fine-tuning or expert annotation. The code and paper are on arXiv, with a companion walkthrough at academy.dair.ai.

#Recursa #DeepSeek-V4.1-Flash #Qwen3.8-Flash #reasoning models

elvis@omarsar0Source

Papers Builders14:02

Chinese contentZETA 架构将路由策略与视觉执行分离，提升科学图像分析智能体可审计性

一篇很硬核的智能体架构论文：把路由和执行拆开，Qwen 3.8 的分割路由从 6.2% 拉到 75%，做科学图像自动化的值得看看思路。

#ZETA #Qwen #智能体 #RAG

arXiv cs.AI@Shubhabrata Mukherjee et al. (3)Source

Papers Builders14:02

Chinese contentTailSteer：面向多轮图像编辑的抗漂移轨迹末端训练框架

多轮改图容易越改越跑偏，TailSteer 只在去噪末端做修正，FLUX、Qwen 上验证有效，做图像编辑的可以看看这篇。

#TailSteer #FLUX #Qwen #图像编辑

arXiv cs.AI@Ruikang Li et al. (9)Source

Papers BuildersMulti-sourceSelected10:33

Tsinghua and Berkeley open-source uni-robot-agent framework

The key insight is that framework design, not model training, drives most of the zero-shot gain: reliable tool feedback and consistent interfaces lifted the same model over 16x on RoboDojo. The five-layer stack plus six reliability rules (snapshot binding, queue flush, pose-error reporting) is a practical blueprint for anyone building real-robot or simulation agents. Simulation and real-machine interaction logs can be combined for later training, and the code is open-source on Hugging Face.

#uni-robot-agent #Qwen #DeepSeek #open-source models

Event

shao__meng@shao__meng11 sourcesSource

Papers BuildersSelected01:19

RSI-Craft paper trains models to run their own post-training

Turning papers into executable post-training environments is a concrete step toward models that improve themselves, and the benchmark parity with GLM-5.2 and GPT 5.5 shows the approach already competes with frontier results. Practitioners fine-tuning smaller open models can study the 281-task pipeline and trajectory filtering method to replicate the gains. The released Qwen-based checkpoints make the results directly testable.

#RSI-Craft #Qwen #PostTrainBench #fine-tuning

DAIR.AI@dair_aiSource

Models BuildersSelected01:13

Zyphra's ZHA hierarchical attention speeds up long-context models

The plug-in design means no retraining of the base model, so teams running open-source models could adopt ZHA by training or downloading a small index module. It matters for anyone serving long-context workloads, where KV-cache reads dominate cost and latency. Compared with MSA, the reported 12.7-point retrieval gain is the concrete benchmark to watch when evaluating sparse attention options.

#Zyphra #ZHA #Qwen #long-context

Gorden Sun@Gorden_SunSource