AITOPAI热报
PapersSelected

Meta and Michigan State paper finds LLM-built reasoning graphs unreliable

rohanpaul_ai@rohanpaul_ai
Post
View the post
Why we picked it

The non-obvious point is that most reasoning-graph pipelines assume the LLM's decomposition is correct without ever verifying it, so confidence scores inherit hidden graph errors. Builders of agent workflows and RAG-style decomposition systems should add graph-level validation before trusting computed confidence. The Qwen3-32B versus Qwen3-8B result is a concrete warning that scaling a model does not guarantee better sub-question answering.

A paper from Meta and Michigan State University examines methods that split a question into a graph of sub-questions and compute answers or confidence scores on top. The authors find that the LLM-generated graphs themselves vary widely in quality, and their errors propagate into downstream confidence scores. They tested 9 models under varying levels of assistance, from wiring pre-given steps to building the entire graph. Graph quality tracked model family and graph shape more than parameter size, and Qwen3-32B answered sub-questions worse than Qwen3-8B.

Original · rohanpaul_ai

New Meta + Michigan State Univ paper finds that LLM-built reasoning graphs vary widely in quality, and their errors carry into the confidence scores computed from them.

Many methods split a question into a graph of sub-questions, then compute answers or confidence on top. They assume the LLM's graph is correct, which is rarely checked.

They tested 9 models with less and less help, from wiring given steps to building the whole graph. Quality tracked model family and graph shape more than size, and Qwen3-32B answered sub-questions worse than Qwen3-8B.

If you build on LLM-made reasoning graphs, check the graph itself before trusting answers or confidence computed from it.

#Qwen3#Meta#reasoning models#AI safety

Keep reading

Recent stories with the same tag or section
How-toBuildersOfficialSelected04:08
AWS shows three ways to train Qwen3 on Trainium with torchtitan

The notable point is that one config setup now covers three different training backends on Trainium, so teams can switch between them without rewriting launch scripts or breaking checkpoint compatibility. Engineers choosing between NVIDIA GPUs and AWS silicon can use this to evaluate Trainium for fine-tuning open-source models like Qwen3. The guide also shows how NKI MoE kernels fit into the same workflow.

#Qwen3#AWS Trainium#torchtitan#fine-tuning
Event
官方一手AWS Machine Learning Blog@Josh Longenecker4 sourcesSource
PapersBuildersSelected00:20
Tencent paper details recursive self-improvement for terminal agents

The reward-design audit is the most transferable part: dense progress rewards let failed runs harvest half the reward, so teams training RL agents should measure how much reward failure earns before trusting their curves. RL and agent-training engineers can apply the strict-success-versus-partial-credit split to their own reward pipelines. Compared with generic terminal-agent RL reports, this one pairs benchmark gains with a concrete failure-mode analysis.

#Tencent#Qwen3#Terminal-Bench#PPO
elvis@omarsar0Source
How-toBuildersSelected21:27
Reasoning From Scratch round 8 covers distillation with Qwen3-0.6B

Distilling reasoning traces from a larger teacher into a 0.6B-parameter model shows practitioners a far cheaper path than RL training runs, with MATH-500 as the benchmark to verify the gains. Builders training small models on limited compute can reuse the standalone training script and data-preparation pipeline directly. The soft vs. hard and on-policy vs. off-policy distillation framing is a useful mental model when choosing a distillation approach.

#Qwen3#DeepSeek R1#distillation#reasoning models
官方账号Sebastian Raschka@rasbtSource
ModelsBuildersMulti-sourceSelected15:31
Hojo-ASR-Multi-V1 posts 3.54 average WER across 5 languages

Replacing a dedicated speech decoder with a general-purpose 4B LLM means recognition inherits stronger word-context modelling, which helps resolve acoustically ambiguous words — an architectural shift other ASR builders may follow. Developers building multilingual voice features can run it locally today under Apache 2.0, inspecting or fine-tuning it without API costs, and benchmark it against Whisper-class models on their own audio.

#Hojo-ASR-Multi-V1#Qwen3-4B#ASR#open-source models
Event
rohanpaul_ai@rohanpaul_ai3 sourcesSource
PapersBuildersSelected12:49
Behavioral Self-Rewarding Lifts Qwen3-14B on SWE-bench

The method requires no new environment interactions, so teams can layer it onto existing reward setups without extra rollout cost. Researchers and engineers training coding agents on sparse test-based rewards get a concrete fix for zero-variance groups, with the bounded-reward design showing how to keep self-judgement from overriding ground-truth correctness. It compares against standard GRPO-style binary reward training on Qwen3-14B.

#Qwen3#SWE-bench#Microsoft Research#reinforcement learning
DAIR.AI@dair_aiSource
PapersBuildersSelected09:22
Primus Projection Cuts Distributed LLM Training Tuning to Single-Node Simulation

Teams burning GPU-hours on OOM crashes and suboptimal parallelism recipes can instead score candidate configurations in seconds on one machine, then commit only to the promising ones. The paper's validated error rates at production scale — including on DeepSeek V3 and Qwen3 — make the projections credible for real deployments. Engineers planning training runs on 256 to 1,024 GPU clusters, or on hardware not yet available, get a practical tuning workflow they can adopt directly.

#Primus Projection#distributed training#DeepSeek#Qwen3
arXiv: DeepSeek@Anshu Raina et al. (10)Source