Latest

Scoring bias in judge models flows directly into training signal and leaderboard rankings, so skewed calibration quietly shapes which outputs get rewarded. Teams using MLLM-as-a-judge pipelines should audit scale usage and split alignment reports by reference culture rather than averaging. Anyone building evaluation or RLHF infrastructure for products serving both Western and Chinese audiences has a concrete methodology to copy here.

Official accountSalesforce AI Research@SFResearchSource

Fixed macro execution works only when the skill matches the task; making granularity adaptive turns brittle agent failures into recoverable ones. Developers building coding agents can apply the macro-to-micro fallback pattern to improve reliability without hand-tuning heuristics. The learned router beating a hand-written heuristic on the 12-task study shows the approach scales beyond scripted workflows.

DAIR.AI@dair_aiSource

Cost and privacy trade-offs in agent execution do not move together — once task completion converges, strategy choice is mostly a privacy decision, and the best-scoring setup is also the most exposed. Teams deploying on-device agents can pick a collaboration mode based on edge model size: route execution to the cloud for small models, use cloud advisory for 9B-class ones. The comparison against Tsinghua-affiliated GLM-5.3 makes the findings directly applicable to Chinese open-source model stacks.

OpenBMB@OpenBMBSource

Sampling multiple answers and diffing them beats asking a model to explain itself — a cheap workflow change that applies to any high-stakes LLM use like research triage or forecasting. The 86x compute saving from the reranker makes it practical to run on everyday tasks. Practitioners building review pipelines or agent self-checks in Singapore and the wider ASEAN developer community can adopt this with any API-based model.

rohanpaul_ai@rohanpaul_aiSource

The result argues that tool selection can be moved off the flagship model entirely: a small RL-trained selector handles catalogs of 10,000+ tools that cannot fit in context, where retrievers alone rank only by relevance. Teams building agents with large tool catalogs can swap in a lightweight selector while keeping their main model unchanged. The performance gap over a prompted 402B model, 41.5% versus 30.6% exact picks, makes the cost saving concrete.

rohanpaul_ai@rohanpaul_aiSource

Verified fitness gates, not raw model skill, are what turn LLM-generated code edits into production-safe optimisations — the correctness pipeline is the real contribution here. Engineering teams maintaining legacy codebases, especially Salesforce Apex and Java shops, can apply the runtime-validated search pattern to automate hot-path optimisation without shipping regressions.

Official accountSalesforce AI Research@SFResearchSource

Most existing engineering benchmarks are textbook QA or toy design problems that leading agents already saturate, so PLE measures the harder step from passing exams to producing work a licensed professional would accept. The 50.6% ceiling plus 27M tokens and 63 minutes per task gives teams concrete cost-versus-capability data for deploying agents on real engineering workloads. Anyone evaluating agent harnesses or building domain-specific benchmarks can reuse its Creator-Solver task-generation pipeline.

DAIR.AI@dair_aiSource

The post captures a design debate inside frontier labs: whether agentic systems should mirror human hierarchies or dissolve into one fused model where 'subagents' are just soft attention regions. Engineers building orchestration frameworks and researchers studying agent scaling laws will find the framing a useful provocation against current agent-architecture orthodoxy. The linked tech report itself is the primary source to consult.

Teortaxes@teortaxesTexSource

Small models trained specifically for tool selection can outperform much larger prompted LLMs on the routing step, so agent builders can cut cost and latency by replacing the proprietary LLM in their tool-selection pipeline. The selector's ability to work with different retrievers and unseen domains makes it practical to slot into existing RAG-style stacks. Teams building large-scale agent tooling should compare this approach before defaulting to a frontier model for routing.

DAIR.AI@dair_aiSource

RSD turns the huge pool of unlabeled agent runs, failures included, into usable training signal without any reward modelling or human annotation. Teams running their own agents can retain completed trajectories and retrain on hindsight reviews, refreshing the reviews each round to avoid the 59.7% versus 67.9% gap seen with stale notes. The comparison against reward-based methods on ALFWorld makes it a practical reference point for anyone choosing a training recipe.

rohanpaul_ai@rohanpaul_aiSource

Interpretability here does not cost accuracy — on out-of-distribution text the transparent version actually beat the black box by 17 F1 points. Teams flagging student essays or peer reviews can audit which cues drive verdicts and remove ones they consider unfair, such as styles correlated with non-native writing. The method also offers a template for making other safety classifiers inspectable.

rohanpaul_ai@rohanpaul_aiSource

Bidding removes the need to know in advance which model handles each question, giving a self-selecting alternative to router systems that degrade badly on outlier prompts. Marketplace and orchestration platform builders can apply a pay-your-bid mechanism when their model pool contains several strong competitors per prompt type. The gap on worst-case clusters (96.0% versus 6.0%) suggests auctions also act as a safety net where fixed routing fails hardest.

rohanpaul_ai@rohanpaul_aiSource

Faithfulness can be fixed at the drafting stage rather than by better retrieval — the writer, not the search, is where most hallucinations enter. Teams building deep research pipelines can plug ClaimWriter into existing frameworks and audit outputs with ClaimProbe using the released code. The benchmark offers a reproducible baseline for measuring claim-level grounding across competing agent stacks.

Official accountSalesforce AI Research@SFResearchSource

The scaffolding, not the model, roughly doubled the improvement — a design lesson for anyone building AI-scientist or agent systems. Builders of research agents can copy the three skill modules (ideation, experiment management, scheduled checklists) and benchmark against Agent4Sci's 20 tasks. The paper also exposes predictable agent failure modes, such as wasted compute and narrow local sweeps, that teams can check for directly.

DAIR.AI@dair_aiSource

The frozen reference model makes agreement checks brittle: if it scores a token lower than both teachers, the teachers count as "agreeing" even when their gap is large, and in-between scores flip the logic. That limitation matters for anyone building multi-teacher distillation pipelines and evaluating whether relative-probability gating or explicit KL divergence between teachers gives a cleaner compatibility signal. Chinese lab output on distillation tricks like this tends to preview what lands in later open-source model training recipes.

Teortaxes@teortaxesTexSource

The dual programmatic-plus-semantic verifier design addresses the hardest part of agent training: rewarding long-horizon tasks without human grading. Teams building enterprise work agents can reuse the recipe to generate their own verifiable task suites from internal workflows rather than hand-crafted benchmarks. It sits alongside agent-environment efforts like task synthesis pipelines from other labs but anchors everything in expert-derived, evidence-backed tasks.

Official account腾讯混元@TencentHunyuanSource