AITOPAI热报
PapersSelected

Salesforce paper on co-evolving models and harnesses

elvis@omarsar0
Post
View the post
Why we picked it

This paper provides a detailed analysis of a common problem in AI development: how to effectively combine model improvements with existing system designs.

A Salesforce paper explores how evolving a model's training harness with a weak model leads to performance drops on enterprise tasks.

Body

The study found that fine-tuning a weaker model on an expert's full trajectories under the same harness caused a 4 to 30 point drop across Qwen3-Coder and Gemma 4.

The analysis suggests that the key factor is the model-harness fit, not just the model's strength.

Original · elvis

Nice paper from Salesforce on co-evolving harnesses and models. Harness engineering is a hot topic right now. So this is a great read. (bookmark it) Salesforce evolved a harness with a weak model across seven enterprise agent tasks, then trained that model on a stronger expert's full trajectories under the same harness. Performance dropped on all seven tasks, by 4 to 30 points across Qwen3-Coder and Gemma 4. The same fine-tuning helps under the unevolved harness. So the harness is what changes the outcome. Their analysis points at model-harness fit. Imitation transfers knowledge and increases scaffold usage, but the weaker model adopts the expert's planning strategy without the competence to execute it, and it no longer matches a harness that was evolved around its own native planning style. The fix is to stop copying whole trajectories. A meta-level agent finds the failing turn in the weaker model's own rollout and asks the expert to rewrite only that turn. That keeps the model's planning style intact and combines the gains from harness evolution and weight updates. Paper: academy.dair.ai/papers/co-evol…📊 8 ⚡

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Salesforce, Qwen3-Coder, Gemma 4, model-harness fit.

Source attribution: curated from elvis (@omarsar0).

#Salesforce#Qwen3-Coder#Gemma 4#model-harness fit#AI research

Keep reading

Recent stories with the same tag or section
PapersBuilders00:49
Chinese contentSalesforce Research 携 16 篇论文参加 NeurIPS 2026

Salesforce Research 这波在 NeurIPS 2026 一口气中了 16 篇论文，主题全是智能体编排、模型自我改进和记忆这些热点难题。做智能体开发的工程师和研究者可以蹲一下推文串，挑评估和记忆那几篇先看，说不定能抄到现成方案。

#NeurIPS#Salesforce#智能体#模型评估
Official accountSalesforce AI Research@SFResearchSource
ModelsBuildersSelected00:31
SOV-Transcribe-1 Tops Open Weights Speech-to-Text Rankings

A 4.6B-parameter open model beating proprietary transcription APIs at 2.3% WER shifts the economics of self-hosted speech workloads, since teams no longer pay a premium for accuracy. Developers building voice agents or call analytics should benchmark it against Whisper-based stacks and hosted APIs on their own audio domains. The small active parameter count also makes GPU deployment affordable for smaller teams and regional SaaS builders.

#SOV-Transcribe-1#Salesforce#speech-to-text#open-source models
Official accountArtificial Analysis@ArtificialAnlysSource
PapersBuildersSelected01:31
GIFT-Eval-CTX Benchmark Tests LLMs on Contextual Forecasting

The non-obvious takeaway is that an LLM layered over a specialised forecasting model beats both pure LLMs and pure statistical models, suggesting a hybrid architecture is the practical path for time-series work. Teams building demand planning, energy load, or capacity forecasting tools can use the open benchmark to validate whether their pipelines actually exploit textual context. Practitioners comparing agent frameworks for forecasting should benchmark MoiraiAgent's pattern of pairing an LLM with code execution and a domain model against their own stack.

#GIFT-Eval-CTX#MoiraiAgent#GPT-5.6 Sol#Claude Opus 4.8
Official accountSalesforce AI Research@SFResearchSource
PapersBuildersSelected00:07
Salesforce paper shows RL experts merged beat external teacher distillation

The finding flips a common assumption that dense teacher feedback is the compute-efficient path, showing the teacher's per-token forward pass overhead usually outweighs its benefit. Teams training domain-specific models can instead run RL per domain and merge experts, cutting compute by roughly 42%. Practitioners weighing teacher choice or FLOP budgets for math and code training get a concrete decision rule.

#Salesforce#distillation#GRPO#reinforcement learning
rohanpaul_ai@rohanpaul_aiSource