Latest

Training on internalized skill workflows lets a 35B open-weights model beat far larger proprietary models on agentic benchmarks, shifting the edge from raw model scale to curated environment data. Builders running Claude Code or Codex harnesses can reuse the released environments and trajectories for their own SFT or RL pipelines. The results give Qwen-based deployments a concrete benchmark point against Claude Sonnet 4.6 and DeepSeek V4 Pro.

The key finding is that skills can be internalized as model weights rather than injected at inference time, since the fine-tuned agent beats skill-assisted base models even without the skills provided. Agent engineers and fine-tuning practitioners can reuse the released environments and trajectories for SFT or RL with outcome-based rewards. The result also shows a 35B open Qwen-based model outscoring much larger closed models on SkillsBench, a useful cost comparison for teams building coding and tool-use agents.

The key shift is replacing expensive persistent repository graphs and LLM-based exploration with a deterministic, feature-indexed retrieval pipeline, making repo-level context cheap enough for routine use. Teams building coding assistants or agent frameworks can adopt the approach to cut context costs without sacrificing dependency recall. For cost-sensitive deployments, pairing it with DeepSeek-V3.2 gives near-identical DIR@1 to GPT-5-mini at lower inference cost.

The benchmark reframes agent evaluation around deliverable outcomes and payment-gated acceptance, a shift from metric-chasing benchmarks that miss corrupted training runs. Teams building or buying post-training-as-a-service can use the acceptance gate and corruption detector to screen automated fine-tuning pipelines. The human FDE baseline gives a concrete performance reference against all four frontier agents.

This research shows a significant efficiency improvement for coding agents, which could make them more accessible for developers and companies, especially those with limited computational resources.

elvis@omarsar0Source

This research provides a critical methodological framework for evaluating the true impact of safety training on large language models, showing that toxicity scores alone are insufficient to measure real-world harm.

The study reveals how different AI models handle ethical boundaries, which is relevant for anyone developing or using multi-agent systems.

This research explores a conditional path toward quantum scaling for long-horizon reasoning, a key challenge in advanced AI systems.

This paper provides a detailed analysis of how different AI models handle impossible tasks, which is relevant for anyone developing or evaluating AI systems for safety and reliability.

This research provides a deeper understanding of how leading AI models think about their own future architecture, which is relevant for anyone interested in the development of next-generation AI systems.

For developers, this matters because it shows that current LLM-based automated program repair tools can be tricked into fixing bugs in a way that introduces new security vulnerabilities.

This research provides a concrete method to evaluate LLMs' ability to reason dynamically, which is crucial for applications like simulating public opinion where accurate belief updates are needed.

This matters for anyone using LLMs to generate content, as it shows that relying solely on LLM judges can lead to incorrect conclusions about the quality or depth of the output.

This paper provides a detailed analysis of how debiasing methods affect the accuracy of LLMs used as judges, which is important for anyone working with RAG or agent-workflow evaluation.

This matters because it shows that the performance of top AI models on physics tasks is often misreported due to flawed evaluation methods, which could impact how we trust AI for scientific work.

This research provides a critical evaluation of how well large language models can handle complex, multi-step procedures, which is a key capability for many real-world applications.

This research highlights a significant limitation of current multilingual AI models when dealing with low-resource languages like Urdu, showing they often fail to generate culturally relevant and accurate content.

This paper provides a detailed analysis of how different AI models handle version constraint resolution, a critical task for coding assistants. It offers concrete performance data that developers can use to evaluate which model is best for their needs.