StarHarness:企业环境中的分层搜索进化框架

StarHarness: Evolving Harnesses with Stratified Search for Enterprise Environments

精选理由

StarHarness能够有效提高企业环境中智能体的性能,通过分层搜索和固定模型权重的方式,实现了20-35个百分点的性能提升,非常适合需要优化智能体性能的企业使用。

AI 摘要

StarHarness是一个在固定模型权重的同时进化特定环境智能体框架的工具。它通过分层任务、分离搜索和选择任务,以及保留任务来评估泛化能力。在ITBench SRE、EnterpriseOps-Gym ITSM和AutomationBench Finance上,经过4-12次接受的变化,框架进化提高了基准性能20-35个百分点。这些改进在排除进化任务和跨GPT和Qwen模型家族的迁移中持续存在。跟踪分析将改进归因于接口修复、环境约定和压缩搜索的操作知识,减少了误诊和缩短了轨迹。因此,StarHarness为减少工具丰富的企业任务中的持续模型-环境不匹配提供了实用方法。

原文 · arXiv cs.AI

StarHarness: Evolving Harnesses with Stratified Search for Enterprise Environments

We present StarHarness, a framework for evolving environment-specific agent harnesses while keeping model weights fixed. The evolved harness can include prompt and task framing, tool interfaces, skills, MCP-backed providers, subagent structure, and agent-loop configuration. StarHarness constructs a compact evolution pool by stratifying tasks according to baseline failure behavior, separates proposer-visible search tasks from proposer-hidden selection tasks, and reserves held-out tasks for evaluating generalization. Across ITBench SRE, EnterpriseOps-Gym ITSM, and AutomationBench Finance, harness evolution improves full-benchmark performance by 20-35 percentage points over the default harness after 4-12 accepted changes per environment. These gains persist on tasks excluded from evolution and transfer without re-evolution across GPT and Qwen model families. Trace analysis links the improvements to interface repairs, environment conventions, and operational knowledge that compresses search, with fewer false-positive diagnoses and shorter trajectories in several settings. StarHarness therefore offers a practical way to reduce persistent model-environment mismatch in tool-rich enterprise tasks.