EngineAgent：用智能体自动化端侧 LLM 推理引擎的模型适配
EngineAgent: Agentizing On-Device LLM Inference Engines with Inference Stack Hierarchy Awareness and Hardware Abilities
端侧适配新模型要在转换、图构建、算子、后端好几层同时改，普通编码 agent 搞不定。这篇论文的 builder+auditor 双角色设计把成功率从 33.3% 拉到 75%，做端侧部署的工程师值得细看实现思路。
论文提出 EngineAgent，一个自动化端侧 LLM 模型适配的智能体系统，在 12 个任务（3 个移动端 LLM、4 组引擎-后端组合，覆盖 CPU/GPU/NPU）上实现 75% 的端到端成功率，而最强编码智能体基线仅 33.3%。
核心机制是把可复用的引擎与硬件知识从模型特定实现中分离：适配前先构建感知引擎抽象层级与跨层依赖的引擎索引，并用已支持模型预验证构建、执行、调试流程；适配新模型时由 builder 协调转换、图构建、算子、硬件后端等跨层改动，auditor 负责完整性检查并通过设备端验证驱动迭代调试。
成功标准要求编译通过、设备上可执行且输出 logit 与参考实现一致，同时平均单任务耗时和模型 API 成本低于同底座模型的基线，对做端侧部署的工程团队有直接参考价值。
EngineAgent: Agentizing On-Device LLM Inference Engines with Inference Stack Hierarchy Awareness and Hardware Abilities
Rapidly evolving LLM architectures and heterogeneous mobile hardware make supporting new models in on-device inference engines a recurring and costly system challenge. Each model may require coordinated changes across conversion, graph construction, operators, and hardware backends. Existing coding agents often miss these cross-layer dependencies and struggle to diagnose failures across host--device execution paths. We present EngineAgent, an agentic system for automating on-device LLM model enablement. Its key insight is to separate reusable engine and hardware knowledge from model-specific implementation. Before enablement, EngineAgent constructs a hierarchy-aware engine index that captures engine abstraction layers and cross-layer dependencies, and pre-validates backend-specific build, execution, and debugging workflows using already supported models. For each new model, a builder uses this knowledge to coordinate cross-layer changes, while an auditor checks implementation completeness and drives iterative debugging through device-side validation. Across 12 tasks spanning three mobile LLMs and four engine--backend pairs on CPUs, GPUs, and NPUs, EngineAgent achieves 75% end-to-end success with either of two underlying LLMs, versus 33.3% for the strongest coding-agent baseline. Success requires compilation, on-device execution, and output-logit agreement with reference implementations. EngineAgent also reduces average per-task time and model-API cost relative to baselines using the same underlying LLMs.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include EngineAgent, 端侧推理, 智能体, LLM 推理引擎.
Source attribution: curated from arXiv cs.AI (@Zhengxiang Huang, Yujie Sun, Shengheng Chen, Zhaode Wang, Chaoyue Niu, Zeyu Zhao, Chengfei Lv, Fan Wu, Guihai Chen).