Papers

用 Jev 决策模型替代大语言模型做边缘服务编排,延迟降15.9%-26.5%

Replacing Large Language Models with Jev Decision Models for Low-Latency Edge Service Orchestration

Why we picked it

有人拿 Jev 决策模型替掉 DeepSeek 和 Qwen 跑边缘编排,延迟最多降26.5%,API 费用省近七成。

论文将 Jev 的决策导向 API 接入边缘服务编排,通过提取四个有界意图字段并复用共享验证器、准入策略和调度器,取代自然语言模型的决策环节。

Body

对比基线包括结构化输出的 DeepSeek 部署和自托管 Qwen 的真实两节点 OCR 服务。三个连续测量块中,Jev 将客户端决策中位延迟降低 15.

9%-26.5%。

八组配对 OCR 条件下,Jev 在七组中与 DeepSeek 的正确按时完成数持平、一组反超;无缓存时端到端中位延迟低 11.

1%-25.3%,每次正确完成的 API 费用低 69.

0%-70.6%。

重复请求缓存基本消除延迟差异,测试路径中的主要节省机会在于新请求的解释环节。

Original · arXiv: DeepSeek

Replacing Large Language Models with Jev Decision Models for Low-Latency Edge Service Orchestration

Natural-language service requests can require a language-model decision before execution starts, consuming part of the request's latency budget. We integrate Jev's decision-oriented application programming interface (API) into edge service orchestration to reduce this overhead while retaining service completion. The integration extracts four bounded intent fields and applies a shared validator, admission policy, and scheduler, accounting for decision waiting throughout the request timeline. We compare Jev with a short, structured-output DeepSeek deployment using live API measurements followed by modeled execution, and then a real two-node optical character recognition (OCR) service with self-hosted Qwen and rule-based references. Across three consecutive measurement blocks, Jev reduces median client decision latency by 15.9-26.5%. In eight paired OCR conditions, Jev matches DeepSeek's correct, on-time completion count in seven and exceeds it in one. Without caching, median end-to-end latency on requests completed correctly by both systems is 11.1-25.3% lower; API fees per correct completion are 69.0-70.6% lower. Repeated-request caching largely removes the latency difference. The results demonstrate that decision-model substitution can lower both response latency and API fees in the tested service path, and identify fresh interpretation as the main opportunity for latency savings.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Jev, DeepSeek, Qwen, edge computing.

Primary source verification: published directly by arXiv: DeepSeek with first-party authority.