FlowMaker：联合优化 LLM 工作流部署，goodput 提升 10.14 倍

核心设计是约束引导的负载感知剪枝算法，可在运行时根据当前集群负载动态决策。相比 Murakkab 等最先进基线，goodput 最高提升 10.

14 倍，成本最高降低 2.77 倍。

FlowMaker: From Tasks to Cost-Efficient Generative AI Workflow Deployments

Generative AI workflows execute user tasks by composing sequences of LLM calls, where each call performs a sub-task (e.g., summarization or translation), forming a workflow. Executing such workflows requires decisions over workflow structure, model assignments, and deployment configurations, which jointly determine latency, cost, and accuracy under dynamic workloads and resource constraints. Existing systems optimize these decisions in isolation or fix them early, leading to suboptimal cost and goodput. We make two key observations. First, efficient serving requires jointly exploring all combinations of workflows, model choices, and deployment configurations under dynamic cluster state. Second, reuse of deployed LLMs is critical for reducing cost, but must be performed on-the-fly, as reuse depends on current load and resource availability. Together, these make selecting a workflow-variant-a combination of workflow, models, and deployment configuration-a time-critical problem. We present FlowMaker, which exposes a task-level interface and selects workflow-variants on behalf of users. To realize this, we design a constraint-guided load-aware pruning algorithm for fast, on-the-fly decisions. FlowMaker improves goodput by up to 10.14$\times$ and reduces cost by up to 2.77$\times$ compared to state-of-the-art baselines, including Murakkab.

This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include FlowMaker, LLM, 推理服务, 工作流.

Source attribution: curated from arXiv cs.LG (@Dimitrios Liakopoulos, Prasoon Sinha, Neeraja J. Yadwadkar).