#EAServe

1 stories · 0 in 7 days · 0 in 30 days
Stories, products and papers tagged "EAServe", newest first.
Sep 28
PapersBuildersMulti-sourceSelected10:18
EAServe Boosts Multimodal LLM Serving Goodput by 4.3x

Disaggregated serving mostly assumes a two-stage text pipeline, so adding Encode as a regulated control point changes how multimodal workloads should be scheduled and provisioned. Engineers running MLLM inference at scale — whether with vLLM or NVIDIA Dynamo — can compare EAServe's EPD design and its automated configuration search against their current setups when encode GPUs sit idle while prefill and decode starve. The 4.3x and 1.7x goodput figures give a concrete yardstick under identical SLOs.

Event
arXiv cs.LG@Kunxiong Zhu et al. (7)Source