做模型服务系统或推理优化的团队，FMplex 的虚拟化思路能直接帮你省显存、降延迟，建议研究其调度和共享机制。
现有大模型服务系统将每个定制任务部署为独立模型实例，导致骨干模型重复部署、加速器内存浪费，且无法分摊批处理和加载成本。FMplex 提出一种新范式，将大模型骨干视为虚拟化基础，为每个任务提供逻辑上私有的虚拟大模型（vFM），实际共享同一物理骨干。该方案支持任务独立扩展、生命周期和隔离性，并设计了批感知公平队列调度器，实现任务间和任务内的批处理。在 7 个骨干模型（16 个变体）和 92 个下游任务上，FMplex 相比空间分区降低延迟 80%，相比尽力共置降低 33.3%，集群规模下可承载多达 6 倍的任务数。
FMplex: Model Virtualization for Serving Extensible Foundation Models
Foundation models (FMs) are increasingly used as backbones for downstream tasks across language, vision, time-series, and multimodal applications. Yet existing model-serving systems deploy each customized task as an independent model instance, thereby replicating heavyweight backbones, wasting accelerator memory, and losing opportunities to amortize batching and loading costs. This paper presents FMplex, a serving system that treats FM backbones as a virtualization substrate for deployment sharing. FMplex presents each task with a virtual foundation model (vFM), a logically private FM instance backed by a shared physical FM. This abstraction lets independently customized tasks share a backbone while preserving task-specific extensions, independent lifecycles, and task-level isolation. In addition, we propose a batch-aware fair-queueing scheduler that combines weighted task-level sharing with inter- and intra-task batching across colocated tasks. We implement a FMplex-based serving stack spanning task construction, sharing-aware deployment, and runtime execution. Across 7 FM backbones (16 variants) and 92 downstream tasks, FMplex reduces latency by up to 80% over spatial partitioning and 33.3% over best-effort co-location, while hosting up to 6x more tasks at cluster scale.