复合AI应用开发者终于有了一个无需重写代码就能加速的工具——PopPy自动找出并行机会，做多模型编排或自动化管道的团队值得一试。
复合AI应用（如用Python调用多个ML模型）的端到端延迟成为瓶颈，传统编译器无法优化外部组件调用。PopPy系统通过结合编译时分析和运行时调度，自动识别并并行化Python中调用外部组件的代码段。它解决了Python语言复杂性、动态分发和变量突变三大挑战，在真实复合AI应用中实现最高6.4倍加速。开发者只需少量标注即可获得并行化收益，且保持程序语义不变。
PopPy: Opportunistically Exploiting Parallelism in Python Compound AI Applications
Compound AI applications, which compose calls to ML models using a general-purpose programming language like Python, are widely used for a variety of user-facing tasks, from software engineering to enterprise automation, making their end-to-end latency a critical bottleneck. In contrast to traditional applications, execution time is dominated by the external components, which cannot be handled by traditional language optimization systems, like optimizing compilers. To address this problem, we develop PopPy, a system that can uncover parallelization opportunities in Python applications that invoke these heavy external components, including those used in compound AI applications. PopPy supports a very expressive fragment of Python and requires minimal developer input to uncover parallelism. It combines an ahead-of-time compiler with a runtime, addressing three key challenges in extracting parallelism from Python applications: language complexity, dynamic dispatch, and variable mutation. On a set of real-world compound AI applications, PopPy achieves up to $6.4\times$ speedups in end-to-end execution time compared to standard Python execution while preserving the sequential program semantics.