PyTorch Conference talk on MoE inference on ARM CPUs
Running MoE models on commodity ARM hardware rather than GPUs can cut inference costs for edge and on-premise deployments, and this session pairs two inference stacks, vLLM and OpenVINO, with hands-on strategies. Engineers deploying LLMs on ARM servers or devices, including the growing ARM server fleet in Southeast Asian data centres, can pick up concrete optimisation techniques from it.
A PyTorch Conference North America session will cover efficient Mixture of Experts LLM inference on ARM CPUs.
Maajid Khan from Fujitsu Research India will present practical strategies using vLLM and OpenVINO.The conference takes place in San Jose on 20-21 October.
Registration is open through the official PyTorch channels.
Interested in optimizing Mixture of Experts LLM inference on ARM CPUs?
In his talk at PyTorch Conference North America, Maajid Khan from Fujistu Research India will explore efficient MoE LLM inference using vLLM and OpenVINO, sharing practical strategies for running these models effectively on ARM architectures.
Register and join us in San Jose, October 20-21: https://t.co/jBApW8nESi
#PyTorchCon
This report is categorized under AITOP's "How-to" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include PyTorch, vLLM, OpenVINO, Mixture of Experts.
Source attribution: curated from PyTorch (@PyTorch).
