MyoMechanix通过多模态感知和结构化表示，在技能活动理解和指导方面取得了突破，为健身、康复、医疗和机器学习领域的物理AI应用提供了新的解决方案。
MyoMechanix是一个多模态生态系统，用于重量负荷动作，将运动与肌肉活动对齐。包含7500多个样本，包括多视图RGB视频、3D姿态、sEMG和生理信号。构建了健身知识图谱（FKG），用于组织动作、阶段、关键步骤、错误和纠正反馈的结构化关系。开发CUBIST，用于细粒度错误归因和反馈生成。实验表明，多模态感知和结构化表示提高了性能、可解释性和错误归因，CUBIST达到最先进的结果；VideoQA增强了基于语言的动作理解；Video2EMG提出了基于视频的EMG传感替代方案。
MyoMechanix: Biomechanically-Grounded Compositional Skilled Activity Understanding and Coaching
Existing action quality assessment (AQA) datasets and methods rely primarily on visual inputs such as RGB and pose, overlooking physiological dynamics such as muscle mechanics and often modeling actions as monolithic patterns. These limitations hinder fine-grained, biomechanically grounded feedback. We introduce MyoMechanix, a multimodal ecosystem for weight-loaded actions that aligns motion with muscle activity. Expert-annotated, it contains 7,500+ samples of 20 actions from 38 subjects, with synchronized multiview RGB video, 3D pose, sEMG, and additional physiological signals, forming the largest multimodal AQA benchmark to date. We further construct the Fitness Knowledge Graph (FKG), which organizes expert annotations into structured relationships among actions, phases, key steps, errors, and corrective feedback, enabling compositional scoring and interpretable assessment. Building on these representations, we develop CUBIST (Compositional Ontological Reasoning Engine), which performs decomposition-analysis-recomposition for fine-grained error attribution and feedback generation. We also establish MyoMechanix-AQA, MyoMechanix-VideoQA, and a novel MyoMechanix-Video2EMG task. Experiments show that multimodal sensing and structured representations improve performance, interpretability, and error attribution, with CUBIST achieving state-of-the-art results; VideoQA enhances language-grounded action understanding; and Video2EMG suggests video-based alternatives to costly EMG sensing. MyoMechanix advances skilled activity understanding toward biomechanically grounded, multimodal, and compositional reasoning for Physical AI applications in fitness, rehabilitation, healthcare, and machine learning. Project page: https://haoyin116.github.io/MyoMechanix/