这篇论文发了个叫 OmniDecVAEs 的框架，能把几十个传感器数据揉在一起，分类和生成都行，参数才 4.1M，比 Transformer 还准。
arXiv 论文提出 OmniDecVAEs 框架，扩展 DecVAEs 学习模态条件时频潜在子空间，可同时处理分类、解纠缠表示、融合与生成。在多达 30 个模态的人类活动识别任务中，OmniDecVAEs 相比 Transformer 和 VAE 方法，活动识别准确率提升 1.01%，身份识别准确率提升 6.75%。合成数据的平均绝对误差改善 76.84%，最大均值差异改善 13.85%。模型仅 4.1M 参数，适合边缘可穿戴设备。
Omni-modal decomposition autoencoders learn full-stack wearable disentangled representations
Learning disentangled representations is a key requirement for developing versatile, general-purpose, and sustainable models in multi-modal wearable computing. However, existing approaches do not operate as full-stack wearable processors, i.e., they do not simultaneously address task-specific classification performance, disentangled and interpretable representation learning, fusion, and generative modeling of highly heterogeneous multi-modal time series. To address this gap, we introduce Omni-modal Variational Decomposition Autoencoders (OmniDecVAEs), a framework that efficiently learns multi-purpose representations in a unified and scalable manner from arbitrarily many modalities. OmniDecVAEs extend DecVAEs by learning modality-conditioned time-frequency latent subspaces through a multi-view self-supervised decomposition loss and a shared asymmetric autoencoder (AE) architecture. Results on a challenging omni-modal human activity recognition (HAR) setting with up to thirty modalities, demonstrate the ability of OmniDecVAEs to learn full-stack wearable representations. When compared to transformer-based and VAE-based methods, OmniDecVAEs full-stack disentangled representation properties lead to accuracy improvements of 1.01% and 6.75% in activity and identity recognition, respectively. Furthermore, OmniDecVAEs synthesize realistic omni-modal time-frequency data that manifest with enhanced reconstructions (mean absolute error improves by 76.84%) and distributional similarity between real and synthetic data (maximum mean discrepancy improves by 13.85%). Our results highlight OmniDecVAEs potential as a lightweight model suitable for intelligent edge wearables and clinical healthcare, unifying processing requirements and abilities in a single model, through its enhanced representational capacity, modality-invariant spatial complexity (4.1M parameters), and real-time latency.