MMAS 开源模型实现逐 0.5 秒音乐情绪连续估计
Concordance-Optimal Continuous Music Emotion Estimation: Variance Decomposition, Dual-Branch Temporal Fusion, and Conformal Prediction Intervals
有人开源了能逐 0.5 秒追踪歌内情绪变化的 MMAS，还顺手指出业界指标其实测不出歌内动态，做音乐分析的研究者可以看看。
论文提出 MMAS（Multimodal Music Analysis System），从原始音频每 0.5 秒预测 Arousal 和 Valence。
作者将标注方差分解为歌曲间与歌曲内两部分，指出常用的池化帧级相关系数主要被歌曲间差异主导，并引入窗口化 Lin's CCC 来聚焦歌曲内动态。在 DEAM 的 1,764 首歌上调节权重 λ 从 0 到 1，池化 Arousal r 从 0.
843 降到 0.810，但歌曲内平均 r 从 0.
204 提升到 0.368。
MMAS 融合 MERT-v1-95M 嵌入与十个符号特征，在 DEAM 和 PMEmo 共 2,531 首歌上联合训练，池化 r 达 0.801/0.
685（DEAM）和 0.742/0.
746（PMEmo），歌曲内 Arousal r 相比 ridge 基线约三倍（0.344 vs 0.
125）。模型以 pip 包形式开源，split-conformal 区间在 95% 目标下达到 93.
7%（Arousal）和 90.0%（Valence）覆盖率。
Concordance-Optimal Continuous Music Emotion Estimation: Variance Decomposition, Dual-Branch Temporal Fusion, and Conformal Prediction Intervals
Continuous music emotion recognition (MER) estimates how Arousal and Valence evolve within a song, yet the field's standard metric, pooled frame-level correlation, is dominated by between-song differences and cannot distinguish a model that tracks emotional dynamics from one that only ranks songs by their average level. We decompose annotation variance into between- and within-song components and show that mean squared error (MSE) targets the former, while windowed Lin's concordance correlation coefficient (CCC = $r \cdot C_b$) targets the latter by penalising the variance compression MSE induces. A single weight $λ$ controls the trade-off: on 1,764 DEAM songs, raising $λ$ from 0 to 1 lowers pooled Arousal $r$ from 0.843 to 0.810 while raising mean within-song $r$ from 0.204 to 0.368. Building on this objective, we present MMAS (Multimodal Music Analysis System), an open-source model that predicts Arousal and Valence every 0.5 s from raw audio. It fuses frozen MERT-v1-95M embeddings with ten symbolic features from automatic transcription through a dual-branch temporal convolutional network with cross-modal attention. Because both branches are audio-derived, one model trains jointly on DEAM and PMEmo without domain conditioning. Under song-grouped 5-fold cross-validation (2,531 songs), pooled $r$ reaches 0.801/0.685 (Arousal/Valence) on DEAM and 0.742/0.746 on PMEmo. Against a ridge baseline on identical features, within-song Arousal $r$ roughly triples (0.344 vs. 0.125; $n = 353$ songs, Wilcoxon $p = 1.7 \times 10^{-21}$, $d = 0.56$). Complete transcription failure costs at most 0.04 pooled $r$. Split-conformal intervals attain 93.7% (Arousal) and 90.0% (Valence) marginal coverage at a 95% target, with the shortfall traced to within-song dependence. MMAS is released as a pip-installable package.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include MMAS, MERT, 音乐情绪识别, conformal prediction.
Source attribution: curated from arXiv cs.LG (@Mohammad Faraz Akhtar, Aditya Parashar, Ayush Jain, Madhur Gupta, Ritesh Kumar, Soubhik Chakraborty).