Google DeepMind 发布 AudioFM 自监督音频表征框架
DeepMind 新出的 AudioFM，掩码频谱图自监督训练，12 个音频任务里 10 个拿第一，连心脏杂音识别都比老方法强。
Google DeepMind 研究人员发布 AudioFM，一个面向通用音频表征的自监督学习框架。训练时学生模型看到时间或频率掩码后的频谱图，需预测被掩码 patch 的表征，并从其他录音的候选中选出匹配的 EMA-teacher 全视图目标，同时叠加掩码时间邻近片段的弱监督。在线性探测评测中，AudioFM 在 12 个任务中的 10 个上取得最佳或并列最佳成绩。在医学音频任务上优势明显：心脏杂音分类 F1 超出此前音频 SSL 方法 4.7，睡眠呼吸暂停亚型分类高出 3.2。
Google DeepMind researchers have developed:
AudioFM, a self-supervised framework for general-purpose audio representations.
How it works: a student sees time- or frequency-masked spectrograms, predicts the masked patch representations, and has to pick the matching full-view EMA-teacher target from candidates in other recordings. Plus weaker supervision from masked temporal neighbors.
Results: on linear probing it achieves the best or joint-best performance on 10 of 12 tasks.
Something particularly interesting: beats prior audio SSL on heart murmur (+4.7 F1) & sleep apnea subtype (+3.2 F1) classification! (audio FMs are very important in medical AI!)
