SheetSage-A2S数据集发布,助力流行音乐音频转乐谱

Audio-to-Score Transcription using Pre-trained Features, Data Augmentation, and the New SheetSage-A2S Dataset

精选理由

这篇论文搞了个流行音乐专用的音频转乐谱数据集,用MuQ预训练模型,古典乐错误率从15.3%降到4.98%,还公开了代码,想做的可以直接上手。

AI 摘要

论文发布SheetSage-A2S数据集,含61小时音频、9,468个片段,覆盖6,066首流行歌曲,是首个面向流行音乐的音频到乐谱数据集。研究者采用MuQ预训练特征提取模型与数据增强,提升泛化能力。在古典音乐Quartets集上实现4.98%符号错误率,显著优于现有最优方法的15.3%。在SheetSage-A2S流行音乐数据集上达到20.92%符号错误率,为后续研究提供强基准。数据集、模型和代码已在GitHub公开。

原文 · arXiv cs.AI

Audio-to-Score Transcription using Pre-trained Features, Data Augmentation, and the New SheetSage-A2S Dataset

Existing audio-to-score (A2S) systems primarily focus on classical music, and the application to popular music remains underexplored. This paper first presents the new SheetSage-A2S Dataset, which includes 61 hours of audio with \texttt{**kern} score encodings for 9,468 clips originating from 6,066 unique songs, the first of its kind to facilitate A2S research for popular music. Additionally, we improve on existing A2S approaches by using data augmentation and MuQ, a pretrained feature-extraction model for music audio, to enhance generalisation abilities and extract meaningful audio features. Results show that the proposed A2S model achieves 4.98\% symbol error rate (SER) on the Quartets collection for classical music, which significantly outperforms the 15.3\% SER from the existing state-of-the-art \cite{alfaro-contrerasTransformer2024}. Additionally, our model achieves 20.92\% SER on the SheetSage-A2S dataset for popular music, serving as a strong benchmark for future research. The dataset, model, and code are made publicly available at: https://github.com/Multimodal-Music-Research-Lab/SheetSage2Kern_model.