MusicTwins 框架让 Audio Flamingo Next 学会像特定个人一样听音乐
MusicTwins: Training Audio-Language Models to Listen like Their Persona
NVIDIA 把 Audio Flamingo Next 微调成能模仿具体真人的音乐品味和说话风格，10 个 persona 测试有效，做个性化 AI 的可以看看。
NVIDIA 研究团队提出 MusicTwins 框架，通过 persona 摘要、个人记忆和后训练生成特定个人的音乐反应。该框架从人类写作中构建 persona 表示，涵盖背景、观点、性格、写作风格和音乐偏好档案。团队对 Audio Flamingo Next 进行监督微调和强化学习后训练。在 10 个保留 persona 上的评估显示，模型在人格保真度、写作风格和音乐反应方面均有明显提升。
MusicTwins: Training Audio-Language Models to Listen like Their Persona
People differ in how they perceive music, what associations they make with it, what they like about it, and how they respond to it. Computational models intended to simulate people in environments that include music should account for these differences. Although LLMs have been shown to reproduce aspects of human behavior, whether audio-language models (ALMs) can reproduce a specific person's responses to textual and musical stimuli remains underexplored. We introduce MusicTwins, a framework that combines persona summaries, personal memory, and post-training to generate person-specific responses. We construct persona representations from human writings, capturing background, views, personality, writing style, and a musical profile, and post-train Audio Flamingo Next through supervised fine-tuning and reinforcement learning. Evaluation on ten held-out personas shows notable improvements for Audio Flamingo Next in persona fidelity, including personality, writing style, and responses to music, supporting MusicTwins as a foundation for training audio-language models to listen like their persona.