FlowEdit:联想记忆用于流匹配TTS终身发音适应

FlowEdit: Associative Memory for Lifelong Pronunciation Adaptation in Flow-Matching TTS

精选理由

想给TTS模型随时纠正陌生名词发音?FlowEdit用Hopfield网络存记忆,错误率直降92.7%,不用重训练。

AI 摘要

流匹配TTS在部署后无法纠正专有名词的发音错误。FlowEdit通过潜在条件编辑而非权重更新来实现终身适应,并用现代Hopfield网络存储纠正。在涵盖18个语系、312个多语言专有名词的基准上,FlowEdit将目标词音素错误率相对降低92.7%,且通用语音质量不变。每次纠正仅需约15秒(单GPU)。

原文 · arXiv cs.AI

FlowEdit: Associative Memory for Lifelong Pronunciation Adaptation in Flow-Matching TTS

Flow-matching text-to-speech systems achieve remarkable zero-shot quality but remain static after deployment: pronunciation errors on out-of-vocabulary proper nouns persist unless the model is retrained. We introduce FlowEdit, a life-long adaptation framework for frozen flow-matching TTS that learns pronunciation corrections as latent conditioning edits rather than weight updates. When corrective feedback is provided, FlowEdit optimizes a token-level perturbation in the text embedding space, then stores the correction in a Modern Hopfield Network serving as content-addressable episodic memory. At inference, corrections are retrieved via soft attention with a similarity gate, enabling fuzzy morphological matching. On our curated benchmark of 312 multilingual proper nouns across 18 language families, FlowEdit reduces target-word Phoneme Error Rate by 92.7% relative to the zero-shot baseline while maintaining identical general-speech quality. Corrections complete in approximately 15 seconds on a single GPU.