论文精选

数据合成+PEFT助力低资源NMT:Q'eqchi'玛雅语案例研究

Data Synthesis and Parameter-Efficient Fine-Tuning for Low-Resource NMT: A Case Study on Q'eqchi' Mayan

精选理由

低资源语言NMT研究者终于有了一个不依赖网络爬虫的可行方案——用社区词典合成数据+LoRA微调就能启动翻译模型,做濒危语言数字化的团队值得关注。

AI 摘要

该研究针对数字低资源土著语言的神经机器翻译(NMT)数据极度稀缺问题,提出了一种无需抓取目标语言平行文本的数据合成方法。以Q'eqchi'玛雅语为例,研究者将社区词典转化为大规模合成语料,并采用LoRA适配器对mT5-base模型进行参数高效微调(PEFT)。域内评估显示模型能有效学习复杂黏着形态和VOS语序(BLEU 42.02),但面对自然语言时存在结构-语义鸿沟(BLEU 0.59),模型过度拟合合成模板的结构约束。消融实验表明多任务学习导致负迁移,LoRA适配器参数容量有限,辅助任务与主任务竞争。最终结论是合成引导是高效的结构入门,但需要真实数据进行课程学习以完善语义。

原文 · arXiv cs.LG

Data Synthesis and Parameter-Efficient Fine-Tuning for Low-Resource NMT: A Case Study on Q'eqchi' Mayan

Neural machine translation for digitally low-resource Indigenous languages is often hindered by extreme data scarcity, prompting reliance on extractive web-scraping. To ensure data sovereignty, this study introduces a data synthesis methodology to bootstrap NMT models without scraping target-language parallel text. Focusing on Q'eqchi' Mayan, we transformed community-sourced dictionaries into a massive synthetic corpus, utilizing Parameter-Efficient Fine-Tuning (PEFT) via LoRA adapters on an mT5-base model. In-domain evaluation demonstrates high structural acquisition (BLEU 42.02), proving that synthetic constraints effectively teach complex agglutinative morphology and VOS word order. However, evaluation against an organic glossary reveals a structural-semantic gap (BLEU 0.59), where the model maintains grammatical integrity but lacks the lexical grounding of natural language. The model exhibits overfitting to the constrained structural variance of the synthetic templates; despite high semantic entropy in the pipeline, it struggles with the syntactic fluidity of natural language, forcing organic inputs into rigid learned patterns. Furthermore, an ablation study utilizing a Multi-Task Learning architecture resulted in negative transfer, suggesting that auxiliary tasks competed for limited parameter capacity within the LoRA adapters, causing over-optimization for synthetic markers at the expense of organic flexibility. Ultimately, we establish that synthetic bootstrapping is a highly effective structural primer, but requires authentic data for semantic refinement via Curriculum Learning.