低资源语言NLP开发者终于有了可本地部署的高效嵌入方案——e5-sk系列在体积缩减62%后仍能匹敌商业API,做斯洛伐克语语义搜索或RAG的团队可以直接用开源模型替代付费服务。
研究团队发布了SkMTEB,这是斯洛伐克语首个全面的MTEB风格文本嵌入基准,包含31个数据集和7种任务类型,覆盖深度是现有多语言基准的近4倍。评估31个嵌入模型后发现,大型指令微调多语言模型表现最佳,而斯洛伐克语专用NLU模型在嵌入任务上迁移效果差。为满足高效本地部署需求,团队通过词汇修剪和微调Multilingual E5模型,开发了e5-sk-small(45M参数)和e5-sk-large(365M)模型,体积最多减少62%,性能却与商业API相当。所有基准、模型、数据集和代码均已开源,为其他低资源语言提供了可复现的路径。
SkMTEB: Slovak Massive Text Embedding Benchmark and Model Adaptation
We introduce SkMTEB, the first comprehensive MTEB-style text embedding benchmark for Slovak, a low-resource West Slavic language, comprising 31 datasets across 7 task types -- nearly 4$\times$ the depth of existing multilingual benchmark coverage for Slovak. Our evaluation of 31 embedding models reveals that large instruction-tuned multilingual models achieve the strongest performance, while existing Slovak-specific models trained for NLU tasks transfer poorly to embedding tasks. To address the need for efficient, locally-deployable Slovak embeddings, we develop \texttt{e5-sk-small} (45M parameters) and \texttt{e5-sk-large} (365M) by applying vocabulary trimming and fine-tuning to Multilingual E5 models. Despite size reductions of up to 62\%, our open-source models achieve competitive performance with proprietary APIs while remaining locally deployable for semantic search and retrieval-augmented generation (RAG). We release the benchmark, models, datasets, and code openly, hoping our approach offers a replicable path for other under-resourced languages.