想了解手语识别和低资源语言翻译怎么落地？这篇用VideoMAE+NLLB-200直接搭了个两阶段流水线，13类测试精度78%，还带Streamlit演示和代码。
该研究提出两阶段深度学习流水线：先用微调后的VideoMAE视频Transformer将手语视频片段分类为英文标签（在AI4Bharat印度手语数据集的13类子集上达到99%训练精度和78%验证精度），再用Meta的NLLB-200模型将英文标签翻译为印地语、泰卢固语和孟加拉语。实验使用13类197个片段、80-20分割，15个epoch后完成训练，并通过混淆矩阵揭示主要失败模式（如ugly/deaf/blind等形容词混淆）。研究还提供了Streamlit演示入口，支持用户上传视频并输出预测标签及三种方言翻译。论文代码已开源。
Deep Learning-Based Sign Language Recognition from Videos and Cross-Lingual Translation to Indian Vernaculars
Sign language is a primary mode of communication for the global deaf and hard-of-hearing community, yet automated tools that recognize sign gestures from video and translate them into natural language text remain limited, particularly for low-resource Indian languages. We present a two-stage deep learning pipeline that (i) classifies short sign language video clips into English word labels using a fine-tuned VideoMAE video transformer, and (ii) translates the predicted English label into Hindi, Telugu, and Bengali using Meta AI's No Language Left Behind (NLLB-200) multilingual translation model. The classification model is fine-tuned on a 13-class subset of the AI4Bharat Indian Sign Language video corpus from IIT Madras, processing 16-frame clips sampled uniformly from each video at 224 x 224 resolution. Under a small-scale academic setting (13 classes, 197 clips, 80-20 split), the fine-tuned model reaches 99% training accuracy and 78% validation accuracy after 15 epochs. We provide a per-class breakdown via a confusion matrix and classification report, identify the dominant failure modes (confusable adjective pairs such as ugly, deaf, blind, hat, and dress), and describe a Streamlit-based inference demo that takes a user-uploaded video and returns the predicted English label alongside its Hindi, Telugu, and Bengali translations. We discuss the scope, limitations (small label set, isolated-word rather than continuous signing, single-signer style sensitivity, ambiguity of single-word machine translation), and directions for future work, including expanding to sentence-level generation and a larger vocabulary. Code is released to support reproducibility.