ASL 训练的姿态编码器可零样本迁移至六种低资源手语检索
Studies on Sign Language Retrieval in Low-Resource Dictionary Settings
只训练美国手语，就能直接检索六种其他手语，连标注语料都没有的词典场景也能用。InfoNCE 加阻断肢体间注意力这两个细节就能涨 5 个点以上，做手语或视频检索的值得看看消融部分。
一项研究考察了仅用美国手语数据训练的姿态编码器，能否在没有任何目标语言训练的情况下服务其他手语。编码器在 ASL-Citizen 上以度量学习训练后冻结评测，覆盖土耳其、俄罗斯、印度、阿根廷、巴基斯坦和斯里兰卡六种手语数据集。每个词条只注册 1 条示例视频时，编码器在 36.4%-80.7% 的查询中首位命中正确词条，优于最佳免学习方法（基于角度的动态时间规整）的 16.0%-69.6%，也超过 ASL 训练的 OpenHands 图网络的六目标平均 39.3%。
消融实验显示两个设计选择最关键：逐对 InfoNCE 训练目标优于监督对比学习和交叉熵；在编码器内部阻断身体、左手、右手之间的注意力可提升最多 5.1 个百分点。最佳配置达到 68.0% 的 one-shot 准确率，参考编码器为 59.0%。在包含 4,765 个词条、每词仅 1 条视频的印度手语词典 CISLR 上，编码器首位命中率 17.3%-19.0%，高于原始 RGB 基线的 16.8%。
研究还测试了球面、欧氏和双曲三种嵌入空间，八次随机种子下差异不超过 1.6 个百分点，仅双曲 token 注意力加 InfoNCE 的组合低 1.4-2.0 个点，说明嵌入几何并非决定性因素。
Studies on Sign Language Retrieval in Low-Resource Dictionary Settings
Most sign languages have no annotated corpus, only small dictionaries with one or a few example videos per sign. This study examines how well a pose encoder trained on a single sign language serves such languages without any target-language training. An encoder trained with metric learning on American Sign Language (ASL-Citizen) is evaluated frozen on six datasets of Turkish, Russian, Indian, Argentinian, Pakistani and Sri Lankan sign languages, with one or four clips per sign enrolled and, where signer identities are available, the query's signer excluded from the gallery. With one enrolled clip, the encoder retrieves the correct sign first in 36.4-80.7 % of queries, against 16.0-69.6 % for the best learning-free method (angle-based dynamic time warping) and a six-target mean of 39.3 % for an ASL-trained OpenHands graph network. In a crossed ablation of the encoder (embedding geometry, aggregation of body and hand embeddings, training objective, and attention between articulators), two choices matter: a per-pair InfoNCE objective outperforms supervised contrastive and cross-entropy training, and blocking attention between body, left hand and right hand inside the encoder raises accuracy by up to 5.1 points. The best configuration reaches 68.0 % one-shot accuracy (59.0 % for the reference encoder). On CISLR, an Indian Sign Language dictionary of 4,765 words with one video per word, the encoders find the correct word first for 17.3-19.0 % of queries, above the 16.8 % of the original RGB baseline. Over eight seeds with the joint encoder, spherical, Euclidean and hyperbolic embedding spaces differ by at most 1.6 points, except the hyperbolic token-attention head trained with InfoNCE, which is 1.4-2.0 points worse, so the geometry is a minor factor.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include 手语识别, ASL-Citizen, OpenHands, InfoNCE.
Source attribution: curated from arXiv cs.AI (@Ushnish Sarkar, Vineet Kumar Rakesh, Bhaswar Chattopadhyay, Tapas Samanta).
