AI模型精选76°

Kwai Keye-VL-2.0:开源MoE多模态模型,支持256K长视频理解

Kwai Keye-VL-2.0 Technical Report

精选理由

长视频理解和智能体场景的开发者终于有了一个开源且高效的MoE模型——Keye-VL-2.0仅激活3B参数就能处理256K上下文,做视频分析或智能体应用的团队可以直接下载权重试试。

AI 摘要

快手发布Keye-VL-2.0-30B-A3B,一个基于MoE架构的开源多模态基础模型,专为长视频理解和智能体场景设计。该模型首次将DeepSeek Sparse Attention适配到GQA架构,实现无损256K上下文处理,能捕捉关键帧和长程时序依赖。通过跨模态多教师策略蒸馏(MOPD)和上下文/视频强化学习,解决了多任务对齐中的灾难性遗忘问题,仅激活3B参数即可在代码、工具和搜索场景中实现智能体协作与多模态自我纠正。在视频理解、时序定位、推理、STEM和智能体基准测试中,Keye-VL-2.0在相似规模模型中达到最优性能,尤其在TimeLens细粒度时序定位和Video-MME-v2长视频理解上表现突出。模型权重已开源。

原文 · arXiv: DeepSeek

Kwai Keye-VL-2.0 Technical Report

We introduce Kwai Keye-VL-2.0-30B-A3B, an open-source Mixture-of-Experts (MoE) multimodal foundation model designed to advance long-video understanding and agentic intelligence. To address the challenges of ultra-long contexts, information redundancy, and prohibitive computational costs inherent in hour-level videos, Keye-VL-2.0 is the first to adapt DeepSeek Sparse Attention (DSA) to GQA-based multimodal architectures, enabling lossless 256K context processing while capturing critical frames and long-range temporal dependencies. This architecture is underpinned by a highly optimized training and inference infrastructure, including scalable video I/O, heterogeneous ViT-LM parallelism, and custom DSA kernels that significantly maximize throughput and minimize computational overhead. Furthermore, to overcome the algorithmic dilemma of catastrophic forgetting during multi-task alignment, we introduce Cross-Modal Multi-Teacher On-Policy Distillation (MOPD) paired with Context-RL and Video-RL. By distilling dense token-level teacher feedback from on-policy rollouts back into the MoE backbone, which activates only 3B parameters, Keye-VL-2.0 natively empowers advanced agent collaboration across Code, Tool, and Search scenarios with multimodal self-correction. Extensive evaluations across video understanding, temporal grounding, reasoning, STEM, and agent benchmarks demonstrate that Keye-VL-2.0-30B-A3B achieves state-of-the-art performance among models of similar scale, particularly excelling in fine-grained temporal localization on TimeLens and long-video comprehension on Video-MME-v2 and LongVideoBench. We release our model checkpoints to accelerate community progress toward scalable and robust multimodal agentic applications.

Kwai Keye-VL-2.0:开源MoE多模态模型,支持256K长视频理解 · AI 热点