NVIDIA开源PixelUMM:一个模型同时支持图文和视频的理解与生成
NVIDIA把PixelUMM开源了,一个模型搞定图文视频的理解和生成,做法是不用视觉编码器直接吃像素块,搞多模态研究的可以看看源码。
NVIDIA开源了无编码器多模态模型PixelUMM,一套架构同时处理图像和视频的理解与双向生成。它去掉了传统独立的视觉编码器,把画面切分成像素块直接送入主模型,让输入理解和输出生成共用同一套底层表示。
这种设计简化了多模态流程,模型和权重已完整开源。
NVIDIA开源PixelUMM:一个模型同时支持图文和视频的理解与生成
NVIDIA发布了无编码器多模态模型PixelUMM,一套架构同时支持图像和视频的理解与双向生成。 它抛弃了传统的独立视觉编码器,把画面切分成像素块直接送入主模型处理,让输入理解和输出生成共用同一套底层表示。
这种设计大幅简化了多模态流程,为研究更原生的图文与视频一体化架构提供了完整开源参考。
模型:https://t.co/2O2lwzNWyY
This report is categorized under AITOP's "Models" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include PixelUMM, NVIDIA, multimodal, open-source models.
Source attribution: curated from Gorden Sun (@Gorden_Sun). Cross-source corroboration: 11 independent sources have covered this development (including kimmonismus, techcrunch, The Business Times: Tech), confirming industry-wide signal strength.
- kimmonismus10-05 12:42Source
- techcrunch10-05 19:33Source
- The Business Times: Tech10-06 05:09Source
- The Straits Times: Business10-06 06:01Source
- marktechpost10-06 17:30Source
- NVIDIA AI10-06 13:12Source
- Justine Moore10-06 19:01Source
- Tech in Asia10-07 01:17Source
- Nathan Lambert10-05 21:00Source
- PyTorch10-06 18:36Source