李飞飞点出了LLM的物理盲区,做机器人、自动驾驶或空间计算的团队,看完会重新思考模型架构。
李飞飞指出,大语言模型擅长处理文本模式,能描述房间,但无法理解椅子移动、玻璃破碎或阳光变化时房间如何改变。她强调,世界模型旨在学习视觉背后的隐藏结构,能预测相机未捕捉的视角、模拟物体行为,并支持实体智能体在真实或虚拟环境中行动。这种模型需要共同理解空间、因果和后果,是AI从文本走向物理世界的关键一步。
Great piece from Dr. Fei-Fei Li (@drfeifei) “The …
Great piece from Dr. Fei-Fei Li (@drfeifei)
“The world is not made of words....
A model that masters simulation can project its understanding into pixels for human consumption, and into action predictions for embodied agents."
LLMs learn patterns in text, so they can explain a room, but they do not naturally know how the room changes when a chair moves, glass breaks, sunlight shifts, or a robot pushes a cup.
A world model tries to learn the hidden structure behind what we see, meaning it can predict views the camera never captured, model object behavior, and support agents that act inside real or virtual environments.
To see a world from a new angle, to predict what happens when something is pushed, and to decide what to do next all require a common internal model of space, causality, and consequence.