多模态推理是 AI 落地的关键瓶颈，MARS 展示了如何整合视频、转录、热成像等异构数据做智能体决策，做多模态 AI 或视频理解的团队值得参考其开源代码。
MARS 是一个多模态智能体推理系统，专为 EgoVis 2026 的 CASTLE 挑战赛设计。该挑战要求回答 185 个关于四天活动、15 个同步视角、官方转录及多种辅助模态（如个人照片、热成像、心率数据）的封闭式问题。MARS 将任务视为多模态证据选择问题，通过构建视频和转录等主要来源以及辅助来源的证据记忆，并使用 DeepSeek 压缩长视频，最后通过 GPT-5.4 决策代理选择继续推理、请求缺失模态或生成答案。该系统在最终排行榜上获得第二名，代码已开源。
MARS: Technical Report for the CASTLE Challenge at EgoVis 2026
This report presents MARS, short for Multimodal Agentic Reasoning with Source selection, our system for the CASTLE Challenge at EgoVis 2026. Participants must answer 185 closed-form questions over the CASTLE 2024 dataset. In contrast to prior single-video egocentric benchmarks, CASTLE requires reasoning over four days of activity, 15 synchronized perspectives, official transcripts, and multiple auxiliary modalities, including personal photos, auxiliary videos, gaze, thermal imagery, and heartrate measurements. MARS therefore treats the task as an agentic evidence-selection problem over multimodal sources rather than a purely text-only pipeline. MARS first follows the official CASTLE directory organization to build evidence memories from two primary sources, videos and transcripts, and four auxiliary sources, gaze, heartrate, photos, and thermal imagery. Long videos are converted into captions and DeepSeek-based summaries only because CASTLE videos are too long to fit directly into the model context for every question; this step compresses temporal evidence while keeping photos and other auxiliary media available as source-specific evidence. At inference time, a GPT-5.4 decision agent repeatedly chooses whether to continue reasoning, request a specific missing modality, produce an answer, or fall back to a random option when the evidence remains insufficient. The resulting system achieved second place on the final CASTLE Challenge leaderboard. Our codes are available at https://github.com/Hyu-Zhang/MARS.