这篇论文介绍了一个大学多模态聊天助手，用RAG把幻觉从31.7%压到6.6%，还能处理图片提问，代码也开源了。
该论文提出一个面向大学利益相关者的多模态聊天助手，基于检索增强生成（RAG）架构。系统结合大语言模型（LLM）与语义检索，能够从大学手册等机构资源中生成上下文相关回答。支持文本和图像查询（通过视觉-语言模型），并采用量化推理在受限硬件上快速部署。后端使用FastAPI构建，前端基于Next.js开发，确保实时可用性。多模态评估显示，该RAG系统将幻觉率从31.7%降至6.6%，文本和图像查询均获得高满意度评分。
Towards Developing a Multimodal Chat Assistant for University Stakeholders: RAG-based Approach
University stakeholders often face difficulties in accessing timely and reliable information, especially in developing countries, where there are very few intelligent support systems. Existing rule-based chatbots are unable to handle complex, domain-specific queries and are not well-equipped to adapt to evolving institutional policies. As a fill-in-the-gap solution, we present the multimodal university chatbot with retrieval-augmented generation. The system combines the large language model with semantic retrieval to produce context-based responses from institution-centric resources, such as the university handbook. The system accepts text and image queries through the vision-language model and applies quantized inference for rapid deployment on constrained hardware. A scalable backend built with FastAPI, adjoined with a responsive frontend developed with Next.js, ensures real-time usability. Our multimodal evaluation demonstrates that the system maintains strong satisfaction scores across both text and image queries, despite increased response time for visual inputs. Furthermore, quantitative evaluation shows that hallucination is reduced from 31.7% to 6.6% in our proposed RAG-based system, confirming the effectiveness of retrieval grounding.