ClinPRISM: A Cost-Effective Multimodal LLM Framework for Clinical Time-Series QA

它通过不规则感知多尺度编码器捕获稀疏临床证据，使用时间证据蒸馏器将多尺度表示压缩为16个LLM兼容tokens。框架引入渐进对齐策略，将不规则轨迹与LLM文本嵌入空间对齐。

训练数据包含30,000个临床时间序列和41,000个指令微调实例，覆盖11个任务。在保留评估基准上，ClinPRISM达到最先进性能，平均推理延迟仅0.

15秒每问题。

A Cost-Effective Multimodal LLM Reasoning Framework for Question Answering over Irregular Clinical Time Series

Question answering (QA) over irregular clinical time series (ICTS) plays a pivotal role in a wide range of healthcare applications. Although recent multimodal time-series large language models (LLMs) have shown considerable promise in general-purpose time-series QA, they remain poorly equipped to model the sparsity, asynchrony, and irregular sampling patterns of clinical observations. To fill this gap, we propose ClinPRISM, a cost-effective multimodal LLM reasoning framework for question answering over ICTS data. First, we devise an irregularity-aware multi-scale encoder to capture sparse clinical evidence at diverse temporal scales. Then, we propose a temporal evidence distiller to integrate representations across these scales and compress them into a small number of LLM-compatible tokens. Moreover, we introduce a progressive alignment strategy that sequentially aligns the irregular trajectories with the LLM's textual embedding space. To facilitate training, we construct 30,000 clinical time series paired with multi-scale descriptions, together with 41,000 instruction-tuning instances spanning 11 tasks. Using a 4-billion-parameter LLM backbone, ClinPRISM achieves state-of-the-art performance on the held-out evaluation benchmark while using only 16 time-series tokens and achieving an average inference latency of 0.15 seconds per question.

This report is categorized under AITOP's "Papers" desk. Key entities and technical topics under continuous tracking include ClinPRISM, multimodal, time series, medical AI.

Source attribution: curated from arXiv cs.AI (@Frank Nie, Ethan B Liu, Yuan Zhu, Wei Fan, Jindong Han).