多模态可穿戴传感可在自闭症青少年情绪爆发前识别焦躁状态
Detecting Agitation Before Behavioral Escalation in Autistic Youth Through Multimodal Wearable Sensing
用腕表、麦克风和动作数据在自闭症孩子情绪爆发前预警，AUC 0.724，15 个孩子共用一个模型就够。
研究针对自闭症青少年中占比 68% 的攻击、自伤等挑战性行为，在其升级前的焦躁阶段进行预警。团队在 30 次临床会话中采集 15 名自闭症青少年的上半身动作、腕部生理信号和领夹麦克风语音，并配以专家行为标注。
方法将四个预训练基础模型按模态分别编码，投影到共享 128 维空间后融合为单一群体模型。模型在临床医生标注的焦躁起点处 AUC 达 0.
724（置换检验 p=0.0005），起点前 30 秒降至 0.
608，15 人中 13 人高于随机水平。消融显示音频贡献最大，仅用手表配置接近随机，从零训练只达 0.
58。
Detecting Agitation Before Behavioral Escalation in Autistic Youth Through Multimodal Wearable Sensing
Challenging behaviors including aggression, self-injury, and property destruction are observed in 68% of autistic youth and pose risks to youth and caregivers. These episodes are preceded by agitation, a rising state of distress expressed through movement, vocalization, and autonomic arousal. Its signs are subtle and individualized, and its autonomic components are invisible without instrumentation. We collected upper-body movement from inertial measurement units, physiology from a wrist-worn device, and vocalizations from lapel microphones across 30 clinician-led sessions with 15 autistic youth, paired with expert behavioral annotations. We adapt four pretrained foundation models, one per modality, project each to a shared 128-dimensional space, and fuse them into a single group model. The model detected agitation with an area under the ROC curve of 0.724 at the clinician-annotated onset (within-participant permutation p=0.0005), declining to 0.608 at 30,s before onset. Thirteen of fifteen participants were above chance. A from-scratch configuration reached only 0.58, while frozen and fine-tuned features performed comparably (0.71 and 0.72). Audio contributed most of the signal, and a watch-only configuration stayed near chance. Individualized agitation is therefore detectable, including in unannotated windows preceding the annotated onset, using foundation-model transfer with one shared model rather than one per child.
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Key entities and technical topics under continuous tracking include multimodal, wearables, foundation models.
Source attribution: curated from arXiv cs.LG (@Nibraas Khan, Abigale Plunk, John Staubitz, Ingrid Shragge, Jordan Brooks, Suzanne Wright, Alec Brewer, James Dieffenderfer, Alper Bozkurt, Amy Weitlauf, Nilanjan Sarkar).
