MAF:面向MLLM的多模态自适应少样本提示情感分析框架

MAF: Multimodal Adaptive Few-shot Prompting for Sentiment Analysis with MLLMs

精选理由

这篇论文提出MAF,能根据输入动态挑选示例来引导MLLM做情感分析,在多个数据集上效果提升明显,比固定提示强很多。

AI 摘要

该论文提出MAF(多模态自适应少样本提示)框架,用于提升多模态大语言模型在情感分析任务中的表现。MAF动态检索与查询相关的演示样本,通过编码面部表情、场景上下文和文本语义,并引入唇动幅度检测机制在多人场景中准确识别说话人。一个轻量级系数生成网络被训练用于实时输出查询条件化融合权重,加权聚合多模态相似度以选取Top-K信息量最大的示例。在多个公开基准数据集上,MAF相比骨干模型取得显著且一致的性能提升,并与强基线方法保持竞争力。

原文 · arXiv cs.LG

MAF: Multimodal Adaptive Few-shot Prompting for Sentiment Analysis with MLLMs

Multimodal large language models (MLLMs) have demonstrated remarkable capabilities in understanding complex multimodal content. However, their performance in sentiment analysis exhibits acute sensitivity to prompt design, rendering static, uniformly applied prompts inherently suboptimal for capturing the nuanced multimodal cues that vary across inputs. To address this limitation, we propose a Multimodal Adaptive Few-Shot Prompting (MAF) framework, which dynamically retrieves and integrates query-relevant demonstrations to elicit the sentiment reasoning capabilities of MLLMs in a context-sensitive manner. MAF constructs a demonstration retrieval module that holistically encodes facial expressions, scene context, and textual semantics, with a lip movement amplitude detection mechanism introduced for accurate speaker identification in multi-person scenarios. Departing from conventional fixed-weight fusion, a lightweight coefficient generation network is trained to output query-conditioned fusion weights in real time, enabling weighted aggregation of multimodal similarity scores to retrieve the top-K most informative demonstrations. Prediction stability is further enhanced through majority voting over multiple candidate outputs generated by the MLLM. Extensive experiments on public benchmark datasets demonstrate that MAF achieves substantial and consistent performance improvements over the corresponding backbone variants and remains competitive with strong multimodal sentiment-analysis baselines.