PapersSelected

Reading Emotions in the Token Space: Discriminative Adaptation of SpeechLLMs for Emotion Recognition

Why we picked it

This research provides a new method for improving emotion recognition in speech, which could be useful for developers working on applications like customer service or mental health support.

A new method adapts speech large language models (LLMs) for emotion recognition by reading the final prompt token's hidden state through a classification head.

Body

This approach, tested on the IEMOCAP benchmark, improves Macro F1 scores and removes hallucinations, especially on realistic ASR transcripts.

The method reveals emotion directions in the LLM's output token space, showing indirect associations mirroring biases in web-scale text.

Original · arXiv cs.AI

SpeechLLMs have shown strong potential for emotion recognition, yet they read the predicted emotion off a generative decoder not suited for classification: it can emit labels outside the target set and favors frequent classes. We propose a discriminative adaptation that reads the final prompt token's hidden state through a classification head, producing a label in one forward pass without modifying the backbone. Because this readout starts from the hidden state the model would otherwise decode, it gives a controlled comparison of generative and discriminative inference in an otherwise identical speechLLM. We keep the head a single linear layer, trading little accuracy for interpretability: each emotion becomes one direction in the LLM output token space, revealing associated tokens. On IEMOCAP, across two speechLLM architectures, it improves Macro F1 and removes hallucinations, with largest gains on realistic ASR transcripts. Our analysis reveals that these emotion directions encode indirect associations mirroring biases in web-scale text.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include SpeechLLMs, emotion recognition, prompt engineering.

Source attribution: curated from arXiv cs.AI (@Hasindri Watawana, Sergio Burdisso, Esaú Villatoro-Tello, Manjunath K E, Kadri Hacioglu, Petr Motlicek, Andreas Stolcke).