List of Papers, Datasets and Code Repositories for Affective Computing with LLM and MLLM. This repo contains a majority of research works in the affective computing field, but it still may not encompass all the noteworthy works.
If you find we miss some related works or have wrong clarifications, please contact us or open issues!
This repo is under W.I.P. Please feel free to open issues and make PRs!
Toward Label-Efficient Emotion and Sentiment Analysis [PIEEE 2023]
Affective Image Content Analysis: Two Decades Review and New Perspectives [TPAMI 2022]
Emotion Recognition From Multiple Modalities: Fundamentals and Methodologies [SPM 2021]
Video Affective Content Analysis: A Survey of State-of-the-Art Methods [TAC 2015]
Aesthetics and Emotions in Images [SPM 2011]
VidEmo: Affective-Tree Reasoning for Emotion-Centric Video Foundation Models [NeurIPS 2025] [Proj] 
Benchmarking and bridging emotion conflicts for multimodal emotion reasoning [ACM MM 2025]
Beyond Emotion Recognition: A Multi-Turn Multimodal Emotion Understanding and Reasoning Benchmark [ACM MM 2025]
Mme-emotion: A holistic evaluation benchmark for emotional intelligence in multimodal large language models [Arxiv 2025] [Code] 
CULEMO: Cultural Lenses on Emotion--Benchmarking LLMs for Cross-Cultural Emotion Understanding [Arxiv 2025]
NUS-Emo at SemEval-2024 Task 3: Instruction-Tuning LLM for Multimodal Emotion-Cause Analysis in Conversations [Arxiv 2024] [Code] 
Open-vocabulary Multimodal Emotion Recognition: Dataset, Metric, and Benchmark [ICML 2024] [Code]
Merbench: A unified evaluation benchmark for multimodal emotion recognitio [Arxiv 2024] [Code]
Emotionqueen: A benchmark for evaluating empathy of large language models [ACL 2024]
Emobench: Evaluating the emotional intelligence of large language models [ACL 2024] [Code]
Mer 2024: Semi-supervised learning, noise robustness, and open-vocabulary multimodal emotion recognition [ACM MRAC 2024] [Code]
Mer 2023: Multi-label learning, modality robustness, and semi-supervised learning [ACM MM 2023] [Code]
Explainable Multimodal Emotion Recognition [Arxiv 2023] [Code]
EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models [Arxiv 2023] [Code]
Celebv-text: A large-scale facial text-video dataset [CVPR 2023] [Code]
Mafw: A large-scale, multi-modal, compound affective database for dynamic facial expression recognition in the wild [ACM MM 2022] [Code]
Ferv39k: A large-scale multi-scene dataset for facial expression recognition in videos [CVPR 2022] [Code]
CelebV-HQ: A large-scale video facial attributes dataset [ECCV 2022] [Code]
Multimodal Event-Aware Network for Sentiment Analysis in Tourism [IEEE MultiMedia 2021] [Code]
Dfew: A large-scale database for recognizing dynamic facial expressions in the wild [ACM MM 2020] [Code]
H-SIMS: A Chinese Multimodal Sentiment Analysis Dataset with Fine-grained Annotation of Modality [ACL 2020] [Code]
Mead: A large-scale audio-visual dataset for emotional talking-face generation [ECCV 2020] [Code]
Lucfer: A large-scale context-sensitive image dataset for deep learning of visual emotions [WACV 2019]
Learning discriminative sentiment representation from strongly-and weakly supervised cnns [ACM TOMM 2019]
Multi-Interactive Memory Network for Aspect Based Multimodal Sentiment Analysis [AAAI 2019] [Code]
Meld: A multimodal multi-party dataset for emotion recognition in conversations [ACL 2019] [Code]
Vistanet: Visual aspect attention network for multimodal sentiment analysis [AAAI 2019] [Code]
ulti-attention recurrent network for human communication comprehension [AAAI 2018] [Code]
Multimodal language analysis in the wild: Cmu-mosei dataset and interpretable dynamic fusion graph [ACL 2018] [Code]
Contemplating visual emotions: Understanding and overcoming dataset bias [ECCV 2018]
Heterogeneous knowledge transfer in video emotion recognition [TAFFC 2018] [Code]
Emotional attention: A study of image sentiment and visual attention [CVPR 2018]
Learning visual sentiment distributions via augmented conditional probability neural network [AAAI 2017]
Emotion recognition in context [CVPR 2017] [Code]
Cross-media learning for image sentiment analysis in the wild [ICCVW 2017]
Towards using visual attributes to infer image sentiment of social events [IJCNN 2017]
Predicting personalized emotion perceptions of social images [ACM MM 2016]
Image sentiment analysis using latent correlations among visual, textual, and sentiment views [ICASSP 2016]
Building a large scale dataset for image emotion recognition: The fine print and the benchmark [AAAI 2016]
Recognizing emotions from abstract paintings using non-linear matrix completion [CVPR 2016] [Code]
Robust image sentiment analysis using progressively trained and domain transferred deep networks [AAAI 2015]
LIRIS-ACCEDE: A video database for affective content analysis [TAFFC 2015] [Code]
Visual affect around the world: A large-scale multilingual visual sentiment ontology [ACM MM 2015]
A mixed bag of emotions: Model, predict, and transfer emotion distributions [CVPR 2015]
How do your friends on social media disclose your emotions [AAAI 2014]
Predicting emotions in user-generated videos [AAAI 2014] [Code]
Why we watch the news: a dataset for exploring sentiment in broadcast video news [ACM ICMI 2014]
Large-scale visual sentiment ontology and detectors using adjective noun pairs [ACM MM 2013]
Youtube movie reviews: Sentiment analysis in an audio-visual context [IEEE Intelligent Systems 2013] [Code]
Utterance-level multimodal sentiment analysis. [ACL 2013]
Deap: A database for emotion analysis; using physiological signals [TAFFC 2011] [Code]
The semaine database: Annotated multimodal records of emotionally colored conversations between a person and a limited agent [TAFFC 2011]
The geneva affective picture database (gaped): a new 730-picture database focusing on valence and normative significance [BRM 2011]
Affective image classification using features inspired by psychology and art theory [ACM MM 2010]
IEMOCAP: Interactive emotional dyadic motion capture database [Language resources and evaluation 2008] [Code]
The eNTERFACE'05 audio-visual emotion database [ICDEW 2006]
Emotional category data on images from the international affective picture system [BRM 2005] [Code]
International affective picture system (iaps): Technical manual and affective ratings [NIMH Center for the Study of Emotion and Attention 1993] [Code]
LaERC-S: Improving LLM-based Emotion Recognition in Conversation with Speaker Characteristics [ACL 2025]
Emotional Support with LLM-based Empathetic Dialogue Generation [Arxiv 2025]
Exploring Emotion-Sensitive LLM-Based Conversational AI [Arxiv 2025]
LLM-Enhanced Multi-Teacher Knowledge Distillation for Modality-Incomplete Emotion Recognition in Daily Healthcare [JBHI 2025]
Emotional Cognitive Modeling Framework with Desire-Driven Objective Optimization for LLM-empowered Agent in Social Simulation [Arxiv 2025]
MultiAgentESC: A LLM-based Multi-Agent Collaboration Framework for Emotional Support Conversation [ACL 2025] [Code]
Predicting Stock Price Movement with LLM-Enhanced Tweet Emotion Analysis [Arxiv 2025]
Revise, reason, and recognize: Llm-based emotion recognition via emotion-specific prompts and asr error correction [ICASSP 2025]
Beyond silent letters: Amplifying llms in emotion recognition with vocal nuances [ACL 2025] [Code]
Interview-based Depression Detection Using LLM-based Text Restatement and Emotion Lexicon [TAFFC 2025]
EQ-Negotiator: An Emotion-Reasoning LLM Agent in Credit Dialogues [Arxiv 2025] [Code]
SEALR: Sequential Emotion-Aware LLM-Based Personalized Recommendation System [ACM SIGIR 2025]
From personas to talks: Revisiting the impact of personas on llm-synthesized emotional support conversations [ACL 2025]
Eeg emotion copilot: Optimizing lightweight llms for emotional eeg interpretation with assisted medical record generation [Neural Networks 2025] [Code]
Emotionally Aware or Tone-Deaf? Evaluating Emotional Alignment in LLM-Based Conversational Recommendation Systems [ACL 2025]
Do large language models have “emotion neurons”? investigating the existence and role [ACL 2025]
Designing Heterogeneous LLM Agents for Financial Sentiment Analysis [ACM TMIS 2025]
DialogXpert: Driving Intelligent and Emotion-Aware Conversations through Online Value-Based Reinforcement Learning with LLM Priors [Arxiv 2025] [Code]
TOOL-ED: Enhancing empathetic response generation with the tool calling capability of LLM [ACL 2025] [Code]
Employing large language models for emotion detection in psychotherapy transcripts [frontiersin 2025]
Fearful falcons and angry llamas: Emotion category annotations of arguments by humans and llms [ACL 2025]
Mechanistic Interpretability of Emotion Inference in Large Language Models [Arxiv 2025] [Code]
Structured Prompting and LLM Ensembling for Multimodal Conversational Aspect-based Sentiment Analysis [ACM MM 2025]
iPET: An Interactive Emotional Companion Dialogue System with LLM-Powered Virtual Pet World Simulation [ACL 2025] [Code]
Fine-Grained Emotion Recognition with In-Context Learning: A Prototype Theory Approach [Arxiv 2025]
Combining Graph NN and LLM for Improved Text-Based Emotion Recognition [AIMSA 2024]
Emotion-Aware Systems for Hazardous Operations: Integrating Mobile Sensors with LLM [RICAI 2024]
The Effect of LLM-Based NPC Emotional States on Player Emotions: An Analysis of Interactive Game Play [CoG 2024]
Both matter: Enhancing the emotional intelligence of large language models without compromising the general intelligence [ACL 2024] [Code]
Empathizing Before Generation: A Double-Layered Framework for Emotional Support LLM [PRCV 2024] [Code]
Fairmindsim: Alignment of behavior, emotion, and belief in humans and llm agents amid ethical dilemmas [Arxiv 2024]
Towards a generative approach for emotion detection and reasoning [Arxiv 2024]
A pilot study of measuring emotional response and perception of LLM-generated questionnaire and human-generated questionnaires [Scientific Reports 2024]
Apathetic or Empathetic? Evaluating {LLM}s' Emotional Alignments with Humans [NeurIPS 2024] [Code]
Emollms: A series of emotional large language models and annotation tools for comprehensive affective analysis [ACM SIGKDD 2024] [Code]
Emotion-Aware Embedding Fusion in LLMs (Flan-T5, LLAMA 2, DeepSeek-R1, and ChatGPT 4) for Intelligent Response Generation [Arxiv 2024]
Emergence of Hierarchical Emotion Representations in Large Language Models [NeurIPSW 2024]
EAI: Emotional decision-making of LLMs in strategic games and ethical dilemmas [NeurIPS 2024] [Code]
Customising general large language models for specialised emotion recognition tasks [ICASSP 2024]
Large Language Models Performance Comparison of Emotion and Sentiment Classification [ACM MS 2024]
Enhancing large language model with decomposed reasoning for emotion cause pair extraction [Arxiv 2024]
PetKaz at SemEval-2024 task 3: Advancing emotion classification with an LLM for emotion-cause pair extraction in conversations [ACL 2024] [Code]
TEII: Think, Explain, Interact and Iterate with Large Language Models to Solve Cross-lingual Emotion Detection [ACL 2024]
Esc-eval: Evaluating emotion support conversations in large language models [ACL 2024] [Code]
Empo: Emotion grounding for empathetic response generation through preference optimization [Arxiv 2024]
EEG emotion copilot: pruning LLMs for emotional EEG interpretation with assisted medical record generation [Arxiv 2024]
Language models (mostly) do not consider emotion triggers when predicting emotion [ACL 2024]
Enhancing emotional generation capability of large language models via emotional chain-of-thought [Arxiv 2024]
Emotionally numb or empathetic? evaluating how llms feel using emotionbench [Arxiv 2023]
Building emotional support chatbots in the era of llms [Arxiv 2023]
Bias in emotion recognition with ChatGPT [Arxiv 2023]
VidEmo: Affective-Tree Reasoning for Emotion-Centric Video Foundation Models [NeurIPS 2025] [Code]
Embedding Analogies for Evaluating Emotion in LLM-Generated Utterances [CEUR-WS 2025]
Reliable Learning From LLM Features for Multimodal Emotion and Intent Joint Understanding [ICASSP 2025]
Affectgpt: A new dataset, model, and benchmark for emotion understanding with multimodal large language models [ICML 2025] [Code]
AffectGPT-R1: Leveraging Reinforcement Learning for Open-Vocabulary Multimodal Emotion Recognition [Arxiv 2025]
OV-MER: Towards Open-Vocabulary Multimodal Emotion Recognition [Arxiv 2024] [Code]
EmoPrefer: Can Large Language Models Understand Human Emotion Preferences? [Arxiv 2025]
Dialoguellm: Context and emotion knowledge-tuned large language models for emotion recognition in conversations [Neural Networks 2025]
EMO-Avatar: An LLM-Agent-Orchestrated Framework for Multimodal Emotional Support in Human Animation [ACM MM 2025]
Deemo: De-identity multimodal emotion recognition and reasoning [ACM MM 2025] [Code]
Domain-Separated Bottleneck Attention Fusion Framework for Multimodal Emotion Recognition [ACM TMCCA 2025]
Emotion across Modalities and Cultures: Multilingual Multimodal Emotion-Cause Analysis with Memory-inspired Framework [ACM MM 2025]
Benchmarking Zero-Shot Facial Emotion Annotation with Large Language Models: A Multi-Class and Multi-Frame Approach in DailyLife [Arxiv 2025]
Emotion-Qwen: Training Hybrid Experts for Unified Emotion and General Vision-Language Understanding [Arxiv 2025] [Code]
Towards multimodal emotional support conversation systems [TMM 2025] [Code]
Omni-emotion: Extending video mllm with detailed face and audio modeling for multimodal emotion analysis [Arxiv 2025] [Code]
Mer 2025: When affective computing meets large language models [ACM MM 2025] [Code]
FaVChat: Unlocking Fine-Grained Facial Video Understanding with Multimodal Large Language Models [Arxiv 2025]
Humanomni: A large vision-speech language model for human-centric video understanding [Arxiv 2025] [Code]
Affective-CoT: Decomposing Multimodal Emotion Reasoning through a Hierarchical Cognitive Workflow [ACM MM 2025]
Emo-Synergy: Synergizing Task-Specific and General Experts for Multimodal Emotion Recognition [ACM MRAC 2025]
ROSA: A Robust Self-Adaptive Model for Multimodal Emotion Recognition With Uncertain Missing Modalities [TMM 2025] [Code]
HumanOmniV2: From Understanding to Omni-Modal Reasoning with Context [Arxiv 2025] [Code]
R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcement Learning [Arxiv 2025] [Code]
Speak From Heart: An Emotion-Guided LLM-Based Multimodal Method for Emotional Dialogue Generation [ICMR 2024]
A LLM-Based Robot Partner with Multi-modal Emotion Recognition [ICIRA 2024]
Multimodal Emotion Recognition Using Feature Fusion: An LLM-Based Approach [IEEE Access 2024]
Emo-llama: Enhancing facial emotion understanding with instruction tuning [Arxiv 2024] [Code]
Emotion-llama: Multimodal emotion recognition and reasoning with instruction tuning [NeurIPS 2024] [Code]
SZTU-CMU at MER2024: Improving Emotion-LLaMA with Conv-Attention for Multimodal Emotion Recognition [ACM MRAC 2024] [Code]
Eald-mllm: Emotion analysis in long-sequential and de-identity videos with multi-modal large language model [Arxiv 2024]
Textualized and feature-based models for compound multimodal emotion recognition in the wild [ECCVW 2024] [Code]
AV-EmoDialog: Chat with Audio-Visual Users Leveraging Emotional Cues [Arxiv 2024]
Vllms provide better context for emotion understanding through common sense reasoning [Arxiv 2024]
Multimodal Emotion Captioning Using Large Language Model with Prompt Engineering [ACM MRAC 2024]
MicroEmo: Time-Sensitive Multimodal Emotion Recognition with Subtle Clue Dynamics in Video Dialogues [ACM MRAC 2024]
E3: Exploring Embodied Emotion Through A Large-Scale Egocentric Video Dataset [NeurIPS 2024] [Code]
Video emotion open-vocabulary recognition based on multimodal large language model [Arxiv 2024]
MODA: MOdular Duplex Attention for Multimodal Perception, Cognition, and Emotion Understanding [ICML 2025] [Code]
Why We Feel: Breaking Boundaries in Emotional Reasoning with Multimodal Large Language Models [CVPR 2025] [Code]
Empathic Prompting: Non-Verbal Context Integration for Multimodal LLM Conversations [Arxiv 2025]
Enhancing Zero-Shot Facial Expression Recognition by LLM Knowledge Transfer [WACV 2025] [Code]
Exploring LLM-generated culture-specific affective human-robot tactile interaction [Arxiv 2025]
Emosym: A symbiotic framework for unified emotional understanding and generation via latent reasoning [ACM MM 2025] [Code]
Understanding emotional body expressions via large language models [AAAI 2025]
Feallm: Advancing facial emotion analysis in multimodal large language models with emotional synergy and reasoning [ACM MM 2025] [Code]
MELT: Towards Automated Multimodal Emotion Data Annotation by Leveraging LLM Embedded Knowledge [Arxiv 2025] [Code]
Rethinking Facial Expression Recognition in the Era of Multimodal Large Language Models: Benchmark, Datasets, and Beyond [Arxiv 2025] [Code]
DEFT-LLM: Disentangled Expert Feature Tuning for Micro-Expression Recognition [Arxiv 2025]
Mellm: Exploring llm-powered micro-expression understanding enhanced by subtle motion perception [Arxiv 2025]
Contextual Attention-Based Multimodal Fusion of LLM and CNN for Sentiment Analysis [Arxiv 2025]
FaceXBench: Evaluating Multimodal LLMs on Face Understanding [Arxiv 2025] [Code]
Beyond Vision: How Large Language Models Interpret Facial Expressions from Valence-Arousal Values [Arxiv 2025]
Unif2ace: Fine-grained Face Understanding and Generation with Unified Multimodal Models [Arxiv 2025] [Code]
FaceInsight: A Multimodal Large Language Model for Face Perception [ACM MM 2025]
AMPLE: Emotion-aware multimodal fusion prompt learning for fake news detection [MMM 2025]
Customizing Visual Emotion Evaluation for MLLMs: An Open-vocabulary, Multifaceted, and Scalable Approach [Arxiv 2025] [Code]
Facial Emotion Detection Research based on an Improved Multi-modal LLM [AIHCIR 2024]
Face-mllm: A large face perception model [Arxiv 2024]
EmoLLM: Multimodal Emotional Understanding Meets Large Language Models [Arxiv 2024] [Code]
EVOLVE: Emotion and Visual Output Learning via LLM Evaluation [Arxiv 2024]
Knowledge-based emotion recognition using large language models [ACII 2024]
Vision-enabled large language and deep learning models for image-based emotion recognition [Cognitive Computation 2024]
Contextual emotion recognition using large vision language models [IROS 2024] [Code]
Analyzing key factors influencing emotion prediction performance of vllms in conversational contexts [ACL 2024]
LLM supervised Pre-training for Multimodal Emotion Recognition in Conversations [ICASSP 2025]
EmoVoice: LLM-based Emotional Text-To-Speech Model with Freestyle Text Prompting [ACM MM 2025] [Code]
AER-LLM: Ambiguity-aware emotion recognition leveraging large language models [ICASSP 2025] [Code]
EmoJudge: LLM Based Post-Hoc Refinement for Multimodal Speech Emotion Recognition [Interspeech 2025]
Prompt-Unseen-Emotion: Zero-shot Expressive Speech Synthesis with Prompt-LLM Contextual Knowledge for Mixed Emotions [Arxiv 2025]
Dynamic Parameter Memory: Temporary LoRA-Enhanced LLM for Long-Sequence Emotion Recognition in Conversation [Arxiv 2025]
RLAIF-SPA: Optimizing LLM-based Emotional Speech Synthesis via RLAIF [Arxiv 2025] [Code]
Towards LLM-Empowered Fine-Grained Speech Descriptors for Explainable Emotion Recognition [Arxiv 2025]
EmoSLLM: Parameter-Efficient Adaptation of LLMs for Speech Emotion Recognition [Arxiv 2025]
EMORL-TTS: Reinforcement Learning for Fine-Grained Emotion Control in LLM-based TTS [Arxiv 2025]
Chain-of-Thought Distillation with Fine-Grained Acoustic Cues for Speech Emotion Recognition [Interspeech 2025]
Toward zero-shot speech emotion recognition using llms in the absence of target data [ICASSP 2025]
Enhanced Emotion-aware Music Recommendation via Large Language Models [KDD 2025]
Emo-dpo: Controllable emotional speech synthesis through direct preference optimization [ICASSP 2025]
Beyond Global Emotion: Fine-Grained Emotional Speech Synthesis with Dynamic Word-Level Modulation [Arxiv 2025]
SMIIP-NV: A Multi-Annotation Non-Verbal Expressive Speech Corpus in Mandarin for LLM-Based Speech Synthesis [ACM MM 2025]
Leveraging speech ptm, text llm, and emotional tts for speech emotion recognition [ICASSP 2024]
Large language model-based emotional speech annotation using context and acoustic feature for speech emotion recognition [ICASSP 2024]
Enhancing multimodal emotion recognition through asr error compensation and llm fine-tuning [Interspeech 2024]
LanSER: Language-Model Supported Speech Emotion Recognition [Interspeech 2023]
List of Papers, Datasets and Code Repositories for Affective Computing with LLM and MLLM. This repo contains a majority of research works in the affective computing field, but it still may not encompass all the noteworthy works.
If you find we miss some related works or have wrong clarifications, please contact us or open issues!
This repo is under W.I.P. Please feel free to open issues and make PRs!
Toward Label-Efficient Emotion and Sentiment Analysis [PIEEE 2023]
Affective Image Content Analysis: Two Decades Review and New Perspectives [TPAMI 2022]
Emotion Recognition From Multiple Modalities: Fundamentals and Methodologies [SPM 2021]
Video Affective Content Analysis: A Survey of State-of-the-Art Methods [TAC 2015]
Aesthetics and Emotions in Images [SPM 2011]
VidEmo: Affective-Tree Reasoning for Emotion-Centric Video Foundation Models [NeurIPS 2025] [Proj]
Benchmarking and bridging emotion conflicts for multimodal emotion reasoning [ACM MM 2025]
Beyond Emotion Recognition: A Multi-Turn Multimodal Emotion Understanding and Reasoning Benchmark [ACM MM 2025]
Mme-emotion: A holistic evaluation benchmark for emotional intelligence in multimodal large language models [Arxiv 2025] [Code]
CULEMO: Cultural Lenses on Emotion--Benchmarking LLMs for Cross-Cultural Emotion Understanding [Arxiv 2025]
NUS-Emo at SemEval-2024 Task 3: Instruction-Tuning LLM for Multimodal Emotion-Cause Analysis in Conversations [Arxiv 2024] [Code]
Open-vocabulary Multimodal Emotion Recognition: Dataset, Metric, and Benchmark [ICML 2024] [Code]
Merbench: A unified evaluation benchmark for multimodal emotion recognitio [Arxiv 2024] [Code]
Emotionqueen: A benchmark for evaluating empathy of large language models [ACL 2024]
Emobench: Evaluating the emotional intelligence of large language models [ACL 2024] [Code]
Mer 2024: Semi-supervised learning, noise robustness, and open-vocabulary multimodal emotion recognition [ACM MRAC 2024] [Code]
Mer 2023: Multi-label learning, modality robustness, and semi-supervised learning [ACM MM 2023] [Code]
Explainable Multimodal Emotion Recognition [Arxiv 2023] [Code]
EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models [Arxiv 2023] [Code]
Celebv-text: A large-scale facial text-video dataset [CVPR 2023] [Code]
Mafw: A large-scale, multi-modal, compound affective database for dynamic facial expression recognition in the wild [ACM MM 2022] [Code]
Ferv39k: A large-scale multi-scene dataset for facial expression recognition in videos [CVPR 2022] [Code]
CelebV-HQ: A large-scale video facial attributes dataset [ECCV 2022] [Code]
Multimodal Event-Aware Network for Sentiment Analysis in Tourism [IEEE MultiMedia 2021] [Code]
Dfew: A large-scale database for recognizing dynamic facial expressions in the wild [ACM MM 2020] [Code]
H-SIMS: A Chinese Multimodal Sentiment Analysis Dataset with Fine-grained Annotation of Modality [ACL 2020] [Code]
Mead: A large-scale audio-visual dataset for emotional talking-face generation [ECCV 2020] [Code]
Lucfer: A large-scale context-sensitive image dataset for deep learning of visual emotions [WACV 2019]
Learning discriminative sentiment representation from strongly-and weakly supervised cnns [ACM TOMM 2019]
Multi-Interactive Memory Network for Aspect Based Multimodal Sentiment Analysis [AAAI 2019] [Code]
Meld: A multimodal multi-party dataset for emotion recognition in conversations [ACL 2019] [Code]
Vistanet: Visual aspect attention network for multimodal sentiment analysis [AAAI 2019] [Code]
ulti-attention recurrent network for human communication comprehension [AAAI 2018] [Code]
Multimodal language analysis in the wild: Cmu-mosei dataset and interpretable dynamic fusion graph [ACL 2018] [Code]
Contemplating visual emotions: Understanding and overcoming dataset bias [ECCV 2018]
Heterogeneous knowledge transfer in video emotion recognition [TAFFC 2018] [Code]
Emotional attention: A study of image sentiment and visual attention [CVPR 2018]
Learning visual sentiment distributions via augmented conditional probability neural network [AAAI 2017]
Emotion recognition in context [CVPR 2017] [Code]
Cross-media learning for image sentiment analysis in the wild [ICCVW 2017]
Towards using visual attributes to infer image sentiment of social events [IJCNN 2017]
Predicting personalized emotion perceptions of social images [ACM MM 2016]
Image sentiment analysis using latent correlations among visual, textual, and sentiment views [ICASSP 2016]
Building a large scale dataset for image emotion recognition: The fine print and the benchmark [AAAI 2016]
Recognizing emotions from abstract paintings using non-linear matrix completion [CVPR 2016] [Code]
Robust image sentiment analysis using progressively trained and domain transferred deep networks [AAAI 2015]
LIRIS-ACCEDE: A video database for affective content analysis [TAFFC 2015] [Code]
Visual affect around the world: A large-scale multilingual visual sentiment ontology [ACM MM 2015]
A mixed bag of emotions: Model, predict, and transfer emotion distributions [CVPR 2015]
How do your friends on social media disclose your emotions [AAAI 2014]
Predicting emotions in user-generated videos [AAAI 2014] [Code]
Why we watch the news: a dataset for exploring sentiment in broadcast video news [ACM ICMI 2014]
Large-scale visual sentiment ontology and detectors using adjective noun pairs [ACM MM 2013]
Youtube movie reviews: Sentiment analysis in an audio-visual context [IEEE Intelligent Systems 2013] [Code]
Utterance-level multimodal sentiment analysis. [ACL 2013]
Deap: A database for emotion analysis; using physiological signals [TAFFC 2011] [Code]
The semaine database: Annotated multimodal records of emotionally colored conversations between a person and a limited agent [TAFFC 2011]
The geneva affective picture database (gaped): a new 730-picture database focusing on valence and normative significance [BRM 2011]
Affective image classification using features inspired by psychology and art theory [ACM MM 2010]
IEMOCAP: Interactive emotional dyadic motion capture database [Language resources and evaluation 2008] [Code]
The eNTERFACE'05 audio-visual emotion database [ICDEW 2006]
Emotional category data on images from the international affective picture system [BRM 2005] [Code]
International affective picture system (iaps): Technical manual and affective ratings [NIMH Center for the Study of Emotion and Attention 1993] [Code]
LaERC-S: Improving LLM-based Emotion Recognition in Conversation with Speaker Characteristics [ACL 2025]
Emotional Support with LLM-based Empathetic Dialogue Generation [Arxiv 2025]
Exploring Emotion-Sensitive LLM-Based Conversational AI [Arxiv 2025]
LLM-Enhanced Multi-Teacher Knowledge Distillation for Modality-Incomplete Emotion Recognition in Daily Healthcare [JBHI 2025]
Emotional Cognitive Modeling Framework with Desire-Driven Objective Optimization for LLM-empowered Agent in Social Simulation [Arxiv 2025]
MultiAgentESC: A LLM-based Multi-Agent Collaboration Framework for Emotional Support Conversation [ACL 2025] [Code]
Predicting Stock Price Movement with LLM-Enhanced Tweet Emotion Analysis [Arxiv 2025]
Revise, reason, and recognize: Llm-based emotion recognition via emotion-specific prompts and asr error correction [ICASSP 2025]
Beyond silent letters: Amplifying llms in emotion recognition with vocal nuances [ACL 2025] [Code]
Interview-based Depression Detection Using LLM-based Text Restatement and Emotion Lexicon [TAFFC 2025]
EQ-Negotiator: An Emotion-Reasoning LLM Agent in Credit Dialogues [Arxiv 2025] [Code]
SEALR: Sequential Emotion-Aware LLM-Based Personalized Recommendation System [ACM SIGIR 2025]
From personas to talks: Revisiting the impact of personas on llm-synthesized emotional support conversations [ACL 2025]
Eeg emotion copilot: Optimizing lightweight llms for emotional eeg interpretation with assisted medical record generation [Neural Networks 2025] [Code]
Emotionally Aware or Tone-Deaf? Evaluating Emotional Alignment in LLM-Based Conversational Recommendation Systems [ACL 2025]
Do large language models have “emotion neurons”? investigating the existence and role [ACL 2025]
Designing Heterogeneous LLM Agents for Financial Sentiment Analysis [ACM TMIS 2025]
DialogXpert: Driving Intelligent and Emotion-Aware Conversations through Online Value-Based Reinforcement Learning with LLM Priors [Arxiv 2025] [Code]
TOOL-ED: Enhancing empathetic response generation with the tool calling capability of LLM [ACL 2025] [Code]
Employing large language models for emotion detection in psychotherapy transcripts [frontiersin 2025]
Fearful falcons and angry llamas: Emotion category annotations of arguments by humans and llms [ACL 2025]
Mechanistic Interpretability of Emotion Inference in Large Language Models [Arxiv 2025] [Code]
Structured Prompting and LLM Ensembling for Multimodal Conversational Aspect-based Sentiment Analysis [ACM MM 2025]
iPET: An Interactive Emotional Companion Dialogue System with LLM-Powered Virtual Pet World Simulation [ACL 2025] [Code]
Fine-Grained Emotion Recognition with In-Context Learning: A Prototype Theory Approach [Arxiv 2025]
Combining Graph NN and LLM for Improved Text-Based Emotion Recognition [AIMSA 2024]
Emotion-Aware Systems for Hazardous Operations: Integrating Mobile Sensors with LLM [RICAI 2024]
The Effect of LLM-Based NPC Emotional States on Player Emotions: An Analysis of Interactive Game Play [CoG 2024]
Both matter: Enhancing the emotional intelligence of large language models without compromising the general intelligence [ACL 2024] [Code]
Empathizing Before Generation: A Double-Layered Framework for Emotional Support LLM [PRCV 2024] [Code]
Fairmindsim: Alignment of behavior, emotion, and belief in humans and llm agents amid ethical dilemmas [Arxiv 2024]
Towards a generative approach for emotion detection and reasoning [Arxiv 2024]
A pilot study of measuring emotional response and perception of LLM-generated questionnaire and human-generated questionnaires [Scientific Reports 2024]
Apathetic or Empathetic? Evaluating {LLM}s' Emotional Alignments with Humans [NeurIPS 2024] [Code]
Emollms: A series of emotional large language models and annotation tools for comprehensive affective analysis [ACM SIGKDD 2024] [Code]
Emotion-Aware Embedding Fusion in LLMs (Flan-T5, LLAMA 2, DeepSeek-R1, and ChatGPT 4) for Intelligent Response Generation [Arxiv 2024]
Emergence of Hierarchical Emotion Representations in Large Language Models [NeurIPSW 2024]
EAI: Emotional decision-making of LLMs in strategic games and ethical dilemmas [NeurIPS 2024] [Code]
Customising general large language models for specialised emotion recognition tasks [ICASSP 2024]
Large Language Models Performance Comparison of Emotion and Sentiment Classification [ACM MS 2024]
Enhancing large language model with decomposed reasoning for emotion cause pair extraction [Arxiv 2024]
PetKaz at SemEval-2024 task 3: Advancing emotion classification with an LLM for emotion-cause pair extraction in conversations [ACL 2024] [Code]
TEII: Think, Explain, Interact and Iterate with Large Language Models to Solve Cross-lingual Emotion Detection [ACL 2024]
Esc-eval: Evaluating emotion support conversations in large language models [ACL 2024] [Code]
Empo: Emotion grounding for empathetic response generation through preference optimization [Arxiv 2024]
EEG emotion copilot: pruning LLMs for emotional EEG interpretation with assisted medical record generation [Arxiv 2024]
Language models (mostly) do not consider emotion triggers when predicting emotion [ACL 2024]
Enhancing emotional generation capability of large language models via emotional chain-of-thought [Arxiv 2024]
Emotionally numb or empathetic? evaluating how llms feel using emotionbench [Arxiv 2023]
Building emotional support chatbots in the era of llms [Arxiv 2023]
Bias in emotion recognition with ChatGPT [Arxiv 2023]
VidEmo: Affective-Tree Reasoning for Emotion-Centric Video Foundation Models [NeurIPS 2025] [Code]
Embedding Analogies for Evaluating Emotion in LLM-Generated Utterances [CEUR-WS 2025]
Reliable Learning From LLM Features for Multimodal Emotion and Intent Joint Understanding [ICASSP 2025]
Affectgpt: A new dataset, model, and benchmark for emotion understanding with multimodal large language models [ICML 2025] [Code]
AffectGPT-R1: Leveraging Reinforcement Learning for Open-Vocabulary Multimodal Emotion Recognition [Arxiv 2025]
OV-MER: Towards Open-Vocabulary Multimodal Emotion Recognition [Arxiv 2024] [Code]
EmoPrefer: Can Large Language Models Understand Human Emotion Preferences? [Arxiv 2025]
Dialoguellm: Context and emotion knowledge-tuned large language models for emotion recognition in conversations [Neural Networks 2025]
EMO-Avatar: An LLM-Agent-Orchestrated Framework for Multimodal Emotional Support in Human Animation [ACM MM 2025]
Deemo: De-identity multimodal emotion recognition and reasoning [ACM MM 2025] [Code]
Domain-Separated Bottleneck Attention Fusion Framework for Multimodal Emotion Recognition [ACM TMCCA 2025]
Emotion across Modalities and Cultures: Multilingual Multimodal Emotion-Cause Analysis with Memory-inspired Framework [ACM MM 2025]
Benchmarking Zero-Shot Facial Emotion Annotation with Large Language Models: A Multi-Class and Multi-Frame Approach in DailyLife [Arxiv 2025]
Emotion-Qwen: Training Hybrid Experts for Unified Emotion and General Vision-Language Understanding [Arxiv 2025] [Code]
Towards multimodal emotional support conversation systems [TMM 2025] [Code]
Omni-emotion: Extending video mllm with detailed face and audio modeling for multimodal emotion analysis [Arxiv 2025] [Code]
Mer 2025: When affective computing meets large language models [ACM MM 2025] [Code]
FaVChat: Unlocking Fine-Grained Facial Video Understanding with Multimodal Large Language Models [Arxiv 2025]
Humanomni: A large vision-speech language model for human-centric video understanding [Arxiv 2025] [Code]
Affective-CoT: Decomposing Multimodal Emotion Reasoning through a Hierarchical Cognitive Workflow [ACM MM 2025]
Emo-Synergy: Synergizing Task-Specific and General Experts for Multimodal Emotion Recognition [ACM MRAC 2025]
ROSA: A Robust Self-Adaptive Model for Multimodal Emotion Recognition With Uncertain Missing Modalities [TMM 2025] [Code]
HumanOmniV2: From Understanding to Omni-Modal Reasoning with Context [Arxiv 2025] [Code]
R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcement Learning [Arxiv 2025] [Code]
Speak From Heart: An Emotion-Guided LLM-Based Multimodal Method for Emotional Dialogue Generation [ICMR 2024]
A LLM-Based Robot Partner with Multi-modal Emotion Recognition [ICIRA 2024]
Multimodal Emotion Recognition Using Feature Fusion: An LLM-Based Approach [IEEE Access 2024]
Emo-llama: Enhancing facial emotion understanding with instruction tuning [Arxiv 2024] [Code]
Emotion-llama: Multimodal emotion recognition and reasoning with instruction tuning [NeurIPS 2024] [Code]
SZTU-CMU at MER2024: Improving Emotion-LLaMA with Conv-Attention for Multimodal Emotion Recognition [ACM MRAC 2024] [Code]
Eald-mllm: Emotion analysis in long-sequential and de-identity videos with multi-modal large language model [Arxiv 2024]
Textualized and feature-based models for compound multimodal emotion recognition in the wild [ECCVW 2024] [Code]
AV-EmoDialog: Chat with Audio-Visual Users Leveraging Emotional Cues [Arxiv 2024]
Vllms provide better context for emotion understanding through common sense reasoning [Arxiv 2024]
Multimodal Emotion Captioning Using Large Language Model with Prompt Engineering [ACM MRAC 2024]
MicroEmo: Time-Sensitive Multimodal Emotion Recognition with Subtle Clue Dynamics in Video Dialogues [ACM MRAC 2024]
E3: Exploring Embodied Emotion Through A Large-Scale Egocentric Video Dataset [NeurIPS 2024] [Code]
Video emotion open-vocabulary recognition based on multimodal large language model [Arxiv 2024]
MODA: MOdular Duplex Attention for Multimodal Perception, Cognition, and Emotion Understanding [ICML 2025] [Code]
Why We Feel: Breaking Boundaries in Emotional Reasoning with Multimodal Large Language Models [CVPR 2025] [Code]
Empathic Prompting: Non-Verbal Context Integration for Multimodal LLM Conversations [Arxiv 2025]
Enhancing Zero-Shot Facial Expression Recognition by LLM Knowledge Transfer [WACV 2025] [Code]
Exploring LLM-generated culture-specific affective human-robot tactile interaction [Arxiv 2025]
Emosym: A symbiotic framework for unified emotional understanding and generation via latent reasoning [ACM MM 2025] [Code]
Understanding emotional body expressions via large language models [AAAI 2025]
Feallm: Advancing facial emotion analysis in multimodal large language models with emotional synergy and reasoning [ACM MM 2025] [Code]
MELT: Towards Automated Multimodal Emotion Data Annotation by Leveraging LLM Embedded Knowledge [Arxiv 2025] [Code]
Rethinking Facial Expression Recognition in the Era of Multimodal Large Language Models: Benchmark, Datasets, and Beyond [Arxiv 2025] [Code]
DEFT-LLM: Disentangled Expert Feature Tuning for Micro-Expression Recognition [Arxiv 2025]
Mellm: Exploring llm-powered micro-expression understanding enhanced by subtle motion perception [Arxiv 2025]
Contextual Attention-Based Multimodal Fusion of LLM and CNN for Sentiment Analysis [Arxiv 2025]
FaceXBench: Evaluating Multimodal LLMs on Face Understanding [Arxiv 2025] [Code]
Beyond Vision: How Large Language Models Interpret Facial Expressions from Valence-Arousal Values [Arxiv 2025]
Unif2ace: Fine-grained Face Understanding and Generation with Unified Multimodal Models [Arxiv 2025] [Code]
FaceInsight: A Multimodal Large Language Model for Face Perception [ACM MM 2025]
AMPLE: Emotion-aware multimodal fusion prompt learning for fake news detection [MMM 2025]
Customizing Visual Emotion Evaluation for MLLMs: An Open-vocabulary, Multifaceted, and Scalable Approach [Arxiv 2025] [Code]
Facial Emotion Detection Research based on an Improved Multi-modal LLM [AIHCIR 2024]
Face-mllm: A large face perception model [Arxiv 2024]
EmoLLM: Multimodal Emotional Understanding Meets Large Language Models [Arxiv 2024] [Code]
EVOLVE: Emotion and Visual Output Learning via LLM Evaluation [Arxiv 2024]
Knowledge-based emotion recognition using large language models [ACII 2024]
Vision-enabled large language and deep learning models for image-based emotion recognition [Cognitive Computation 2024]
Contextual emotion recognition using large vision language models [IROS 2024] [Code]
Analyzing key factors influencing emotion prediction performance of vllms in conversational contexts [ACL 2024]
LLM supervised Pre-training for Multimodal Emotion Recognition in Conversations [ICASSP 2025]
EmoVoice: LLM-based Emotional Text-To-Speech Model with Freestyle Text Prompting [ACM MM 2025] [Code]
AER-LLM: Ambiguity-aware emotion recognition leveraging large language models [ICASSP 2025] [Code]
EmoJudge: LLM Based Post-Hoc Refinement for Multimodal Speech Emotion Recognition [Interspeech 2025]
Prompt-Unseen-Emotion: Zero-shot Expressive Speech Synthesis with Prompt-LLM Contextual Knowledge for Mixed Emotions [Arxiv 2025]
Dynamic Parameter Memory: Temporary LoRA-Enhanced LLM for Long-Sequence Emotion Recognition in Conversation [Arxiv 2025]
RLAIF-SPA: Optimizing LLM-based Emotional Speech Synthesis via RLAIF [Arxiv 2025] [Code]
Towards LLM-Empowered Fine-Grained Speech Descriptors for Explainable Emotion Recognition [Arxiv 2025]
EmoSLLM: Parameter-Efficient Adaptation of LLMs for Speech Emotion Recognition [Arxiv 2025]
EMORL-TTS: Reinforcement Learning for Fine-Grained Emotion Control in LLM-based TTS [Arxiv 2025]
Chain-of-Thought Distillation with Fine-Grained Acoustic Cues for Speech Emotion Recognition [Interspeech 2025]
Toward zero-shot speech emotion recognition using llms in the absence of target data [ICASSP 2025]
Enhanced Emotion-aware Music Recommendation via Large Language Models [KDD 2025]
Emo-dpo: Controllable emotional speech synthesis through direct preference optimization [ICASSP 2025]
Beyond Global Emotion: Fine-Grained Emotional Speech Synthesis with Dynamic Word-Level Modulation [Arxiv 2025]
SMIIP-NV: A Multi-Annotation Non-Verbal Expressive Speech Corpus in Mandarin for LLM-Based Speech Synthesis [ACM MM 2025]
Leveraging speech ptm, text llm, and emotional tts for speech emotion recognition [ICASSP 2024]
Large language model-based emotional speech annotation using context and acoustic feature for speech emotion recognition [ICASSP 2024]
Enhancing multimodal emotion recognition through asr error compensation and llm fine-tuning [Interspeech 2024]
LanSER: Language-Model Supported Speech Emotion Recognition [Interspeech 2023]
