🤖 ReflectDiffu-Reproduce

Reproduction implementation for: ReflectDiffu: Reflect between Emotion-intent Contagion and Mimicry for Empathetic Response Generation via a RL-Diffusion Framework

This repository provides a complete implementation for training and evaluating the ReflectDiffu model, which generates empathetic responses by combining emotion-intent contagion with reinforcement learning and diffusion frameworks.

Train file explination

其它模型组件介绍

Emotion Contagion部分介绍
ERA部分介绍
Intent Twice部分介绍

模型整体训练与 model_intergration.py 中 ReflectDiffu.forward 说明

本节详细解释训练主流程中各组件的输入 / 输出张量、形状、以及它们之间的衔接。源码核心调用链：

train.py -> ReflectDiffu.initialize_all() -> 迭代 batches -> train_step() -> model.forward(batch)。

各子模块内部更细的结构、公式与背景可参考对应子目录内的 README：

• 情感传染编码（Emotion Contagion）：src/emotion_contagion/README.md
• ERA（情感原因标签/推理，可选）：src/era/README.md
• Intent Twice（RL + Diffusion 意图二次机制）：src/intent_twice/README.md

1. 批数据准备 _prepare_batch_data

从 batches（由 _init_data_loader 生成）中取出一个元素，构造成训练所需张量：

备注：B=batch size，L=编码序列最大长度，T_dec=解码序列最大长度。

2. 情感传染编码器阶段

调用：

enc_out = self.encoder.forward(tokens, label_ids, attention_mask, h_tilde)

主要输出：

损失：

3. 语义意图分布 + Intent Twice 集成

语义打分：

P_semantic, _ = self.semantic_scorer(Q)   # [B, N_intent]
enc_out['P_semantic'] = P_semantic

随后调用意图二次模块：

it_out = self.it_module.forward(encoder_out=enc_out, intent_out={'p_intent': p_intent})

意图二次模块（内部含 EMU + Policy + 融合）核心产物：

4. 响应解码器阶段（Pointer-Generator）

准备：如果 Emofused 是 [B,D] 则 unsqueeze(1) → [B,1,D]。

源码调用：

dec_out = self.decoder_with_loss(
    trg_input_ids, emofused=Emofused, src_token_ids=src_ids_from_encoder,
    gold_ids=gold_ids, tgt_key_padding_mask=tgt_key_padding_mask)

输出（PaperCompliantDecoderWithLoss）：

定义响应损失：Lres = dec_out['loss']。

5. Forward 最终返回字典

ReflectDiffu.forward 返回：

训练中联合损失（train_step 内）：joint_loss = δ·Lem + ζ·Ltwice + η·Lres，对应命令行或配置中的 --delta / --zeta / --eta 权重。

6. 组件协同总结

1. Data Processor：清洗并对齐用户/回应，产生 token ids 与标签。
2. Emotion Contagion Encoder：建模上下文情感动态 → 输出序列表示 H 与全局表示 Q 及情感分布 P。
3. Semantic Scorer：基于 Q 额外生成语义意图分布 P_semantic 与外部意图分布结合。
4. Intent Twice (EMU + Policy)：利用正/负分支扩散 + 意图候选策略，校正并融合为 Emofused，提供情感-意图一致性。
5. Response Decoder：仅以 Emofused 为跨注意键值进行受控生成；Pointer-Generator 结合复制以提升可复制实体/情感词。
6. Loss Aggregation：三块损失加权合成，支持灵活调参；可扩展加入 ERA 或策略损失权重。

7. 快速定位排查建议

---

通过上述分层结构与张量流向，可以快速理解从原始对话输入到最终 empathetic response 生成的全链路：情感与意图在编码阶段耦合，再经 RL-Diffusion 双分支强化后，在解码阶段被集中用于生成控制。若需要深入内部公式或细节，请跳转各自子目录 README。

🚀 Quick Start

Final Result Fast Review

You can review current the best result in

./output/eval_logs/

Evaluate Pre-trained Model

Run the best pre-trained model with comprehensive evaluation metrics:

python display.py --model_path output/best_model/best_model.pt

Output: Evaluation results will be saved in output/eval_logs/

Metrics Evaluated:

• 📊 Relevance: BLEU-1/2/3/4, BARTScore
• 🎯 Informativeness: Perplexity (PPL), Distinct-1/2

Custom Evaluation

# Evaluate with custom parameters
python display.py --model_path path/to/model.pt --data_path dataset/test.pkl --max_samples 200

# Use specific device
python display.py --model_path output/best_model/best_model.pt --device cuda

🏗️ Training Setup

1. 📁 Data Preparation

Place your data files in the dataset/ directory:

dataset/
├── emotion_labels_user_response.pkl  # Training data
└── emotion_labels_test.pkl           # Test data

Data Format:

[
    [
        [user_data, response_data]
    ],
    [
        [user_data1, response_data1]
    ],
    # ... more conversation pairs
]

Data Structure:

• user_data: [(word1, <em>), (word2, <noem>), ...]
• response_data: [(word1, <em>), (word2, <noem>), ...]

2. 🧠 EmpHi Intent Prediction Setup

Download the required EmpHi models and reflect-Diffu best models from Google Drive and organize as follows:

pre-trained/
├── intent_prediction/
│   └── paras.pkl          # Intent prediction parameters
└── model/
    └── model              # Pre-trained EmpHi model

output
└── best_model/
    └── best_model.pt      # trained best model

3. 🚂 Start Training

python train.py

📊 Evaluation Metrics

Relevance Metrics

• BLEU-1/2/3/4: N-gram overlap with reference responses
• BARTScore: Semantic similarity using BART model
• Brevity Penalty: Length normalization factor

Informativeness Metrics

• Perplexity (PPL): Language model confidence (lower is better)
• Distinct-1/2: Lexical diversity at unigram/bigram level (higher is better)

🛠️ Dependencies

Key requirements:

• Python 3.10
• PyTorch 2.0+
• transformers
• sacrebleu
• torcheval (optional, for optimized perplexity computation)

Install dependencies:

pip install -r requirement

📝 Usage Examples

Basic Evaluation

python display.py --model_path output/best_model/best_model.pt

Training from Scratch

# Ensure data is in dataset/ and pre-trained models are in pre-trained/
python train.py

Custom Evaluation with Specific Parameters

python display.py \
    --model_path checkpoints/epoch_10.pt \
    --data_path dataset/emotion_labels_test.pkl \
    --max_samples 500 \
    --device cuda

📂 Project Structure

ReflectDiffu-Reproduce/
├── src/                           # Source code modules
│   ├── emotion_contagion/         # Emotion contagion components
│   ├── intent_twice/              # Intent processing modules
│   └── era/                       # ERA components
├── evaluation/                    # Evaluation scripts
│   ├── relevance_evaluation.py    # BLEU & BARTScore evaluation
│   └── informativeness.py         # Perplexity & Distinct-n evaluation
├── dataset/                       # Training and test data
├── pre-trained/                   # Pre-trained models
├── output/                        # Model outputs and logs
│   ├── best_model/               # Best trained model
│   └── eval_logs/                # Evaluation results
├── display.py                     # Main evaluation script
├── train.py                       # Training script
└── README.md                      # This file