🧠 LLM Reasoning for Machine Translation: Synthetic Data Generation over Thinking Tokens

📘 Overview

In this work, we explore fine-tuning Large Language Models (LLMs) for Machine Translation (MT) so that they generate intermediate reasoning tokens before producing the final translation.

Starting from a parallel dataset D and a teacher model IT, we synthesize intermediate reasoning traces and use them to fine-tune a student model m.
Given a source–target pair (x, y), we prompt ITto produce an intermediate reasoning sequence r (linking x to y), and fine-tune m to model the joint distribution p(y, r|x).

Two strategies are used to obtain intermediate tokens:

• 🧩 CoT Prompting – The teacher model is prompted using a Chain-of-Thought (CoT) template to explain step by step how to translate the source into the provided target.
  This mimics a human translator’s reasoning process and is analogous to CoT distillation used in reasoning tasks.

• ⚙️ Stepwise Prompting Strategies for MT – Translation is decomposed into several stages, each guided by a specific prompt contributing to the final output.
  The intermediate results from all steps are concatenated into a single sequence $r$, used as the reasoning trace.

The resulting "extended" dataset {(x_i, r_i, y_i)}_{i=1}^{|D|} enables CoT Fine-Tuning (CoTFT), where models learn to generate reasoning and translation jointly.
We compare this with Input–Output Fine-Tuning (IOFT), where models directly produce the translation without intermediate reasoning.

For CoT prompting, we experiment with six prompt templates inspired by MT-R1-Zero.
For stepwise prompting, we use strategies introduced in MAPS, SBYS, TEaR, Self-Refine, and CompTra.

📊 Dataset Summary

This dataset builds on almanach/topxgen-llama-4-scout-and-llama-4-scout.
We use Llama-4-Scout-17B-16E-Instruct as the teacher model, applying CoT prompting to generate intermediate reasoning traces that enrich the data for training thinking MT models.

Each entry contains:

• source: Input sentence in English
• translation: Target translation
• source_language: Source language
• target_language: Target language
• refined translations: list containing the teacher's zero-shot translation and between 3 and 5 self-refined versions.
• better-translation: The highest-quality translation of the source, selected between translation and all the entries in refined translations using BLASER-2.0 QE
• target: Concatenated sequence of reasoning tokens and translation, used for CoT fine-tuning
• better-target: Concatenated sequence of reasoning tokens and better-translation, used for CoT fine-tuning (Max).

🌍 Supported Languages

• Xhosa (xho)

⚙️ Data Generation Process

• Base dataset: almanach/topxgen-llama-4-scout-and-llama-4-scout
• Teacher model: Llama-4-Scout-17B-16E-Instruct
• Generation method: Self-refine (as described above)

💻 Example Usage

from datasets import load_dataset

dataset = load_dataset("almanach/topxgen-llama-4-scout-REFINE", split="REFINE")
print(dataset)

Output

Dataset({
    features: ['source', 'source_language', 'target_language', 'translation', 'phrases', 'refined translations', 'better-translation', 'target', 'better-target'],
    num_rows: 145441
})

📈 Intended Uses

This dataset is intended for:

• Fine-tuning LLMs for reasoning-based translation
• Studying thinking tokens in MT
• Comparing CoTFT vs IOFT in MT setups

⚖️ Licensing

This dataset is derived from the outputs of Llama-4-Scout-17B-16E-Instruct. Users must comply with the licenses and usage policies of the base model.

📚 Citation

If you use this dataset, please cite:

@misc{zebaze2025llmreasoningmachinetranslation,
  title={{LLM Reasoning for Machine Translation: Synthetic Data Generation over Thinking Tokens}}, 
  author={Armel Zebaze and Rachel Bawden and Benoît Sagot},
  year={2025},
  eprint={2510.11919},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2510.11919}, 
}