MoralAltDataset

MoralAltDataset accompanies the manuscript "Can LLMs Imagine Moral Alternatives Beyond Binary Dilemmas?" It studies whether humans and large language models move beyond forced binary choices in moral dilemmas by considering compromise and reframed alternatives.

Dataset summary

Table. Summary statistics of the dataset construction, alternative generation, and judgment collection.

The dataset contains 307 base moral dilemmas:

• 156 narrative Advisor dilemmas
• 151 AI-facing Agent dilemmas

Each dilemma is associated with original binary options and, where applicable, compromise and novel/reframed alternatives. The data includes human and LLM outputs used for judgment analysis and alternative-generation analysis.

Scenario sources

Following the taxonomy used in MoReBench, MoralAltDataset distinguishes between Agent and Advisor dilemmas.

• Agent scenarios are based on AIRiskDilemmas, a collection of dilemmas involving AI-risk behaviors.
• Advisor scenarios are based on movie-plot data from the MPST dataset.

The dilemma_source column records the dataset used as the basis for Agent scenarios: AIRiskDilemmas. In the Advisor subsets, synopsis_source records the original plot-synopsis provider reported by MPST—either wikipedia or imdb. Thus, synopsis_source identifies the upstream synopsis provider, while the Advisor dilemmas themselves are based on MPST.

Dataset files

The four CSV files are exposed as separate Dataset Viewer subsets because their schemas differ.

• advisor-judgment: 156 Advisor dilemmas with human and LLM A/B and A/B/C/D judgments.
• advisor-alternatives: 75 Advisor dilemmas with human-authored and LLM-generated compromise and reframed alternatives.
• agent-judgment: 151 Agent dilemmas with human and LLM A/B and A/B/C/D judgments.
• agent-alternatives: 89 Agent dilemmas with human-authored and LLM-generated compromise and reframed alternatives.

How to read the data

A compromise alternative balances the values represented by options A and B. A reframed alternative changes the framing of the conflict to propose a different path forward.

Judgment subsets

advisor-judgment and agent-judgment contain the dilemma, options A and B, both alternatives, and four judgment columns:

• human_abcd_judgment: five human choices among A, B, the compromise alternative, and the reframed alternative, plus the stored majority vote.
• LLMs_abcd_judgments: the same five-choice structure and majority vote for each of 15 LLMs.
• human_ab_judgment: five human choices when only A and B are available, plus the stored majority vote.
• LLMs_ab_judgments: the same A/B structure for the same 15 LLMs.
• alternatives_source: identifies whether the two alternatives in that row were written by a human or GPT-5.

Both the A/B and A/B/C/D evaluations therefore cover 16 sources: one human source and 15 LLMs. Each LLM record stores its five choices in model_judgments and the aggregate choice in model_majority_vote. Three A/B records from claude-4.5-sonnet are refusals and contain an empty model_judgments list with model_majority_vote set to REFUSAL.

The 15 judgment models are: claude-4-sonnet, claude-4.5-sonnet, claude-haiku-4.5, claude-opus-4.5, gemini-2.5-flash, gemini-2.5-pro, gpt-4o, gpt-5, gpt-5-mini, llama-3.3-70b, llama-4-scout, mistral-large-123b, mistral-small-3.1-24b, qwen-3-32b, and qwen-3.5-122b-a-10b.

Alternative-generation subsets

advisor-alternatives contains 75 Advisor dilemmas, and agent-alternatives contains 89 Agent dilemmas. For each dilemma and its original A/B options:

• human_compromise alternative and human_reframed_alternative contain the two human-authored alternatives as plain text.
• LLMs_compromise_alternatives is a JSON object keyed by model. Each entry contains the generated alternative, its justification, and a trade-off rule.
• LLMs_reframed_alternatives is a JSON object keyed by model. Each entry contains the generated alternative, its reframing type, and its justification.

These files contain outputs from one human source and 13 LLMs. The 13 generation models are: claude-haiku-4.5, claude-opus-4.5, claude-sonnet-4.5, gemini-2.5-flash, gemini-2.5-pro, gpt-4o, gpt-5, gpt-5-mini, llama-3.3-70b, llama-4-scout, mistral-large-123b, mistral-small-3.1-24b, and qwen-3.5-122b.

Paper

• Manuscript: "Can LLMs Imagine Moral Alternatives Beyond Binary Dilemmas?"
• Authors: Jongchan Choi, Nari Yang, Sung Soo Park, Jaemin Cho, Han Seoyoung, Haerin Shin, and Jun-Hyung Park
• Conference: Accepted to Findings of EMNLP 2026

Data organization

The repository uses the following high-level structure:

data/
  advisor_all_abcd_judgment_156.csv
  advisor_alternatives_generation_75.csv
  agent_abcd_judgment_dataset_151.csv
  agent_alternatives_generation_89.csv

Each CSV is configured as an independent subset in the Dataset Viewer. The advisor-judgment subset is the default.

Loading the dataset

from datasets import load_dataset


dataset = load_dataset("jongchanch/MoralAltDataset", "advisor-judgment")

Replace advisor-judgment with advisor-alternatives, agent-judgment, or agent-alternatives to load another subset. Authentication is required while this repository remains private.

Citation

@misc{choi2026moralalt,
  title  = {Can LLMs Imagine Moral Alternatives Beyond Binary Dilemmas?},
  author = {Jongchan Choi and Nari Yang and Sung Soo Park and Jaemin Cho and Han Seoyoung and Haerin Shin and Jun-Hyung Park},
  year   = {2026},
  eprint={2606.31213},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2606.31213}, 
}