Durham-Jazan at Daleel 2026

This repository accompanies the Durham-Jazan system description paper for the Daleel 2026 shared task on Arabic Argumentative Discourse Mining (ArabicNLP 2026, held with EMNLP 2026). It contains our code, our synthetic training data, and the scripts behind every figure and analysis in the paper.

Paper: Durham-Jazan at Daleel 2026: Domain-Routed Ensembles, LLM Augmentation, and the Limits of Decision-Rule Tuning for Arabic Argument Mining. Amal Saad Alshehri, Nelly Bencomo, Housam Babiker, and Amir Atapour-Abarghouei.

The central finding of our participation is summarised in one figure. Changes to the model or its data delivered what validation promised, while tuning of the decision rules on top did not.

Two further findings come from post-submission analysis. Per-label performance on editorials tracks how much of that label's training data came from editorials, which explains the editorial deficit every team suffered, and threshold recalibration gains measured on the tuning half do not survive on held-out data.

A short demonstration

The paper introduces segment-concatenation generation, a recipe that lets a large language model produce span-annotated training data with exact character offsets and no alignment step. The following command builds and verifies one such paragraph from the data in this repository.

python demo_segment_concat.py

Final official results

Taken from the organisers' official results sheet. Ed and Db are the editorial and debate columns. The entrant count excludes the organisers' baseline.

In the development phase we finished first on both Open leaderboards. The evaluation-phase ranks above are the final official standings and include submissions made after ours.

Setup

pip install -r requirements.txt

The official task data and scoring scripts are not redistributed here. They are available from the organisers through the task pages at https://qatardebate.org/programs/academic-programs/daleel2026-shared-task and CodaBench. Place train_task_1.jsonl, train_task_2.jsonl, dev_in.jsonl, dev_task_1_ref.jsonl, dev_task_2_ref.jsonl, and related files under data/.

The scripts in paper_analysis/ and src/ resolve their paths against the repository root, so they can be run from any working directory and will look for their inputs in the data/, oof/, and preds/ layout used in the paper. DALEEL_ROOT overrides the repository root as a whole rather than the data directory alone, so point it only at a tree that also holds oof/, preds/ and configs/. To keep the organisers' files elsewhere, symlink them into data/ instead:

export DALEEL_ROOT=/path/to/a/full/copy/of/this/repository

Two further dependencies are on the organisers' Task 2 scorer, which we do not redistribute. Five scripts in src/ (ensemble_task2.py, ensemble_task2_camel.py, ensemble_task2_marbert.py, t2_open_eval.py, train_task2_cv.py) import it as task2_scoring, so place the organisers' task2_scoring.py inside src/. Three others (t2_blend_build.py, t2_blend_oof_exp.py, t2_closed_recal.py) invoke it as a file and read its location from DALEEL_SCORER:

export DALEEL_SCORER=/path/to/task2_scoring.py

What is included alongside the code

oof/ contains the model outputs behind the paper's analyses: out-of-fold probabilities and character offsets, and the saved seed-curve and genre-transfer results. These are our own numbers and carry no corpus text, so they are released in full.

preds/ holds our development-set predictions. The Task 1 files are label sets only. The Task 2 files carry labels and character offsets with the span text removed, because that text is the organisers' corpus and is theirs to distribute. Removing it changes no result: every analysis below works from labels and offsets.

Synthetic data

The data/ directory holds the synthetic training data generated with a proprietary large language model. All of it is machine-generated: the statistics inside synthetic ST spans are plausible but invented, so the files are unsuitable as factual text. It was used in training only, and thresholds and all held-out evaluation used real data exclusively. For Task 2 the paragraphs were authored as ordered labelled segments and joined with single spaces, so the character offsets are exact by construction (paper, Section 3.3).

The four systems at a glance (solid boxes are the Closed track, dashed the Open-track additions):

Post-submission analyses

The paper_analysis/ scripts are the code behind the paper's figures and appendix tables. Run them from anywhere, for example:

python paper_analysis/threshold_transfer.py

Six of them need nothing beyond this repository:

Three more need only the organisers' Task 1 files in data/, because the out-of-fold predictions they run on (oof/t1_recal_oof_closed.npy) are included here:

Four more need the organisers' files in data/, together with our dev predictions, which are now included under preds/:

The last four need more than this repository provides:

The dev predictions under preds/ are our own model outputs rather than task data. The Task 1 files are label sets. The Task 2 files keep labels and character offsets but not the span text, which belongs to the organisers' corpus. No analysis here reads that field, and removing it leaves every reported number unchanged. Figure output goes to figs/, and the Arabic examples to paper_analysis/.

Reproducing the final systems

The two Task 1 entries cannot be reproduced here. Their editorial predictions are the union of three Qwen3-32B seeds, and t1_llm_sft.py trains and predicts one seed at a fixed seed value, so the seed runs, the union, and the routing step are not released. The Task 1 Open span fusion is released as rule configurations only. The union rule can still be checked on the retained holdout predictions with paper_analysis/union_holdout_check.py.

The configs/ directory holds the evaluation-phase decoding thresholds and rule configurations referenced in the paper's Appendix A: the recalibrated Task 1 sets (t1_recal_ths_closed.json, t1_recal_ths_open.json), the routed Task 2 sets (task2_camel_ens.json for editorials, task2_marbert_deb.json for debates), the Open blend (t2_blend_cfg.json), and the fusion and per-label rules. The development-phase Task 1 thresholds quoted in the appendix are the BASE constant of paper_analysis/threshold_transfer.py (Closed) and the THS constant of src/test_task1_open.py (Open). Validation utilities include clean_holdout.py, model_search.py, dapt_eval.py, rare_aug_eval.py, and ens_check.py. PROMPTS.md documents the fine-tuning prompt and the synthetic-data generation templates.

Hardware

One RTX 4080 SUPER with 16 GB served all encoder work and models of up to 14B parameters. One A100 with 80 GB ran Qwen3-32B for about six GPU hours.

Licence

The code is released under the MIT licence (LICENSE) and the synthetic data under CC-BY-4.0 (LICENSE-DATA). The shared-task data released by the organisers is not redistributed here and remains subject to their terms. The official task data belongs to the Daleel 2026 organisers and is not included.

Citation

@inproceedings{durham-jazan-daleel2026,
  title  = "Durham-Jazan at Daleel 2026: Domain-Routed Ensembles, LLM Augmentation, and the Limits of Decision-Rule Tuning for Arabic Argument Mining",
  author = "Alshehri, Amal Saad and Bencomo, Nelly and Babiker, Housam and Atapour-Abarghouei, Amir",
  booktitle = "Proceedings of ArabicNLP 2026",
  year   = "2026"
}