🏛️ WangchanX-Legal-ThaiCCL-RAG

[Technical Report]

The WangchanX-Legal-ThaiCCL-RAG dataset supports the development Retrieval-Augmented Generation (RAG) for Thai Legal question answering. This dataset is allows developers to finetune both retrieval model - to better retrieve relevant law section, and Large Language Model (LLM) - for instruction tuning. Our dataset supports Corporate and Commercial Law (thus ThaiCCL name). See legislation section for more details on supported legislations.

Training Set:

The dataset includes 35 legislations covering various laws, such as the Civil and Commercial Code, Securities and Exchange Act, and Petroleum Income Tax Act, etc.

The procedure for collecting the training data was as follows:

1. We sampled a single law section from 35 legislations.
2. We prompt Gemini 1.5 Pro to generate a legal question given the sampled law section in the prompt.
3. We retrieve a semantically relevant law sections using BGE-M3, marked as positives
4. Legal experts are then reviewed and tasked to 1) remove irrelevant law sections (removed law sections are treated as hard_negative_contexts) 2) rerank relevant law sections to put most relevant law section on top.
5. Answers were then generated using Meta-Llama-3-70B, with Claude-3 Sonnet assisting for Thai responses when necessary.
6. Legal experts finally reviewed and edit the answers if necessary to ensure accuracy. If the legal expert edited the answer, we kept the old response as hard_negative_answer.

Given the curated approach, the training samples contain multiple positives.

Test Set:

The test set was curated using expert-formulated legal questions and answers containing only 21 major legislations. The test set was fully annotated manually using Legal experts. The data collection procedure was outlined as follows:

1. We sampled a single law section from 35 legislations.
2. Legal expert was tasked to create a question, answer pair based on the sampled law section.
3. Another independent group of legal experts are then tasked to either discard the data point if not accurate.

Unlike training set which contains multiple positives, the test set only contains a single positive law section. Additionally, there's no hard negative contexts and hard negative answers for the test set.

Data Fields

Legislations

Citation

@misc{akarajaradwong2025nitibenchcomprehensivestudiesllm,
      title={NitiBench: A Comprehensive Studies of LLM Frameworks Capabilities for Thai Legal Question Answering}, 
      author={Pawitsapak Akarajaradwong and Pirat Pothavorn and Chompakorn Chaksangchaichot and Panuthep Tasawong and Thitiwat Nopparatbundit and Sarana Nutanong},
      year={2025},
      eprint={2502.10868},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2502.10868}, 
}

License

This dataset is provided under the MIT License.

Acknowledgement

We sincerely appreciate the generous support from the WangchanX program sponsors—PTT, SCB, and SCBX—whose funding made this project possible. We are also grateful for the invaluable collaboration with VISTEC, which was crucial in bringing this project to fruition.

---