LegalRAG

Legal case outcome prediction requires more than simple text classification. A strong system must understand the legal dispute, identify the claims of the parties, retrieve relevant factual evidence from the case record, retrieve applicable legal provisions, and reason about how the court is likely to resolve the dispute.

This task aims to encourage research on:

• Vietnamese legal judgment understanding.

• Retrieval-augmented legal reasoning.

• Agentic interaction with legal APIs.

• Evidence-grounded legal prediction.

• Vietnamese legal corpus retrieval.

• Transparent and verifiable legal AI systems.

The task setting reflects a realistic legal AI scenario: a system receives an initial case description, then must actively retrieve additional information before making a prediction.

So, this is a Retrieval-Augmented Generation (RAG) system for it, built with FastAPI, Pinecone, and NVIDIA NIM's Llama3.2 8B Instruct.

Overview

For each test instance, participants are given a short natural-language query describing the dispute. The query includes the main parties, the disputed legal relationship or asset, a brief summary of the plaintiff's claim and the defendant's position, and a question asking whether the plaintiff or the defendant is more likely to win.

Participants must build a system that:

1. Reads the provided case query.

2. Calls the official Case Content API to retrieve relevant case segments.

3. Retrieves relevant legal provisions from the provided law corpus.

4. Predicts the final outcome of the case.

5. Submits the predicted outcome together with supporting case evidence and legal provisions.

The competition contains only one task:

Legal Case Outcome Prediction with Evidence Retrieval The expected prediction is one of four labels:

• A_WIN: The court fully accepts all of the plaintiff's claims.

• PARTIAL_A_WIN: The court partially accepts the plaintiff's claims, and the accepted portion is greater than 50%.

• PARTIAL_B_WIN: The court partially accepts the plaintiff's claims, but the accepted portion is 50% or less.

• B_WIN: The court fully rejects all of the plaintiff's claims.

System Architecture & Pipeline

                                +-------------------+
                                |    Case Query     |
                                +---------+---------+
                                          |
                                          v
                   +----------------------+----------------------+
                   |    Semantic Splitting (POS Tagging)         |
                   |          (semantic_splitting.py)            |
                   +----------------------+----------------------+
                                          |
                                          v (Sub-queries)
           +-----> +----------------------+----------------------+
           |       |              Case Content API               |
           |       +----------------------+----------------------+
           |                              |
           |                              v (Raw Segments)
           |       +----------------------+----------------------+
    (Has gaps? &   |           Segment Chaining Module           |
    < Threshold)   |   (Deduplication, Overlap, Ordering, Merge) |
           |       +----------------------+----------------------+
           |                              |
           +------------------------------+
                                          | (Threshold met or No gaps)
                                          v
                            Coherent Case Evidence & Report
                                          |
                  +-----------------------+-------------+
                  |                                     |
                  v                                     v
              +-------------------+     +-------------------------------+
              |     Base RAG      |     |      Multi-Query RAG          |
              |   (fact2law.py)   |     |     (mulquery2law.py)         |
              |                   |     |                               |
              | Report -> BM25 +  |     | Report -> QueryGen(LLM) ->    |
              |   BamiBERT ->     |     |   BM25 + BamiBERT ->          |
              |   Top Candidates  |     |   Sub-query search            |
              |                   |     |                               |
              | Strength:         |     | Strength:                     |
              |   High Recall     |     |   High Precision              |
              +--------+----------+     +--------------+----------------+
                       |                               |
                       +---------------+---------------+
                                       v
                          Fusion Retriever (RRF k=40)
                            (fusion_retriever.py)
                                       |
                                       v
                               Fused Candidates
                                       |
                                       v
                   +--------------------------------------+
                   |              LLM Select              |
                   |            (llm_select.py)           |
                   |                                      |
                   |  - Final Law Selection & Evaluation  |
                   |  - Filters irrelevant laws           |
                   +-------------------+------------------+
                                       |
                                       v
                             Top Law Provisions
                                       |
                                       v
                   +--------------------------------------+
                   |       Legal Case Predictor           |
                   |         (llm_routing.py)             |
                   |                                      |
                   |  - Dynamic Routing (Vanilla vs RAG)  |
                   |  - Majority Voting (5 votes)         |
                   |  - Base: meta/llama-3.1-8b-instruct  |
                   +-------------------+------------------+
                                       v
                                Final Judgment

Installation

2. Create a virtual environment

python -m venv venv
# On Windows
venv\Scripts\activate
# On macOS/Linux
source venv/bin/activate

3. Install dependencies

pip install -r requirements.txt

---

Configuration

Create a .env file:

cp .env.example .env

and add the following variables:

PINECONE_API_KEY=your_pinecone_api_key
INDEX_NAME=your_pinecone_index_name
ALQAC_TOKEN=alqac_xxxxxxxxxxxxxxxxxxxxxxxx

---

Execution & Demo

1. Run Full Pipeline via CLI

To run the end-to-end pipeline on the private test set (60 cases):

# Activate your virtual environment
venv\Scripts\activate  # Windows
source venv/bin/activate  # Linux/Mac

# Execute the pipeline script
bash run_private_pipeline.sh

2. Run Live UI Dashboard

To explore the cases and run the LLM prediction pipeline interactively:

streamlit run app.py

---

Results

• Private Test Set (60 cases): Accuracy = 41.7% (25/60 cases)

License

This project is open-source under the MIT License.