---

🚀 Goal:
This project aims to improve public service efficiency by using GenAI-powered document processing. The system can:

• Extract and clean text/images from documents (PDFs).
• Retrieve relevant information using RAG (Retrieval-Augmented Generation).
• Enable voice interaction for visually impaired users.
• Describe images for accessibility.

De Facto drastically reduces magistrates' factual review time of criminal cases, thereby enhancing the efficiency and quality of justice for our fellow citizens. Its generic backend is adaptable across the entire public sector and includes a voice interaction mode for visually impaired agents.

💡 A sovereign, trustworthy, and efficient AI, leveraging frugal open-source models!

---

Backend

⚙️ Configuration (config.py)

The config.py file contains all the key parameters for the pipeline, ensuring flexibility and easy adjustments. Note that all the parameters are not used in the MVP version.

🔹 What Does It Contain?

🔹 Example config.py File

TEXT_EXTRACTION_MODEL = "stepfun-ai/GOT-OCR-2.0-hf"
IMAGE_EXTRACTION_MODEL = "vidore/colpali-v1.3"
EMBEDDINGS_MODEL = "intfloat/multilingual-e5-small"
RERANKING_MODEL = "BAAI/bge-reranker-v2-m3"
CHAT_MODEL = "AgentPublic/llama3-instruct-8b"
MULTIMODAL_MODEL = "mistralai/Pixtral-12B-2409"
SPEECH_TO_TEXT_MODEL = "openai/whisper-large-v3"
TEXT_TO_SPEECH_MODEL = "hexgrad/Kokoro-82M"

SERVER_URL = "http://yourserver:port/v0/"

CHUNK_SIZE = 512
OVERLAP_SIZE = 128

FAISS_INDEX_PATH = "vector_store/index.faiss"

ENABLE_IMAGE_RETRIEVAL = True
ENABLE_TEXT_TO_SPEECH = False

---

📂 Repository Structure

1️⃣ Core Scripts (/scripts/)

---

2️⃣ Pipeline Files

---

3️⃣ Utility Functions (/utils/)

---

4️⃣ Directories

---

🛠️ Installation

1️⃣ Build images

• OCR API

cd ocr && docker build -t got:v0.1.0 .

• QA API

cd qa && docker build -t qa:v0.1.0 .

2️⃣ Deploy

docker-compose up -d 

Export endpoints:

export QA_URL=http://localhost:7042/v0/ask
export GOT_URL=http://localhost:7041/v0/extract

2️⃣ Lauch Defacto

• Install poetry

curl -sSL https://install.python-poetry.org | python3 - 
export PATH="/root/.local/bin:$PATH"

• Install Defacto

poetry install

• Launch Defacto

poetry run python3 frontend/ui.py

Open localhost:7860 in your web browser to interact with Defacto assistant.

3. TODO

Colpali: deploy the Colpali retriever

cd colpali
docker build -t colpali:v0.1.0 .
docker save colpali:v0.1.0 -o colpali_v0.1.0.tar
scp colpali_v0.1.0.tar user@your-server:/path_in_your_server
cd /path_in_your_server
docker load -i colpali_v0.1.0.tar
docker-compose up -d colpali

Then, in the code:

• Modify the url to embed all the dataset with http://yourserver:port/v0/visual_embed

• Modify the url to return the oracle page based on a query with http://yourserver:port/v0/search

---

🚀 Usage

Run Inference (Document Retrieval & Questions Answering)

python main.py

---

Frontend

🛠️ Installation

1️⃣ Install Python3 (skip if backend installed)

Ensure you have Python 3.8+ installed. To install it, please run:

• apt update
• apt install python3
• apt install python3-pip

Note: you may need to run above commands with root access (then prefix above commands with sudo).

2️⃣ Virtual environment

It is advised to use a python virtualenv to manage dependencies. To set it up, please run:

pip install virtualenv

Then you will create a virtual environment to sandbox your dependencies installation to the defacto project:

python3 -m venv virtualenv

Then you can enter this virtualenv:

source virtualenv/bin/activate

Note: at this point your CLI prompt will be prefixed with (virtualenv) string.

3️⃣ Install Dependencies

When your virtual environment is up, you can install dependencies from the frontend folder via:

pip install -r requirements.txt

The dependencies and their versions are listed in the frontend/requirements.txt file.

To quit the virtualenv, simply enter deactivate in the CLI. The CLI will not be prefixed with (virtualenv) anymore.

Note: Do not commit virtualenv directory as it can be generated with requirements.txt and instructions from this readme (moreover "virtualenv" line must have been added to your .gitignore file).

🚀 Usage

• Activate the virtual env: source virtualenv/bin/activate
• Run the front file: python3 ui.py.
• To quit the python process, you can Ctrl+C on the CLI.
• To quit the virtualenv, run deactivate command in the CLI. The CLI will not be prefixed with (virtualenv) anymore.

---

💡 How Does the System Work?

The features included in the MVP (Most Valuable Product) are highlighted.

🔹 1. Document Processing

• Extracts text and images from PDFs (either text-based or image-based PDFs). MVP
• Extract text through OCR. MVP
• Apply heuristics to split the text into consistent documents types. MVP
• Cleans and chunks text for efficient retrieval.
• Stores embeddings in FAISS for fast semantic search.

🔹 2. Context-only Prompting

• Summarizes the document automatically by prompting a LLM. Depending on the model used, the whole document text is passed in context or a map-reduce strategy applies. MVP
• Generate answers to user queries. This can be done through context-only prompting (MVP) or RAG.

🔹 3. Retrieval-Augmented Generation (RAG)

• Finds the most relevant document snippets using vector search.
• Uses an LLM to generate an answer based on retrieved context.
• Can return image descriptions for accessibility.

🔹 4. Voice & Accessibility Features

• Supports speech-to-text (voice input) using Whisper.
• Converts AI-generated responses to speech for accessibility. MVP
• Uses multimodal AI to describe figures/images for blind users.

---

📢 Acknowledgments

Built for GenAI for Public Good, focused on GenAI-powered public service improvements.

---

🚀 Let's build AI for efficiency and accessibility!