ARECCA — Automated Real Estate Contract Compliance Auditor

ARECCA ingests commercial real estate lease PDFs, extracts structured terms via a legal-fine-tuned LLM, validates rent math deterministically (never trusts the LLM with numbers), flags compliance violations against state laws, and indexes everything for hybrid semantic search.

---

Architecture

                        ┌──────────────────────────────┐
                        │   Chainlit Chat UI (8501)     │
                        │   or HTTP Client (8000)       │
                        └──────────┬───────────────────┘
                                   │
                        ┌──────────▼───────────────────┐
                        │   FastAPI + LangGraph App     │
                        │   src/api/main.py             │
                        └──────────┬───────────────────┘
                                   │
              ┌────────────────────┼────────────────────┐
              │                    │                     │
     ┌────────▼────────┐  ┌───────▼────────┐  ┌────────▼────────┐
     │  Input Guardrail │  │  Local Storage │  │  PostgreSQL     │
     │  (injection,     │  │  (data/uploads)│  │  (results)      │
     │   malware scan)  │  └────────────────┘  └─────────────────┘
     └────────┬────────┘
              │
     ┌────────▼─────────────────────────────────────────────┐
     │           LangGraph StateGraph Pipeline               │
     │                                                       │
     │  ┌──────────┐   ┌─────────┐   ┌────────────────┐     │
     │  │  Ingest   │──►│  Chunk  │──►│  Extract       │     │
     │  │(PDFReader)│   │(Semantic│   │(HF Legal LLM + │     │
     │  │          │   │ Splitter)│   │ ContextAssembly)│     │
     │  └──────────┘   └─────────┘   └────────┬───────┘     │
     │                                         │             │
     │  ┌──────────┐   ┌─────────┐   ┌────────▼───────┐     │
     │  │  Report  │◄──│ Output  │◄──│  Math Validate │     │
     │  │          │   │ Guardrail│   │  (NumPy, no LLM)│     │
     │  └──────────┘   └─────────┘   └────────┬───────┘     │
     │                                         │             │
     │  ┌──────────┐   ┌─────────┐            │             │
     │  │  Index   │◄──│Compliance│◄───────────┘             │
     │  │ (Qdrant) │   │(Rule Eng)│                         │
     │  └──────────┘   └─────────┘                          │
     └──────────────────────────────────────────────────────┘
                                   │
                    ┌──────────────┴──────────────┐
                    │                              │
           ┌────────▼────────┐          ┌──────────▼───────┐
           │ Qdrant Cloud    │          │  Entity Memory   │
           │ (Hybrid Search) │          │  + Conversation  │
           │ dense + BM25    │          │  Summary (Gemini)│
           │ alpha=0.3       │          │                  │
           └─────────────────┘          └──────────────────┘

5-Phase Pipeline

---

Tech Stack

---

Quick Start

Prerequisites

• Python 3.11+
• PostgreSQL 15+
• Qdrant Cloud account (or local Qdrant: docker run -p 6333:6333 qdrant/qdrant)
• (Optional) Gemini API key for conversation summaries
• (Optional) GPU for local LLM inference; falls back to CPU

Local Development

# 1. Clone & enter
cd arecca

# 2. Environment
cp .env.example .env
# Edit .env with your keys (DATABASE_URL, QDRANT_URL, QDRANT_API_KEY, etc.)

# 3. Install
make install

# 4. Run the API
make dev
# → http://localhost:8000
# → health: http://localhost:8000/health

# 5. Run the Chainlit UI (separate terminal)
chainlit run src/ui/app.py --port 8501
# → http://localhost:8501

Docker

# Build
docker build -t arecca:latest .

# Run
docker run -p 8000:8000 \
  -v $(pwd)/.env:/app/.env \
  -v $(pwd)/data:/app/data \
  arecca:latest

Railway Deployment

1. Push to GitHub
2. Connect repo to Railway
3. Add PostgreSQL plugin (Railway injects DATABASE_URL automatically)
4. Set env vars: GEMINI_API_KEY, QDRANT_URL, QDRANT_API_KEY, STORAGE_PATH
5. Deploy — Railway auto-detects the Dockerfile

---

Chainlit UI

The Chainlit UI provides a ChatGPT-style interface on port 8501.

chainlit run src/ui/app.py --port 8501

Features:

• Upload & Audit — Click the upload button, select a PDF, the full pipeline runs automatically
• Document History — Browse previously audited documents with status indicators
• Search — Natural language search across indexed lease clauses
• Audit Results — Formatted extraction, math validation, and compliance report with risk level icons

---

API Reference

POST /api/v1/upload

Upload a lease PDF for auditing.

curl -X POST http://localhost:8000/api/v1/upload \
  -F "[email protected]"

{
  "document_id": "uuid-...",
  "filename": "lease.pdf",
  "status": "processing",
  "message": "Document uploaded and queued for audit"
}

POST /api/v1/audit/{document_id}

Run the full 5-phase LangGraph audit pipeline.

curl -X POST http://localhost:8000/api/v1/audit/uuid-...

{
  "document_id": "uuid-...",
  "filename": "lease.pdf",
  "status": "completed",
  "extraction": { "lease_terms": { ... }, "confidence_score": 0.94 },
  "math_validation": { "is_valid": true, "details": { ... } },
  "compliance_report": {
    "overall_risk_level": "medium",
    "flag_count": 1,
    "flags": [ { "rule_id": "GRACE_PERIOD", "risk_level": "medium", ... } ]
  }
}

POST /api/v1/search

Hybrid semantic search over ingested lease clauses.

curl -X POST http://localhost:8000/api/v1/search \
  -H "Content-Type: application/json" \
  -d '{ "query": "late fee grace period", "top_k": 5, "alpha": 0.3 }'

{
  "query": "late fee grace period",
  "results": [
    { "section_title": "Section 3: Late Fee...",
      "content": "A late fee of 5% applies after 10...",
      "score": 0.87,
      "page_number": 4 }
  ]
}

GET /api/v1/documents

List all audited documents.

GET /api/v1/documents/{document_id}

Get audit result for a specific document.

GET /health

Liveness probe.

---

Configuration

.env (secrets)

configs/config.yaml (runtime tuning)

llm:
  model: AdaptLLM/law-llm-7b           # HuggingFace model ID
  device: auto                          # cpu / cuda / auto
  load_in_8bit: true                    # memory optimization

gemini:
  model: gemini-2.0-flash               # Gemini model for summaries
  temperature: 0.0
  max_tokens: 4096

chunking:
  strategy: semantic                    # semantic | sentence
  semantic_buffer_size: 3
  semantic_breakpoint_percentile: 85

retrieval:
  hybrid_search: true
  alpha: 0.3                            # 0 = dense only, 1 = sparse only

validation:
  rounding_decimals: 2
  tolerance: 0.01                       # $0.01 discrepancy threshold

context:
  target_context_tokens: 8000           # context window budget

prompts:
  version: v1
  hot_reload: false                     # set true during prompt development

---

Project Structure

src/
├── ui/
│   └── app.py                Chainlit chat UI (port 8501)
├── api/
│   ├── main.py               FastAPI app + lifespan (tracing, llamaindex, DB)
│   ├── routes.py             /upload, /audit, /search, /documents
│   └── schemas.py            Pydantic request/response models
├── graph/
│   ├── graph.py              StateGraph compiler (10 nodes, conditional edges)
│   ├── state.py              AgentState TypedDict
│   ├── nodes.py              Pipeline nodes (guardrail → ingest → ... → report)
│   ├── edges.py              Conditional routing (blocked → rejection, errors → end)
│   ├── guardrails.py         Input (ext/malware/injection) + output (confidence/risk)
│   └── tools.py              NumPy @tool for rent schedule validation
├── ingestion/
│   ├── llamaindex_pipeline.py    PDFReader + SemanticSplitterNodeParser
│   └── storage.py                Local filesystem storage (save_file, copy_file, delete_file)
├── extraction/
│   ├── schemas.py            Pydantic: LeaseTerms, RentSchedule, ExtractionResult
│   └── hf_extractor.py       HuggingFace pipeline + ContextAssembler
├── validation/
│   └── math_validator.py     Decimal-based rent math (backup for tests)
├── compliance/
│   ├── rules.py              13 compliance rules (late fee, deposit, grace, etc.)
│   └── engine.py             Rule engine → ComplianceReport
├── vectordb/
│   └── qdrant_store.py       Qdrant client + collection + hybrid store
├── retrieval/
│   └── hybrid_retriever.py   LlamaIndex retriever (alpha=0.3, hybrid mode)
├── database/
│   ├── models.py             SQLAlchemy: LeaseDocument, Extraction, Validation, Flag
│   └── session.py            AsyncSession factory
├── memory/
│   ├── entity_memory.py      KV facts per document (JSON + Fernet)
│   └── conversation.py       Rolling summary via Gemini
├── core/
│   ├── config.py             Pydantic Settings + YAML merge
│   ├── context_assembler.py  Prompt + entity_memory + summary + token trim
│   ├── prompt_manager.py     Versioned prompt loader from prompts/ tree
│   ├── token_counter.py      tiktoken cl100k_base
│   ├── tracing.py            LangSmith opt-in
│   ├── llamaindex_setup.py   Centralised embed model init
│   └── logging.py            structlog JSON/console
├── configs/config.yaml        Runtime configuration
├── prompts/
│   ├── extraction/            Lease extraction prompt (v1)
│   ├── compliance/            Compliance rules reference
│   └── summarizer_system.md   Rollup summarizer system prompt
└── tests/                     ~44 tests (pytest)
    ├── test_guardrails.py
    ├── test_guardrail_efficiency.py
    ├── test_graph_tools.py
    ├── test_compliance.py
    ├── test_math_validator.py
    ├── test_chunker.py
    ├── test_storage.py
    ├── test_parser.py
    └── test_llamaindex_pipeline.py

---

Testing

# Run all tests
make test

# Or directly
python -m pytest tests/ -v

# Run a specific module
python -m pytest tests/test_math_validator.py -v
python -m pytest tests/test_guardrail_efficiency.py -v

Test coverage:

---

Key Design Decisions

---

GitHub Actions CI / Deploy

The project includes a CI/CD pipeline in .github/workflows/deploy.yml:

• CI — Runs on every PR/push to main: lint (ruff), type check (mypy), tests (pytest)
• Docker — Builds the Docker image with BuildKit caching
• Deploy — Deploys to Railway via the official GitHub Action

Required secrets:

• RAILWAY_TOKEN — Railway API token
• RAILWAY_PROJECT_ID — Railway project ID
• RAILWAY_SERVICE_ID — Railway service ID