VeriFoam

VeriFoam is an evidence-based diagnosis assistant for OpenFOAM cases. It reads a user's case files and optional solver log, identifies likely setup or runtime problems, and returns a structured diagnosis with cited evidence, confidence, and next checks. This repository is the final project implementation for CSC 7644: Applied LLM Development.

The project is intentionally focused on diagnosis, not automatic case repair. A failed OpenFOAM run often leaves useful evidence scattered across system/controlDict, system/fvSchemes, system/fvSolution, 0/ field files, and long solver logs. VeriFoam organizes that evidence and uses a local LLM pipeline to make the failure mode easier to inspect.

Key Features

• Upload an OpenFOAM case archive through a React web interface.
• Optionally provide a solver log file or pasted log text.
• Parse case files into evidence chunks with file and line metadata.
• Run a Gemma-based local diagnosis pipeline through llama.cpp.
• Use curated diagnostic cards and few-shot examples for common OpenFOAM failure categories.
• Add deterministic static checks for missing dictionaries, missing required sections, and boundary-field inconsistencies.
• Display benchmark results for controlled OpenFOAM failure cases.
• Cache completed diagnoses locally so repeated uploads do not rerun inference unnecessarily.

Architecture

VeriFoam has four main parts:

• ui/frontend/ contains the React + Vite interface.
• ui/backend/ contains the FastAPI service that accepts uploads, tracks jobs, runs static linting, and calls the diagnosis pipeline.
• scripts/ contains the core parsing, inference, calibration, and benchmark scripts.
• data/ contains diagnostic cards, few-shot support data, knowledge-base files, benchmark cases, labels, and evaluation outputs.

The default inference route uses Gemma-4 E4B-it Q4_K_M as a local GGUF model through llama-cpp-python. The project does not require an external API key for the default local workflow.

Setup

Prerequisites

• Linux, macOS, or WSL on Windows.
• Python 3.11 or newer.
• Node.js and npm. The UI was developed with Node 23 and npm 11.
• A local GGUF model file for the inference backend.

The code expects the default GGUF model at:

~/.cache/gguf/gemma-4-E4B-it-Q4_K_M.gguf

The model file is intentionally not committed to this repository.

Backend

From the repository root:

python3 -m venv .venv
source .venv/bin/activate
pip install -r ui/backend/requirements.txt

If you want to run the optional Hugging Face / Transformers backend instead of the GGUF backend, install the additional packages used by that path:

pip install torch transformers accelerate sentencepiece

Frontend

cd ui/frontend
npm install

Running the Application

Start the backend:

source .venv/bin/activate
cd ui/backend
python -m uvicorn app.main:app --reload --port 8000

In a second terminal, start the frontend:

cd ui/frontend
npm run dev -- --port 5173

Open the app at:

http://localhost:5173

The backend API documentation is available at:

http://localhost:8000/docs

To try the interface quickly, use one of the sample zip files in ui/testing_cases/.

Running Evaluations

The expanded benchmark summary is already included under data/benchmark_expanded/. To inspect or rerun parts of the evaluation pipeline, use the scripts in scripts/.

Examples:

python scripts/test_card_retrieval.py
python scripts/run_expanded_benchmark.py --backend llama-cpp
python scripts/final_decision_controller.py

The final reported benchmark used 21 items: 13 controlled broken cases, 4 control or insufficient-evidence cases, and 4 supplemental public-log validation items. The final decision controller reached 100% top-1 accuracy on the controlled broken cases and 85.7% top-1 accuracy over all combined items.

Repository Organization

data/
  benchmark/              Original benchmark cases and generated chunks.
  benchmark_expanded/     Final benchmark labels, reports, and diagnoses.
  diagnostic_cards/       Curated OpenFOAM diagnostic cards.
  knowledge_base/         OpenFOAM knowledge-base JSONL files.
  web_real_logs/          Reviewed public-log examples and summaries.

examples/few_shot/        Per-category few-shot examples.

scripts/
  parser.py               Case parser and chunk builder.
  llm_infer.py            Structured LLM diagnosis pipeline.
  calibrate_confidence.py Confidence calibration helpers.
  aggregate_expanded_benchmark.py
                          Final benchmark metric aggregation.
  build_base_case_manifest.py
                          Rebuilds expected-file metadata for file inventory checks.
  build_public_val_chunks.py
                          Rebuilds public-log validation chunks.
  final_decision_controller.py
                          Deterministic final-category and abstention logic.
  run_expanded_benchmark.py
                          Batch benchmark runner.
  test_card_retrieval.py  Diagnostic-card retrieval smoke test.

ui/
  backend/                FastAPI backend.
  frontend/               React + Vite frontend.
  testing_cases/          Small zipped cases for manual UI testing.

Secrets and Local Files

The default project path uses a local model and does not need API keys. Do not commit .env files, model weights, cache folders, uploaded job files, or generated frontend builds. Large model files such as .gguf, .safetensors, .pt, and .bin should stay outside the repository.

Attributions and Citations

This implementation is original project code. External resources were used for domain grounding, data construction, and validation:

• OpenFOAM documentation and source code were used to verify error messages and expected case-file structure.
• The FoamGPT OpenFOAM dataset informed the knowledge-base construction and is cited in the project report.
• Public OpenFOAM troubleshooting examples were summarized for supplemental validation; they are not treated as full reproducible benchmark cases.
• The UI and backend use standard open-source libraries including React, Vite, FastAPI, scikit-learn, and llama-cpp-python.

The repository does not intentionally include private API keys, proprietary solver results.