Neuron Analysis — Multimodal AI Platform for Brain Diagnostics

Neuron Analysis is an end‑to‑end platform for AI‑assisted analysis of brain data. It combines medical imaging (MRI), EEG signals, and clinical text to support two diagnostic workflows — Alzheimer's disease and epilepsy/seizure detection — and adds an interactive Brain Assistant with real‑time brain simulation powered by The Virtual Brain (TVB).

⚠️ Research use only. This project is a research and engineering prototype. It is not a certified medical device and must not be used for real clinical decision‑making.

---

What it does

A request is automatically routed to the right workflow based on the data attached to the study: MRI present → Alzheimer's, EEG present → epilepsy.

---

How it works (high level)

flowchart TD
    U[Upload MRI / EEG / clinical notes]

    subgraph FE [Frontend · Next.js]
        FEc[Dashboard · Brain Assistant · 3D visualization Three.js]
    end

    subgraph BE [Backend · FastAPI]
        BEc[Auth · Studies · WebSocket streaming]
    end

    AG[Multi-agent core<br/>orchestrator + specialists]
    DX[Diagnostic models<br/>ClinicalBERT · ResNet3D · CNN-Transformer]
    TVB[TVB simulation<br/>The Virtual Brain neural dynamics]

    subgraph DATA [Data & infra layer]
        DATAc[PostgreSQL · Redis · Celery workers · S3 / R2 object storage]
    end

    U --> FE
    FE -- REST + WebSocket --> BE
    BE --> AG
    BE --> DX
    BE --> TVB
    AG --> DATA
    DX --> DATA
    TVB --> DATA

The orchestrator agent receives a request, validates the study, and dispatches work to specialist agents (imaging, EEG, diagnostic, simulation, report). Heavy jobs run asynchronously through Celery + Redis, and progress is streamed back to the UI over a WebSocket.

---

Repository layout

neuron-analysis/
├── agents/                 # Multi-agent system (orchestrator + specialists)
│   ├── orchestrator/       #   Routes requests & coordinates the pipeline
│   ├── imaging/            #   MRI segmentation, connectivity, visualization
│   ├── eeg/                #   EEG feature extraction & spike detection
│   ├── diagnostic/         #   Diagnosis inference & clinical prompts
│   ├── simulation/         #   Brain simulation (TVB, Kuramoto, biophysical)
│   └── report/             #   Clinical report generation
│
├── models/                 # Neural network architectures
│   ├── diagnostic/         #   ClinicalBERT, BioGPT (text)
│   ├── imaging/            #   ResNet3D, UNet3D, segmentation (MRI)
│   ├── eeg/                #   CNN-Transformer, anomaly detector (EEG)
│   └── simulation/         #   Neural mass models
│
├── backend/                # FastAPI application
│   ├── main.py             #   App entry point, loads models on startup
│   ├── app/api/endpoints/  #   REST + WebSocket endpoints
│   └── services/           #   Unified diagnostic service, orchestration, storage
│
├── frontend/               # Next.js + React dashboard (3D brain via Three.js)
│
├── training/               # Training scripts for every model
├── preprocessing/          # MRI / EEG / text preprocessing pipelines
├── data_preparation/       # Dataset builders (e.g. TUH EEG seizure corpus)
├── database/               # SQLAlchemy models, repositories, migrations
├── evaluation/             # Model evaluation
├── kubernetes/             # K8s deployment manifests
├── docker-compose*.yml     # Container orchestration (dev & prod)
└── docs/                   # Additional documentation

---

Tech stack

• Backend: Python, FastAPI, SQLAlchemy, Celery, Uvicorn/Gunicorn
• ML: PyTorch, Transformers (ClinicalBERT / BioGPT), MNE (EEG), MONAI/Nibabel (MRI), TVB
• Frontend: Next.js, React, TypeScript, Tailwind CSS, Three.js / React Three Fiber
• Data & infra: PostgreSQL, Redis, S3‑compatible object storage (AWS S3 / Cloudflare R2), Ollama (local LLM)
• Deployment: Docker, Docker Compose, Kubernetes

---

Quick start (Docker)

The fastest way to run the full stack. Requires Docker, Docker Compose, and a PostgreSQL + Redis instance (the compose files can also start these for you).

# 1. Clone
git clone <repo-url> && cd neuron-analysis

# 2. Configure environment
cp .env.example .env      # then edit values (DB, Redis, storage, secrets)

# 3. Build & start
make build
make up                   # or: docker-compose up -d

# 4. Open the app
#    Frontend  → http://localhost:3000
#    API       → http://localhost:8000
#    API docs  → http://localhost:8000/docs

Useful commands: make logs, make down, make migrate, make shell-backend. See QUICK_START.md and DOCKER_SETUP.md for details.

---

Quick start (local, without Docker)

# Backend
python -m venv venv && source venv/bin/activate
pip install -r requirements.txt          # or environment.yml with conda
cd backend
uvicorn main:app --host 0.0.0.0 --port 8000
# Models auto-load on startup if checkpoints exist under models/checkpoints/

# Frontend (in another terminal)
cd frontend
npm install
npm run dev                               # http://localhost:3000

---

Using the diagnostic API

Check which models are loaded:

curl http://localhost:8000/api/v1/models/health          # no auth
curl http://localhost:8000/api/v1/models/status \        # auth required
  -H "Authorization: Bearer YOUR_TOKEN"

Run a diagnosis through the Brain Assistant WebSocket (streams progress + result):

const ws = new WebSocket(
  `ws://localhost:8000/api/v1/ws/brain-assistant/SESSION_ID?token=${token}`
);

ws.send(JSON.stringify({
  type: "message",
  message: "Please analyze and diagnose this patient",
  study_id: "123",
}));

ws.onmessage = (e) => {
  const msg = JSON.parse(e.data);
  if (msg.type === "prediction_progress") console.log(`${msg.progress}% — ${msg.message}`);
  if (msg.type === "ai_message" && !msg.isStreaming) console.log("Diagnosis:", msg.content);
};

---

Training your own models

Training scripts live in training/. Each targets a specific model:

# Alzheimer's (multimodal: MRI + clinical text)
python training/train_multimodal.py --help

# Epilepsy (multimodal: EEG + clinical text)
python training/train_multimodal_epilepsy.py \
    --text_dir data/epilepsy_clinical_notes \
    --eeg_dir data/tuh_eeg \
    --epochs 50 --batch_size 8 --fusion attention

# Single-modality examples
python training/train_eeg_cnn_transformer.py
python training/train_unet3d.py

Public datasets used include the TUH EEG Seizure Corpus and ADNI MRI. See QUICKSTART_TUH.md and TUH_PREPARATION_SUMMARY.md for dataset preparation.

---

Further documentation

---

License

Released under the MIT License. © 2025 Baraa Rayed.

Neuron Analysis is a research prototype and is not approved for clinical use.