Multi-Agent VLM-RAG Federated Prototype

This repository contains a research prototype implementing a simulated Multi-Agent System (MAS) for decentralized disease identification using VLM+RAG and federated aggregation.

Structure:

• agents.py: LocalDiagnosticAgent and simple encoder
• datasets.py: synthesizes multimodal EHR data when real MIMIC features are not present
• losses.py: LDAM and supervised contrastive losses
• federated.py: FAA that performs Fisher-weighted aggregation and server momentum
• train.py: simulation script to run federated rounds
• evaluate.py: metrics functions
• job.slurm: example Slurm script for HPC

Usage:

1. pip install -r requirements.txt
2. Edit train.py or call it as a module: python -m train

Notes:

• This is a lightweight prototype for research and educational purposes. Replace the synthetic dataset with real MIMIC-IV loaders and integrate real VLM/RAG components for full experiments.