Local AIChat Prototype — OpenCampus 2026

Overview

A desktop AI chat demo for OpenCampus in academic year 2026. Visitors can send messages and explore conversations with a locally hosted language model through a simple chat interface.

System Architecture

• Frontend: React 19 and Vite, with Zustand for chat state.
• Desktop application: Tauri 2 and Rust manage conversation history and forward requests to the local API.
• Inference server: Python, FastAPI, and Hugging Face Transformers run ibm-granite/granite-4.1-3b with 8-bit quantization.

React UI → Tauri / Rust → FastAPI (127.0.0.1:8000) → Local LLM

The Rust backend sends chat messages to POST /generate. The server returns generated text and context token usage.

Requirements

• Node.js 20.19+ or 22.12+ and npm.
• Rust and the platform build dependencies for Tauri 2. On Windows, install Microsoft C++ Build Tools and WebView2.
• Python 3.14+ and uv.
• A compatible GPU and driver for the configured CUDA 13.0 PyTorch build and bitsandbytes inference.
• Internet access for the initial dependency installation and model download.

Setup and Boot Commands

Run the following commands from the repository root, using two separate terminals.

1. Start the LLM server

cd llm_server
uv sync
uv run uvicorn src.api_server:app --host 127.0.0.1 --port 8000

The first launch downloads the model and may take some time. Wait for the server to finish loading before starting the desktop application. You can check the server at http://127.0.0.1:8000/.

2. Start the desktop application

npm install
npm run tauri dev

Tauri starts the Vite development server automatically. Keep both terminals running during the demo.

For later launches, run the same boot commands without uv sync or npm install unless dependencies have changed.

Build

npm run tauri build

This builds the desktop application. Run the Python inference server separately when using the built application.

Project Structure

src/                 React interface and chat state
src-tauri/           Tauri configuration and Rust backend
llm_server/          Python API and model inference