InferenceX Dashboard / InferenceX 仪表板

A Next.js dashboard for visualizing ML inference benchmark data. DB-backed with Neon PostgreSQL, React Query for data fetching, D3.js for interactive charts.

Stack: Next.js 16 (App Router) · TypeScript · Tailwind CSS 4 · shadcn/ui · D3.js · Neon PostgreSQL · Vercel · Cypress

Overview

LLM inference performance is a major concern of providing AI services, but accurate performance analysis remains elusive. Fast cadence of software development and model releases makes comparing performance between setups difficult. Existing performance benchmarks quickly become obsolete due to being static, and participants game the benchmarks with unrealistic, highly specific configurations. InferenceX tackles these issues by benchmarking popular models on major hardware platforms nightly with the latest software. For each model and hardware combination, InferenceX sweeps through different tensor parallel sizes and max concurrent requests, showing the throughput vs. latency graph for the full picture. In terms of software configurations, we ensure they are generally applicable across different serving scenarios, and we open source the repo to welcome community contributions. We hope InferenceX informs the community up-to-date and realistic LLM inference performance.

Architecture

Neon PostgreSQL → API routes (/api/v1/*) → React Query hooks → Context providers → D3.js charts

The frontend fetches data from API routes backed by a Neon PostgreSQL read replica. All presentation logic lives in the frontend — API routes return raw DB data. The MCP server exposes read-only benchmark tools through the same db and constants packages, without depending on frontend code.

Monorepo Structure

packages/
├── app/          # Next.js frontend
├── constants/    # Shared constants (GPU keys, model mappings)
├── db/           # DB layer, ETL, migrations, queries, ingest scripts
└── mcp/          # MCP server exposing read-only benchmark tools

Prerequisites

• Node.js: 24.x
• Bun: 1.3.14+

Install Bun on macOS or Linux:

curl -fsSL https://bun.sh/install | bash

Install Bun on Windows PowerShell:

powershell -c "irm bun.sh/install.ps1|iex"

Getting Started

feel free to file an github issue if u need help setting up the dashboard

1. Clone & Install

git clone https://github.com/SemiAnalysisAI/InferenceX-app.git
cd InferenceX-app
bun install

2. Set Up a Database

The dashboard requires a live PostgreSQL database. Set DATABASE_READONLY_URL in .env; the application supports both standard PostgreSQL servers and Neon.

Option A: Standard PostgreSQL

Use the postgres driver for a local or remotely hosted PostgreSQL server:

cp .env.example .env
cat >> .env <<'EOF'
DATABASE_READONLY_URL=postgresql://postgres:postgres@localhost:5432/postgres
DATABASE_DRIVER=postgres
DATABASE_SSL=false
EOF

Remote PostgreSQL servers use TLS by default. Omit DATABASE_SSL unless the server explicitly requires it to be disabled.

Option B: Neon

Set DATABASE_READONLY_URL to a Neon PostgreSQL connection string. Neon hosts use the serverless HTTP driver automatically; DATABASE_DRIVER=neon can be set explicitly.

cp .env.example .env
cat >> .env <<'EOF'
DATABASE_READONLY_URL=postgresql://user:password@ep-example.us-east-1.aws.neon.tech/database
DATABASE_DRIVER=neon
DATABASE_SSL=true
EOF

3. Run the Development Server

bun run dev

Open http://localhost:3000 with your browser.

Development Scripts

These are the main scripts you'll use during development. Admin scripts for database and cache management are listed separately below. Some of these may require additional setup or environment variables.

This repository uses Vitest and Cypress. Use the bun run test:* scripts above; bun test invokes Bun's separate built-in test runner and is not supported.

Admin Scripts

These are meant to be used for database and cache management and maintenance tasks, and should not be necessary during regular development. However, using bun run admin:cache:invalidate pointed at your local development server can be useful to test after making changes to the database or API routes.

Changes to packages/db/src/etl/run-overrides.ts that reach main or master are applied to the production database automatically by CI, followed by database verification, cache invalidation, and cache warmup. The override command remains available for local previews and manual recovery.

Deployment

Deployed on Vercel. See .env.example for all required environment variables.