RTX PRO 6000 Blackwell LLM Wiki

This repository is a field wiki for running frontier LLMs on NVIDIA RTX PRO 6000 Blackwell / SM120 PCIe systems. It is more than a few launch snippets: it contains reproducible Docker builds, exact vLLM and SGLang runbooks, benchmark tables, KLD quality checks, quantization notes, DCP/MTP/DSpark/DFlash debugging, PCIe topology work, and regression history.

Community workbench for RTX PRO 6000 / SM120 serving: https://discord.gg/X54jjmcxWJ

Start Here

If you just want to run a model, use these stable hub pages first:

Need the complete map of every Markdown page?

What Is In This Repository?

Recommended Production-Style Pages

Older pages are intentionally preserved. Prefer the hub page for each model family unless you are reproducing a specific old result.

Core Topics

Benchmarks And Quality

KLD is a regression and quantization-sanity tool, not a complete quality metric. Use it together with long-context decode, coding probes, acceptance-rate checks, and task-level benchmarks.

Hardware And Topology

Most current measurements target RTX PRO 6000 Blackwell / GB202 / SM120 cards: 96 GB GDDR7 per GPU, PCIe 5.0 x16, no NVLink, usually 4-GPU, 8-GPU, or 16-GPU PCIe-switch systems.

Inference Engines

Acronym Policy

The wiki uses many acronyms: DCP, MTP, DSpark, DFlash, MLA, MoE, KLD, TP, CC, P2P, NVFP4, MXFP8, and more. To make pages readable for newcomers:

• Expand important acronyms on first use: Decode Context Parallelism (DCP).
• Do not expand acronyms inside commands, Docker tags, environment variables, JSON, file paths, or raw logs.
• Use Glossary And Acronym Guide as the source of truth.
• Run python3 scripts/check-acronyms.py before polishing a major page.

Keeping The Community Useful

The wiki should reduce accidental gatekeeping: newcomers should be able to find the right runbook, decode acronyms, and reproduce a known-good launch without needing tribal knowledge. That does not mean every Discord question can be answered from memory.

Use Newcomer Onboarding as the support contract: bring the model page, Docker image, full launch command, GPU layout, TP/DCP/MTP or DSpark settings, client command, and logs. That keeps the server welcoming without turning it into an unstructured support queue.

Common Operational Rules

• Do not launch with NCCL_GRAPH_FILE= set to an empty string. Unset it if no real XML graph file is used.
• Reuse cache directories while debugging; otherwise TileLang, Triton, CuTe, and FlashInfer rebuilds dominate iteration time.
• For quick smoke tests, use small MAX_NUM_SEQS and graph caps. For published tables, use the graph sizes documented in the model page.
• For DFlash and DSpark, confirm backend markers and acceptance rates before trusting throughput numbers.
• For GLM-5.2, keep the exact index_topk_pattern and DCP policy from the relevant runbook; a truncated pattern can silently degrade output.

Maintaining The Wiki

When adding a page:

• Link it from the relevant model hub.
• Add a short status block if it is a current runbook.
• Keep exact Docker image tags, source commits, model snapshot IDs, GPU layout, backend choices, and benchmark commands.
• Regenerate the full index:

python3 scripts/generate-wiki-index.py > INDEX.md

For performance claims, include both the server launch config and the client command so results can be reproduced on another PCIe-only Blackwell host.

Maintained from community Discord experiments through July 2026.