GRIMOIRE

LLM inference built for Intel Arc Pro B70 Battlemage GPUs.

GRIMOIRE is a native inference engine written in C++ with SYCL and Level Zero. It is being developed to run and optimize large language models directly on Intel Battlemage hardware, with a focus on the Arc Pro B70.

The aim is straightforward: make high-performance local LLM inference possible on this hardware, using kernels and memory paths designed for the GPU instead of relying on a general-purpose inference stack.

Project status: Experimental. GRIMOIRE is actively developed, but it is not yet a stable, production-ready release. Expect incomplete features, changing model support, and performance or compatibility issues. A ready-to-use inference image or downloadable model package will be added here later.

Why GRIMOIRE was created

Most inference tooling and performance guidance is centered on other GPU platforms. Battlemage owners need an engine that treats Intel hardware as a first-class target and makes its real performance measurable.

GRIMOIRE began as a ground-up inference project for the Arc Pro B70. Building the engine directly exposed important details that a port or a high-level wrapper could hide: how the GPU represents quantized weights, where attention kernels can go wrong, how to keep decode bound by useful memory traffic, and what multi-GPU communication actually costs on this setup.

The project exists to turn those findings into working inference code, reproducible checks, and practical performance improvements for Battlemage users.

What it is for

• Running supported large language models locally on Intel Arc Pro B70 GPUs.
• Exploring native C++ / SYCL / Level Zero kernels for model inference.
• Measuring prefill and token-generation performance on real hardware.
• Improving single-GPU execution and developing multi-GPU paths for models that need more memory.
• Checking changes against a growing set of model and checkpoint combinations.

GRIMOIRE is a hardware-focused inference engine, not a hosted model service. Model compatibility and performance depend on the architecture, weight format, configuration, and hardware available.

How it is built

• C++ inference engine
• SYCL GPU programming
• Level Zero device runtime
• Intel Battlemage, with the Arc Pro B70 as the primary target
• Custom GPU work for operations such as attention, matrix multiplication, quantized weight handling, and mixture-of-experts layers

The project aims to keep inference native to the target hardware and does not use vLLM, PyTorch, or OpenVINO as its inference backend.

Recent measured results

These are results recorded in the linked project commits on the developer's Arc Pro B70 system. They are examples from specific models and test runs, not universal performance guarantees.

The two-GPU figures demonstrate an actively developed path. Tensor and pipeline parallelism are intended for models that need multiple GPUs for capacity; they are not automatically faster for a model that already fits on one card.

Results change as kernels and test coverage evolve. The commit links include the measurements, validation notes, and limitations for each change. The latest model-zoo audit reports 27 confirmed working model/checkpoint combinations in the local regression sweep, with additional formats and configurations correctly identified as out of scope. Latest audit

Supported models

Everything below is validated end-to-end — loaded, generating coherent text, checked against a reference where one exists — on one Intel Arc Pro B70, by the project's own regression suite. That suite runs after every change that touches shared decode, prefill, attention, or MoE code, not as a one-time check.

All seven weight formats (BF16, FP8 E4M3/E5M2, INT8, INT4, MXFP8, MXFP4) run through one decode path shared by every model above and by the host-side tests.

Tensor-parallel and pipeline-parallel both run correctly across two B70s, but they exist for checkpoints whose own footprint doesn't fit one card's VRAM — every model in the table fits a single B70 and runs fastest that way.

Speculative decoding (the MTP and DFlash2 rows above) is currently correct but slower than plain decode on every model it's paired with — functional, not yet a throughput win.

Getting started

The engine and build scripts are in this repository, along with a Dockerfile that builds a minimal runtime image -- just GRIMOIRE's own binaries plus Intel's GPU driver stack, nothing else:

docker build -t grimoire-b70 .
docker run --init --stop-timeout 300 --device /dev/dri/renderDXXX \
    -v /path/to/your/models:/models -p 8000:8000 \
    grimoire-b70 server --model /models/<checkpoint> --proj mxfp4 --port 8000

--init and a generous --stop-timeout are not optional: GPU work in flight when a container is killed without them can wedge the card hard enough to need a power cycle. Replace server with generate -m ... -p "..." -n <tokens> for a one-shot CLI run instead of the HTTP server. Check the project status and the model table above before choosing a model or format.

Development notes

GRIMOIRE is under active development. Model support, build requirements, and performance may change as the engine evolves. Benchmarks should be read with their linked commit notes, which record the model, test conditions, correctness checks, and known trade-offs.

Links

• Source code
• Recent commits
• Issues