StarForge

One inference engine per model, Metal only, forked from antirez/ds4 (DwarfStar).

ds4 runs several large open models on several backends from one codebase. It is excellent and it is also a lot to hold in your head. StarForge produces children: full forks of ds4 reduced to one model on Apple Metal, each a normal GitHub repo you can build, read and modify without the other models and backends in the way. Children keep merging upstream through plain git merge, so fixes and speedups from ds4 keep flowing in.

Each child is a separate repository. This repository is the orchestrator: rules, procedures, tools, and a read-only mirror of upstream. No inference code lives here.

Why

ds4 is built around a few models rather than as a general GGUF runner, and it is meant to be read and changed with a coding agent: a working template to adapt to your model and hardware, not a product that covers every setup. StarForge pushes both ideas to the end and makes the result repeatable, one child per model:

• Smaller code, clearer to an LLM. SPEC.md decides what every child deletes and what it must keep, so each child shrinks the same way. What is left is a tree a person or a coding agent can load whole and reason about, which makes it cheap to try an idea on one model, measure it and keep or drop it.
• Metal only. The children are developed and tested on one machine, an Apple M5 Max, so Metal is the only GPU backend any child keeps.
• Small, but not frozen. The orchestrator's tools keep every child merging upstream/main with plain git merge, rerere and the parity oracle, so ds4's fixes and speedups keep flowing into a tree that stays small.
• Room to investigate. Because each child holds one model, open ds4 pull requests can be analysed against that model and ported where they hold up, and further improvements can be investigated for Apple Silicon. That work happens and is recorded inside each child (its docs/upstream-prs.md), not here.

If you just want to run a model

Go to the child's repo. Its README tells you make, ./download.sh <quant>, ./sf-<model>. download.sh uses the Hugging Face CLI and creates a gguf/ symlink to the shared Hugging Face cache; it does not duplicate the model in the repo. You do not need this repo.

If you maintain the children

git clone https://github.com/Chida82/StarForge && cd StarForge
tools/clone-all.sh          # upstream/ds4 (full) + children/* (those that exist on GitHub)
tools/status.sh             # where each child stands vs upstream/main

Tools

All bash + git + coreutils, in tools/:

Layout

AGENTS.md  SPEC.md  BOOTSTRAP.md  SYNC.md  README.md
templates/child-AGENTS.md        copied into a child at bootstrap
tools/                           scripts + parity prompt sets
children/<name>/                 plain clones (gitignored)
upstream/ds4/                    plain full clone of ds4 (gitignored, read-only)

Children and upstream are plain clones, not submodules: each child already records its own upstream position (tags, merge-base). There is deliberately no central SHA/status file; tools/status.sh derives live state from Git. See SPEC.md §H.

Design in five lines

1. A child is a full git fork; git merge upstream/main is how it stays current.
2. ds4_* file names and identifiers never change; only binary names, paths and help do.
3. Removal is deletion, not #ifdef. Cuts inside live files carry /* sf-ablate(...) */.
4. git rerere replays conflict resolutions, so ablated regions stop hurting after the first sync.
5. The parity oracle (same GGUF, same prompts, greedy, token-identical, speed ±2%) is the definition of correct.

Always kept in every child: steering, TP/RDMA, CPU reference path, server, CLI, bench, eval. Speculative decoding follows the table: notably, sf-ds4flash keeps DSpark but deletes legacy MTP. Never in a child: ds4-agent, CUDA, ROCm, other models.

Agents may prepare and test changes, but commit and push only after an explicit user request.

Status

sf-q3-8flash and sf-ds4-1flash are bootstrapped, published and synced, and each now carries its own performance work. sf-ds4flash and sf-glm5-3flash have not been started.

Which upstream commit each child sits on is deliberately not written here -- tools/status.sh reads it from Git (SPEC.md §A, §H).

Licence

Tools and documents in this repository: MIT. Children inherit ds4's MIT licence and keep its LICENSE file, including the GGML notice, unchanged. Everything works because of ds4, llama.cpp and GGML.