Ornith Thinking-Cap

Training and evaluation code behind the Ornith Thinking-Cap models — reasoning-efficiency fine-tunes that cut a coding model's reasoning-token spend without losing accuracy.

This is a reproduction of the ThinkingCap method by BottleCap AI (bottlecapai/ThinkingCap-Qwen3.6-27B), applied to the Ornith coding models. Independent work; not affiliated with or endorsed by BottleCap AI.

Released models (Hugging Face):

Both were trained on a single NVIDIA DGX Spark (GB10).

The idea

I had two goals with this project. One: small models with reasoning enabled are prone to what I call "anxiety loops" - they can often struggle to wrap up the reasoning chain of thought and spin for a long time or even sometimes infinitely loop. Can we train a smaller model to avoid anxiety looping? And Two: can we improve the coding capability of already strong small models with good fine tuning? I ran across BottleCap's thinking cap model and was impressed at its performance, and wanted to see if I could replicate that approach with the incredibly capable Ornith coding models.

I was wondering if the Thinking Cap method would curb anxiety looping by helping to shorten the reasoning traces. Seems to work out, but there's some caveats that are called out at the bottom of this README.

So, I had a conversation with Claude and this is what came out. Here's Claude's explanation:

Chain-of-thought reasoning is expensive, and a lot of it is padding — the model over-explains problems it has already effectively solved. Length-penalized reinforcement learning can trim that padding, but the obvious naive version teaches the model to answer fast and wrong. The fix is a correctness gate:

reward = correctness                              # 1.0 if the answer/tests pass, else 0.0
       − (λ · normalized_length   if correct)     # length penalty applies ONLY to correct answers
       + format_bonus                             # well-formed <think>…</think> + answer

Because the length penalty only ever applies to answers that are already correct, the model can never trade accuracy for brevity. It learns to compress reasoning only on problems it has mastered — dramatically on math (where reasoning is cheap), barely at all on code (where it's load-bearing). A KL leash to the frozen base and a λ ramp keep it stable and self-limiting.

The reward is verifiable: the exact same code-execution and numeric-answer checkers used to grade the models are reused as the training reward (code_runner.py + the numeric checker in rewards.py). Training and evaluation share one oracle, so a "win" can't be an artifact of a mismatched grader.

Results

Greedy decode, in-process. accuracy @ mean completion tokens.

9B (dense):

35B (MoE):

A common failure mode of length-penalized RL is repetitive degeneration ("anxiety loops"). Tested on 24 loop-prone prompts with a repeated-span detector, both models loop 0% of the time — and produce fewer runaway completions than their base. The raw result JSONs are in results/.

Repo layout

thinking_cap/          the method
  rewards.py             correctness-gated length-penalty reward harness (TRL reward_funcs)
  code_runner.py         vendored code-execution scorer — the eval oracle, reused as reward
  grpo_config.py         LoRA / GRPO / reward hyperparameters + 9B & 35B profiles + λ ramp
  grpo_train.py          training loop — dense 9B path (Unsloth + TRL)
  grpo_train_hf.py       training loop — MoE 35B path (transformers + PEFT + TRL)
  build_dataset.py       assemble the verifiable-prompt training set
  bench_suite.py         publish-grade accuracy@tokens benchmark (GSM8K/HumanEval/MBPP)
  loop_eval.py           loop/runaway detector on adversarial prompts
  ollama_bench.py        same benchmark against GGUF quants via Ollama
  probe_sweep.py         decode-parameter sweep
  test_rewards.py        offline test: asserts the reward-hacking gate holds (no GPU needed)
  smoke_train.py         offline harness↔trainer contract check (no GPU needed)
tools/                 adapter merge + GGUF conversion helpers
data/                  pre-built verifiable-prompt seed set
results/               benchmark + loop-eval result JSONs
model_cards/           the Hugging Face model cards

Quick start

pip install -r requirements.txt

# Verify the method with no GPU — asserts every wrong answer scores below every correct one:
python thinking_cap/test_rewards.py

# (Re)build training data from HuggingFace benchmarks:
python thinking_cap/build_dataset.py --hf --gsm8k 2000 --mbpp 500 --out data/train.jsonl

Training needs a CUDA GPU. The dense-9B path uses Unsloth (grpo_train.py); the MoE-35B path uses plain transformers + PEFT + TRL (grpo_train_hf.py), because on a 256-expert MoE the experts are frozen and LoRA targets attention only (see the model cards for the why). Wire the reward harness into TRL's GRPOTrainer:

from thinking_cap.rewards import RewardHarness
from thinking_cap.grpo_config import PROFILES, lam_at_step

prof = PROFILES["ornith-9b"]            # or "ornith-35b"
harness = RewardHarness(
    lam=prof.reward.lam_start,
    max_think_tokens=prof.reward.max_think_tokens,
    format_bonus=prof.reward.format_bonus,
    tokenizer=lambda s: tok(s).input_ids,
    lam_schedule=lambda: lam_at_step(trainer.state.global_step, prof.reward),
)
GRPOTrainer(..., reward_funcs=harness.reward_funcs)

Caveats and open questions

This project was a mixed success - the 9B model tune worked extremely well. Not so much the 35B due to my lack of hardware resources - I'd need to train the experts to compress their output, too to get the near 75% compression (well, on math problems) I found on the 9B.

Likewise: with thinking max_length set to 2048 tokens, I'm honestly not sure I covered long-run thinking sessions, but that's not usually how I use these smaller models. I don't one-shot vibe code with this class of models, but I do use them in VSCode as a sidekick to give me boilerplate examples, sanity check correctness, and help with writing tests. In this use case, the long traces shouldn't be something that happens often anyway.

Credit & license

Method: ThinkingCap by BottleCap AI (independent reproduction here). Base models: Ornith by DeepReinforce AI. This code is released under the MIT License.