Introduction

ZGCM-1 is a 7.39B-parameter dense language model trained from scratch, built for mathematical reasoning and tool-assisted search. It combines deliberate internal thinking with active information gathering, supporting 256K-token context and both thinking and direct-response modes in a single model.

The project brings together an efficient hybrid-attention architecture, FP8 training with Muon, progressive long-context mid-training, and general-agentic supervised fine-tuning. Researcher-directed AI agents contribute throughout development, from data curation and cluster operations to evaluation and deployment.

Highlights

• Math and reasoning at 7B scale. ZGCM-1 achieves 97.13% on MATH-500, 75.00% on AIME 2026, and 70.42% on HMMT 2025, with the best average rank across the report's 14 reasoning benchmarks among the seven compared 7B–8B models.
• Search beyond model memory. Multi-step tool use reaches 63.09% on WebWalkerQA, 19.43% on BrowseComp, and 62.00% on Binary Function Search.
• Efficient long context. Gated sliding-window and global attention deliver 3.94× training throughput at 256K compared with full attention in the report's architecture experiments. The report estimates an approximately 4.2× improvement in 16K pretraining time-to-loss from combined architecture, precision, optimizer, and normalization gains.
• An open research recipe. The training repository releases the data-processing, pretraining, mid-training, and SFT workflows, with stage-specific configurations and runtime documentation.

Quickstart

ZGCM-1 ships its own modeling code, so trust_remote_code=True is required.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "zgcagi/ZGCM-1-7B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype="bfloat16",
    device_map="auto",
)

messages = [{"role": "user", "content": "Compute 1+1."}]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=4096,
    do_sample=True,
    temperature=1.0,
    top_p=1.0,
)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Evaluation Results

The following results are from the technical report, using the 256K SFT checkpoint in thinking mode. Non-agentic evaluations use temperature 1.0, top-p 1.0, and mean pass@1 over 32 runs unless otherwise specified.

Selected reasoning benchmarks

Selected rows and models from Table 2; bold marks the best score in each displayed row. The full evaluation covers 20 benchmarks, including code, knowledge, and instruction following.

Agentic search

Source: Tables 3–4. Web research allows up to 64 search-and-read steps. Binary Function Search uses a separate protocol on 50 tasks from 10 held-out projects.

Architecture and Training

The training recipe described in the report has three main stages:

1. Pretraining: approximately 4.19T tokens, combining curriculum-based data mixing with hybrid FP8 precision and Muon optimization.
2. Mid-training: approximately 600B tokens with context extended from 16K → 64K → 256K. Interaction traces are reformulated as Markov Decision Process (MDP) state-action transitions to supervise individual decisions.
3. Supervised fine-tuning: joint general and agentic training with mixed thinking/direct-response examples, execution-verified trajectories, and assistant-only loss. The report also explores mixed RL for mathematics and code.

Resources

For dataset usage, see the ZGCM-1-Data card. Model specifications, evaluation details, and figures are presented in ZGCM-1: A Fully Open and Extremely Efficient Foundation Model for Math and Agentic Search.

Training Code

The complete training pipeline is released at github.com/zgcagi/ZGCM-1, covering data processing, pretraining, mid-training, supervised fine-tuning, and reinforcement learning. Each stage is a top-level directory with its own code, configurations, and runtime documentation.

WeChat Community

Scan the QR code to join the ZGCM-1 community group. Click the image to open it at full size. If the code has expired, please open a Discussion and ask the maintainers for the latest one.

License

ZGCM-1 is released under the MIT License. Third-party licenses and notices bundled with the training code are listed in the training repository; the dataset carries its own terms, given on the ZGCM-1-Data card.

Citation

If you find ZGCM-1 useful in your research, please cite our technical report:

@misc{zgcm1,
  title={ZGCM-1: A Fully Open and Extremely Efficient Foundation Model for Math and Agentic Search},
  author={Jiyan He and Guang Liang and Hao Liu and Haoxiang Guan and Jinbo Sun and Junyi Guo and Wenjun Feng and Yantai Xie and Yifei Shen and Bin Shao and Chuyang Wei and Kai Chen and Kexin Zhou and Minghang Zhu and Shuxin Zheng and Tie-Yan Liu and Taine Zhao and Wenhui Zhu and Xueyin Xu and Xiaoqing Zhang and Yatao Li and Yuxuan Ren},
  year={2026},
  eprint={2609.13356},
  archivePrefix={arXiv},
  primaryClass={cs.AI},
  url={https://arxiv.org/abs/2609.13356}
}