Qwen3-8B-Kernelbook-SFT-filtered

This repository contains a fine-tuned Qwen3-8B model using Supervised Fine-Tuning (SFT) on filtered KernelBook data.

Model Details

• Base Model: Qwen3-8B
• Training Method: Supervised Fine-Tuning (SFT)
• Training Data: Filtered KernelBook dataset
• Checkpoint: Iteration 515
• Format: Megatron-LM checkpoint format

Repository Structure

.
├── iter_0000515/                  # Checkpoint at iteration 515
│   ├── __*.distcp                 # Distributed checkpoint files (sharded model weights)
│   ├── common.pt                  # Common parameters
│   └── metadata.json              # Checkpoint metadata
├── rollout/                       # Training rollout data
│   └── global_dataset_state_dict_515.pt
└── latest_checkpointed_iteration.txt

Model Format

This repository contains the model in Megatron-LM checkpoint format, which includes:

• Distributed checkpoint files (.distcp) containing sharded model weights across 8 tensor parallel ranks
• Common parameters file (common.pt) with shared embeddings and other common weights
• Training metadata and configuration in JSON format
• PyTorch distributed checkpoint format (torch_dist backend)

Note on HuggingFace Format

A HuggingFace Transformers compatible version will be added in a future update.

Usage

Loading Megatron Format

To use the Megatron format checkpoint, you'll need the Megatron-LM framework with the appropriate model configuration.

Loading HuggingFace Format

(Instructions will be added after conversion)

Training Details

This model was fine-tuned using the SLIME framework on filtered KernelBook data to improve performance on kernel and system-level tasks.

Citation

If you use this model, please cite both the original Qwen3 model and the KernelBook dataset papers.