Pretrain-Behaviors

Dataset Description

Behavior-focused text covering reasoning, planning, data science, games, general content, and format rewriting. This repository is part of the K2 Horizon collection.

The repository is organized into multiple subsets. Every subset has a train split backed by Parquet shards, which supports Dataset Viewer inspection and streaming access.

K2 Horizon Dataset Series

Dataset Subsets

Repository Structure

README.md
reasoning/
  <source-file>-<stable-id>-00000.parquet
  <source-file>-<stable-id>-00001.parquet
general/
  <source-file>-<stable-id>-00000.parquet
  <source-file>-<stable-id>-00001.parquet
planning/
  <source-file>-<stable-id>-00000.parquet
  <source-file>-<stable-id>-00001.parquet
data-science/
  <source-file>-<stable-id>-00000.parquet
  <source-file>-<stable-id>-00001.parquet
games/
  <source-file>-<stable-id>-00000.parquet
  <source-file>-<stable-id>-00001.parquet
other/
  <source-file>-<stable-id>-00000.parquet
  <source-file>-<stable-id>-00001.parquet
format-rewrites/
  <source-file>-<stable-id>-00000.parquet
  <source-file>-<stable-id>-00001.parquet

The shard prefix is derived from the source JSONL filename and a stable identifier. Updating one source JSONL file replaces only that file's Parquet shards.

Data Fields

Records originate as JSON objects and are converted to Parquet for release. Field names and nested structures can differ by subset. Inspect features before building a processing pipeline:

from datasets import load_dataset

dataset = load_dataset(
    "IFM/Pretrain-Behaviors",
    "reasoning",
    split="train",
    streaming=True,
)
print(dataset.features)
print(next(iter(dataset)))

Data Provenance and Processing

Individual subsets may have undergone source-specific filtering, cleaning, deduplication, quality scoring, or synthetic-data generation. Users should evaluate each subset for their target use case and inspect the available provenance metadata.

Intended Use

This dataset is intended for language-model training and research. The subsets can be streamed independently, combined with user-defined sampling weights, or inspected through the Hugging Face Dataset Viewer.

Limitations and Responsible Use

Large-scale training data can contain factual errors, duplicated material, sensitive topics, stereotypes, unsafe content, and other artifacts. Dataset users are responsible for performing evaluations, risk assessment, and filtering appropriate to their application.

License and Terms of Use

This dataset is licensed under the Apache License 2.0 available at https://www.apache.org/licenses/LICENSE-2.0.