TxT360-v2

Dataset Description

Pre-training sources for the K2 Horizon training data release. This repository is part of the K2 Horizon collection.

The repository is organized into multiple subsets. Every subset has a train split backed by Parquet shards.

K2 Horizon Dataset Series

Dataset Subsets

Repository Structure

README.md
web-high-nltk-qa/
  <source-file>-<stable-id>-00000.parquet
  <source-file>-<stable-id>-00001.parquet
web-high-medium/
  <source-file>-<stable-id>-00000.parquet
  <source-file>-<stable-id>-00001.parquet
txt360-qa/
  <source-file>-<stable-id>-00000.parquet
  <source-file>-<stable-id>-00001.parquet

The shard prefix is derived from the source JSONL filename and a stable identifier. Updating one source JSONL file replaces only that file's Parquet shards.

Data Fields

Records originate as JSON objects and are converted to Parquet for release. Field names and nested structures can differ by configuration. Inspect features before building a processing pipeline:

from datasets import load_dataset

dataset = load_dataset(
    "IFM/TxT360-v2",
    "web-high-nltk-qa",
    split="train",
    streaming=True,
)
print(dataset.features)
print(next(iter(dataset)))

Data Provenance and Processing

Individual subsets may have undergone source-specific filtering, cleaning, deduplication, quality scoring, or synthetic-data generation. Users should evaluate each subset for their target use case and inspect the available provenance metadata.

Intended Use

This dataset is intended for language-model training and research. The subsets can be streamed independently, combined with user-defined sampling weights, or inspected through the Hugging Face Dataset Viewer.

Limitations and Responsible Use

Large-scale training data can contain factual errors, duplicated material, sensitive topics, stereotypes, unsafe content, and other artifacts. Dataset users are responsible for performing evaluations, risk assessment, and filtering appropriate to their application.

License and Terms of Use

This dataset is licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0) available at https://creativecommons.org/licenses/by/4.0/legalcode.