MatrAIx Persona 1M

999,847 personas, each described by 1,290 categorical attributes. 599,847 are derived from real records, 400,000 are synthetic. 10 Zstandard Parquet shards, 4.17 GB.

Read it with pyarrow, not datasets

Attributes are packed: one persona's 1,290 attributes are 645 bytes of 4-bit codes, low nibble first. datasets cannot open these files at all. Use pyarrow and decode against persona_codes.schema.json.

import json, pyarrow.parquet as pq

schema = json.load(open("persona_codes.schema.json"))["columns"]   # 1,290 entries
table  = pq.read_table("data/persona-1m-0000.parquet")             # 100,000 personas

def decode(attributes, null_bitmap):
    """One row -> {field id: value}. Missing attributes are omitted."""
    out = {}
    for i, col in enumerate(schema):
        if null_bitmap is not None and (null_bitmap[i // 8] >> (i % 8)) & 1:
            continue                                   # bit set = missing
        code = (attributes[i // 2] & 0x0F) if i % 2 == 0 else (attributes[i // 2] >> 4)
        if code < len(col["values"]):
            out[col["id"]] = col["values"][code]
    return out

person = decode(table["attributes"][1].as_py(), table["null_bitmap"][1].as_py())
person.get("age_bracket"), person.get("region")

Three things to get right:

• Use .get(), not [...]. Rows are sparse: 656 of 1,290 attributes are populated on average, and age_bracket for instance appears in about a fifth of rows. Missing means the source did not support it; nothing is imputed.
• In null_bitmap, a set bit means missing, LSB first. A null bitmap means nothing is missing in that row.
• attribute_overrides beats the decoded code. It holds exact values that fall outside the current codebook.

To filter without scanning 4 GB, indexes/postings.sqlite maps each value to the global row ids that carry it.

Columns

Files

What the viewer shows

The viewer cannot read the packed shards, so it is pointed at sample/ instead: 999 personas as rows, 990 attribute ids as columns, each cell the decoded value. All seven sources appear. Columns run densest first, and rows are ordered by how many attributes are populated, so the table opens full and thins out further down.

How much a persona carries depends on where it came from. Synthetic personas are complete by construction; a persona extracted from one Amazon review supports around 16 attributes. A blank cell is an attribute the source did not support, never an imputed one.

300 of the 1,290 fields are left out: the Dataset Viewer refuses more than 1,000 columns. The release carries all of them.

The release is 999,847 personas, not the 999 rows shown above. The row count on this page, and anything load_dataset returns, describes that sample.

Composition

Four dimensions are calibrated against 2024 global population margins: age_bracket and region from UN WPP 2024, gender_identity and urbanicity from UN and World Bank totals with a schema prior over the remaining categories. Build is deterministic for seed 20260720. audit.json reports achieved versus target share per category; RESULTS.md summarises the build.

Limits

• Not a representative sample of any population. Calibration matches one-dimensional margins among rows where the field is known. It does not fix the joint distribution or remove source-selection bias.
• "Human-grounded" is not "verified". Wiki, Amazon, Stack Overflow and PRISM attributes come from model extraction and can carry extraction errors; GSS and survey mappings depend on crosswalk quality. Descriptions are model-generated.
• The 60/40 human-to-synthetic split is a design choice, not an estimate of any real ratio.
• Under-18 records were removed. 153 of the original 508 Real Human Survey records declared an age bracket under 18 and are gone; every count here reflects that, which is why the total is 999,847 rather than 1,000,000.

License and Terms

MatrAIx Persona 1M is released for non-commercial research use only. Use of the dataset, any subset of it, or derivatives of it in a commercial product or paid hosted service is not permitted. The MIT license on the MatrAIx-Persona-8B GitHub repository covers the software in that repository, not these dataset files.

Subsets inherit these terms: extracting a subset, including the 400,000 full-DAG synthetic records, does not relicense it. Synthetic records carry categorical attributes from the shared schema plus model-generated descriptions; text generated with a language model remains subject to the respective model provider's terms. Several upstream sources carry their own restrictions, so commercial rights are not ours to grant.

Source licenses and terms continue to apply to the underlying data:

Responsible-use expectations, described in the paper (arXiv:2608.04205, Appendix N), apply to all use: no impersonation of real individuals, no attribution of the data to identifiable people, no re-identification attempts, and no targeting of individuals or protected groups. Attribution: cite the MatrAIx paper and link this dataset card.

Versioning and takedown

The dataset is versioned on the Hub and ships with a manifest and per-file hashes, so every change is visible as a new revision. If records are removed, for example when a survey participant withdraws consent, the removal will be documented here; downstream users are expected to move to the latest revision and delete copies of removed records. Questions and takedown requests: open a discussion on this dataset or an issue on the GitHub repository.