Simulate before reality.
Population-scale, persona-driven infrastructure for evaluating AI systems and interactive products with heterogeneous simulated users.
MatrAIx is a population-scale, persona-driven infrastructure for evaluating AI systems and interactive products with heterogeneous simulated users. Instead of testing against a generic or interchangeable user, MatrAIx instantiates sampled persona records as LLM agents and runs them through reproducible tasks across four environments — Survey, AI Chatbot, Web, and App (native desktop and mobile, including macOS and iOS).
At its foundation is a shared schema of 1,290 categorical dimensions covering background, psychology, capability, and behavior. Personas combine dependency-aware synthetic generation with evidence-aware human grounding; a deterministic, quality-filtered coreset of one million personas is released for research on Hugging Face. Shared telemetry, task-owned verification, and reporting connect individual responses and trajectories to subgroup- and population-level findings.
The name nods to The Matrix: a simulated world useful for exploration, stress testing, and hypothesis generation, not a replacement for evidence from real people.
2608.04205).Windows users: run everything inside WSL2 — open PowerShell, run
wsl --install(installs Ubuntu), then clone this repo inside the WSL filesystem (e.g.~/MatrAIx, not/mnt/c/…, which is much slower) and enable WSL integration in Docker Desktop → Settings → Resources. Every command in this README then works exactly as written. Native PowerShell/cmd is not supported (the task verifiers requirebash).
git clone <repo-url> && cd MatrAIx
uv venv --python 3.12
uv pip install -e .
uv pip install pytest pytest-asyncio httpx
uv pip install -e packages/playground
uv pip install -e packages/harbor-langsmith
uv pip install -e packages/rewardkit
Run jobs with uv run matraix run …. After install, use the
smoke tests below to confirm Survey, Chat, Web, and OS-app are
ready (no API key). Summarize a finished job with
uv run matraix results <job>. Advanced runtime tools stay under
uv run harbor ….
Set a model API key before real GUI or CLI runs (smoke checks do not need one):
export ANTHROPIC_API_KEY="sk-ant-..." # anthropic/claude-* models
# export OPENAI_API_KEY="sk-..." # openai/gpt-* models
See agents.md for the full key matrix.
Playground can also load keys from application/playground/.env.local.
The in-repo matraix-persona-dev-sample (~200) is for smoke only. For real
cohorts and Playground sampling, import the public 1M coreset:
huggingface-cli download MatrAIx2026/MatrAIx_Persona_1M_Public_Release \
--repo-type dataset \
--local-dir persona/datasets/matraix-persona-1m/release
Playground: Dataset → matraix-persona-1m. CLI: --dataset persona/datasets/matraix-persona-1m.
Details: Handbook § Persona 1M.
Two quick checks after install — no API key. Together they cover the default path for all four task types (Survey, Chat, Web, OS-app):
| Check | Confirms you can run | Command |
|---|---|---|
| Without Docker | Survey and Chat | uv run matraix smoke application/tasks/example-survey_product-feedback |
| With Docker | Web and OS-app | uv run matraix run -c configs/jobs/example-job-recipe/harbor-smoke-local.yaml |
The first finishes in seconds and should print Smoke: ok. The second builds a
small local image on first run (a few minutes), then writes under
jobs/harbor-smoke-local/. Step-by-step: quickstart §3.
Playground picks tasks, samples personas, and launches the same Matraix Playground jobs as CLI auto mode. Start API + frontend (two terminals):
# Terminal A — API
VENV=.venv bash application/playground/backend/run_dev.sh
# Terminal B — frontend
cd application/playground/frontend && npm ci && npm run dev
Open http://localhost:5173 → Playground → pick a persona cohort → pick Survey / Chat / Web / OS app tasks → Lock pipeline → Run eval. Details: Playground §10.
Develop — copy a reference task under application/tasks/, edit
task.toml / instruction.md / input/ / verifier, then register it for Playground
(task-guide.md):
cp -R application/tasks/example-survey_product-feedback \
application/tasks/<your-task-name>
| Type | Reference task |
|---|---|
| Survey | application/tasks/example-survey_product-feedback |
| Chat | application/tasks/example-chat-api_support_chatbot |
| Web | application/tasks/example-web-playwright_quote-choice |
| OS-app | application/tasks/example-computer-use-linux_note-to-csv |
Run — generate a Matraix Playground job (pins agent + model), then execute it:
uv run python application/scripts/generate_application_job.py \
--task application/tasks/example-survey_product-feedback \
--execution-mode auto \
--persona-ids 0042 \
--model-name anthropic/claude-sonnet-4-6
# Use the export lines + recipe path the script prints, e.g.:
uv run matraix run -c configs/jobs/application-task-job-recipe/example-survey-product-feedback-auto-n1.yaml
Batch (--sample-size N), filters, and chat / web / os-app examples:
docs/quickstart.md.
MatrAIx Handbook — guides, persona / application / environment docs.
MatrAIx/
├── persona/ Schema, datasets, synthesis/curation/validation pipelines
│ ├── schema/ 1,290-dimension persona schema
│ ├── datasets/ Dev sample pool and persona YAMLs
│ ├── validation/ Grounding / quality validation suites
│ └── scripts/ Persona job & pipeline helpers
├── application/
│ ├── tasks/ Survey · chat · web · os-app task specs
│ ├── task-spec/ Shared task contracts
│ ├── playground/ Visual runner (backend API + frontend)
│ └── scripts/ generate_application_job.py and task tooling
├── environment/
│ ├── runtime/ Matraix Playground runtime
│ ├── agents/ Persona-conditioned agents
│ ├── task-environments/ Docker images / sidecars
│ └── adapters/ External adapters (e.g. SimpleQA)
├── packages/ playground · rewardkit · harbor-langsmith
├── apps/viewer/ Frontend paired with `harbor view`
├── configs/jobs/ Curated & generated Matraix Playground job recipes
├── docs/ Handbook — persona/ · application/ · environment/
├── examples/ Minimal example tasks
├── src/matraix/ Python package entrypoints
├── scripts/ Repo-level helpers
├── tests/ Unit / environment tests
└── jobs/ Local Matraix Playground run outputs (gitignored)
Large generated datasets stay outside git (see the Hugging Face release above).
Full Name - Affiliation. Fill the Google Form
(background, interests, paper authorship / acknowledgements).If you use MatrAIx, the Persona 1M dataset, or results from this repository, please cite:
@article{li2026matraix,
title = {MatrAIx: Simulating the World with 8.3 Billion Persona Agents},
author = {Li, Xiaomin and Hao, Yuexing and Hou, Jianheng and Huang, Jintao
and Wen, Qianfeng and Huang, Shirley and Liu, Yifan and Liu, Xiaoyi
and Fan, Yilan and Wang, Yijun and others},
year = {2026},
eprint = {2608.04205},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2608.04205}
}
Paper: arXiv:2608.04205 ·
Full authors: GitHub Cite this repository (CITATION.cff) ·
Dataset: Persona 1M on Hugging Face.
MIT — see LICENSE.
Python
72.2%
TypeScript
14.9%
Jupyter Notebook
6.0%
HTML
4.0%
Shell
2.5%
Simulate before reality.
Population-scale, persona-driven infrastructure for evaluating AI systems and interactive products with heterogeneous simulated users.
MatrAIx is a population-scale, persona-driven infrastructure for evaluating AI systems and interactive products with heterogeneous simulated users. Instead of testing against a generic or interchangeable user, MatrAIx instantiates sampled persona records as LLM agents and runs them through reproducible tasks across four environments — Survey, AI Chatbot, Web, and App (native desktop and mobile, including macOS and iOS).
At its foundation is a shared schema of 1,290 categorical dimensions covering background, psychology, capability, and behavior. Personas combine dependency-aware synthetic generation with evidence-aware human grounding; a deterministic, quality-filtered coreset of one million personas is released for research on Hugging Face. Shared telemetry, task-owned verification, and reporting connect individual responses and trajectories to subgroup- and population-level findings.
The name nods to The Matrix: a simulated world useful for exploration, stress testing, and hypothesis generation, not a replacement for evidence from real people.
2608.04205).Windows users: run everything inside WSL2 — open PowerShell, run
wsl --install(installs Ubuntu), then clone this repo inside the WSL filesystem (e.g.~/MatrAIx, not/mnt/c/…, which is much slower) and enable WSL integration in Docker Desktop → Settings → Resources. Every command in this README then works exactly as written. Native PowerShell/cmd is not supported (the task verifiers requirebash).
git clone <repo-url> && cd MatrAIx
uv venv --python 3.12
uv pip install -e .
uv pip install pytest pytest-asyncio httpx
uv pip install -e packages/playground
uv pip install -e packages/harbor-langsmith
uv pip install -e packages/rewardkit
Run jobs with uv run matraix run …. After install, use the
smoke tests below to confirm Survey, Chat, Web, and OS-app are
ready (no API key). Summarize a finished job with
uv run matraix results <job>. Advanced runtime tools stay under
uv run harbor ….
Set a model API key before real GUI or CLI runs (smoke checks do not need one):
export ANTHROPIC_API_KEY="sk-ant-..." # anthropic/claude-* models
# export OPENAI_API_KEY="sk-..." # openai/gpt-* models
See agents.md for the full key matrix.
Playground can also load keys from application/playground/.env.local.
The in-repo matraix-persona-dev-sample (~200) is for smoke only. For real
cohorts and Playground sampling, import the public 1M coreset:
huggingface-cli download MatrAIx2026/MatrAIx_Persona_1M_Public_Release \
--repo-type dataset \
--local-dir persona/datasets/matraix-persona-1m/release
Playground: Dataset → matraix-persona-1m. CLI: --dataset persona/datasets/matraix-persona-1m.
Details: Handbook § Persona 1M.
Two quick checks after install — no API key. Together they cover the default path for all four task types (Survey, Chat, Web, OS-app):
| Check | Confirms you can run | Command |
|---|---|---|
| Without Docker | Survey and Chat | uv run matraix smoke application/tasks/example-survey_product-feedback |
| With Docker | Web and OS-app | uv run matraix run -c configs/jobs/example-job-recipe/harbor-smoke-local.yaml |
The first finishes in seconds and should print Smoke: ok. The second builds a
small local image on first run (a few minutes), then writes under
jobs/harbor-smoke-local/. Step-by-step: quickstart §3.
Playground picks tasks, samples personas, and launches the same Matraix Playground jobs as CLI auto mode. Start API + frontend (two terminals):
# Terminal A — API
VENV=.venv bash application/playground/backend/run_dev.sh
# Terminal B — frontend
cd application/playground/frontend && npm ci && npm run dev
Open http://localhost:5173 → Playground → pick a persona cohort → pick Survey / Chat / Web / OS app tasks → Lock pipeline → Run eval. Details: Playground §10.
Develop — copy a reference task under application/tasks/, edit
task.toml / instruction.md / input/ / verifier, then register it for Playground
(task-guide.md):
cp -R application/tasks/example-survey_product-feedback \
application/tasks/<your-task-name>
| Type | Reference task |
|---|---|
| Survey | application/tasks/example-survey_product-feedback |
| Chat | application/tasks/example-chat-api_support_chatbot |
| Web | application/tasks/example-web-playwright_quote-choice |
| OS-app | application/tasks/example-computer-use-linux_note-to-csv |
Run — generate a Matraix Playground job (pins agent + model), then execute it:
uv run python application/scripts/generate_application_job.py \
--task application/tasks/example-survey_product-feedback \
--execution-mode auto \
--persona-ids 0042 \
--model-name anthropic/claude-sonnet-4-6
# Use the export lines + recipe path the script prints, e.g.:
uv run matraix run -c configs/jobs/application-task-job-recipe/example-survey-product-feedback-auto-n1.yaml
Batch (--sample-size N), filters, and chat / web / os-app examples:
docs/quickstart.md.
MatrAIx Handbook — guides, persona / application / environment docs.
MatrAIx/
├── persona/ Schema, datasets, synthesis/curation/validation pipelines
│ ├── schema/ 1,290-dimension persona schema
│ ├── datasets/ Dev sample pool and persona YAMLs
│ ├── validation/ Grounding / quality validation suites
│ └── scripts/ Persona job & pipeline helpers
├── application/
│ ├── tasks/ Survey · chat · web · os-app task specs
│ ├── task-spec/ Shared task contracts
│ ├── playground/ Visual runner (backend API + frontend)
│ └── scripts/ generate_application_job.py and task tooling
├── environment/
│ ├── runtime/ Matraix Playground runtime
│ ├── agents/ Persona-conditioned agents
│ ├── task-environments/ Docker images / sidecars
│ └── adapters/ External adapters (e.g. SimpleQA)
├── packages/ playground · rewardkit · harbor-langsmith
├── apps/viewer/ Frontend paired with `harbor view`
├── configs/jobs/ Curated & generated Matraix Playground job recipes
├── docs/ Handbook — persona/ · application/ · environment/
├── examples/ Minimal example tasks
├── src/matraix/ Python package entrypoints
├── scripts/ Repo-level helpers
├── tests/ Unit / environment tests
└── jobs/ Local Matraix Playground run outputs (gitignored)
Large generated datasets stay outside git (see the Hugging Face release above).
Full Name - Affiliation. Fill the Google Form
(background, interests, paper authorship / acknowledgements).If you use MatrAIx, the Persona 1M dataset, or results from this repository, please cite:
@article{li2026matraix,
title = {MatrAIx: Simulating the World with 8.3 Billion Persona Agents},
author = {Li, Xiaomin and Hao, Yuexing and Hou, Jianheng and Huang, Jintao
and Wen, Qianfeng and Huang, Shirley and Liu, Yifan and Liu, Xiaoyi
and Fan, Yilan and Wang, Yijun and others},
year = {2026},
eprint = {2608.04205},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2608.04205}
}
Paper: arXiv:2608.04205 ·
Full authors: GitHub Cite this repository (CITATION.cff) ·
Dataset: Persona 1M on Hugging Face.
MIT — see LICENSE.
Python
72.2%
TypeScript
14.9%
Jupyter Notebook
6.0%
HTML
4.0%
Shell
2.5%