Graph of Skills (GoS)

Dependency-Aware Structural Retrieval for Massive Agent Skills

Dawei Liu^* · Zongxia Li^* · Hongyang Du · Xiyang Wu · Shihang Gui · Yongbei Kuang · Lichao Sun

🔥 Updates

• [2026-08-20] 🎉 Graph of Skills is accepted to EMNLP 2026 Main Conference (Budapest). The camera-ready adds a component ablation, a budget-matched retrieval study, library-size scaling to 2,000 skills, and graph-construction diagnostics — all summarized in Results.
• [2026-04-15] Released a Claude Code MCP plugin for graph of skills retrieval — drop-in integration for Claude Code agents.
• [2026-04-07] Paper released on arXiv:2604.05333 and HuggingFace Papers.
• [2026-04-06] Code open-sourced on GitHub.
• [2026-04-04] Skill libraries, prebuilt workspaces, and benchmark data released on HuggingFace.

Overview

Graph of Skills builds a skill graph offline from a library of SKILL.md documents, then retrieves a small, ranked set of relevant skills at task time. Instead of flooding the agent context with an entire skill library, GoS surfaces only the skills most likely to help -- along with their prerequisites and related capabilities.

How It Works

Retrieval pipeline:

1. Seed -- retrieve semantic candidates (embedding similarity) and lexical candidates (exact-match tokens)
2. Merge -- combine both candidate pools
3. Rerank -- rerank using the skill-graph structure (dependencies, co-occurrence)
4. Return -- emit a capped, agent-readable skill bundle

Results

GoS is evaluated on SkillsBench (87 dockerized coding tasks) and ALFWorld (140 episodes) across three model families. R = average reward (%), T = average total tokens (uncached input + cached input + output), S = agent-only runtime (s). ↑ higher is better, ↓ lower is better. All numbers are means over two runs per setting.

Main results (end-to-end)

GoS achieves the highest reward on every model on both benchmarks while cutting total tokens by up to 56× (ALFWorld, Claude Sonnet 4.5) vs. Vanilla Skills.

Scaling with library size

Reward and cost as the repository grows (SkillsBench, GPT-5.2 Codex). T in millions of tokens.

From 500 skills onward GoS leads both baselines by 5.4, 7.0, and 4.6 reward points. Flat exposure rises from 1.93M to 5.84M tokens (~3x) across the same range while both retrievers stay near-constant — the scaling burden is in the prompt, not in retrieval.

Component ablation

Full SkillsBench, GPT-5.2 Codex, 1,000-skill library.

Traversal direction matters more than the graph itself. Replacing reverse traversal with forward propagation costs 9.1 reward points — a larger loss than removing graph propagation entirely (5.1). The gain comes from traversing dependencies backwards, not from graph diffusion as such.

Isolating the graph operation

The table above compares full agent runs, which entangle the graph with hydration, reranking, and the agent interface. This budget-matched offline study holds seeding, reranker, top-N selection, character budgets, and hydration fixed across all arms, so only the graph operation changes (200-skill library, 87 queries).

R_avail / R_full = available- and full-oracle recall, C = exact bundle completeness, D = dependency-pair co-recovery. The same ordering reproduces at retrieval level: forward propagation is worse than not using the graph at all, and dependency-pair co-recovery falls from 0.654 to 0.362.

Construction cost and robustness

• One-time offline cost: $0.29 at 200 skills, $2.36 at 2,000. Online propagation adds ~1.5 ms at p50.
• Threshold is not delicate: sweeping the acceptance threshold ζ from 0.4 to 0.8 moves the deterministic edge count from 172 to 46 while changing recall by less than 0.005.
• Documentation quality is the main limiting factor: dropping half of all I/O fields costs 0.024 recall, whereas dropping half of the descriptions costs 0.146.

Citation

If you find this work useful, please cite the EMNLP 2026 version:

@inproceedings{liu2026graphofskills,
  title     = {Graph-of-Skills: Dependency-Aware Structural Retrieval for Massive Agent Skills},
  author    = {Liu, Dawei and Li, Zongxia and Du, Hongyang and Wu, Xiyang and
               Gui, Shihang and Kuang, Yongbei and Sun, Lichao},
  booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year      = {2026}
}

@misc{liu2026graphofskillsarxiv,
  title         = {Graph-of-Skills: Dependency-Aware Structural Retrieval for Massive Agent Skills},
  author        = {Dawei Liu and Zongxia Li and Hongyang Du and Xiyang Wu and
                   Shihang Gui and Yongbei Kuang and Lichao Sun},
  year          = {2026},
  eprint        = {2604.05333},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI},
  url           = {https://arxiv.org/abs/2604.05333}
}

Installation

Requirements

• Python 3.10 -- 3.12
• uv (recommended) or pip
• An embedding API key (OpenAI, Gemini, or any OpenAI-compatible provider)

Setup

git clone https://github.com/davidliuk/graph-of-skills.git
cd graph-of-skills
uv sync
cp .env.example .env   # then fill in your API keys

OPENAI_API_KEY=sk-...
# Use the ``openai/...`` prefix so LiteLLM targets the OpenAI API (omit OPENAI_BASE_URL).
GOS_EMBEDDING_MODEL=openai/text-embedding-3-large
GOS_EMBEDDING_DIM=3072

OPENROUTER_API_KEY=<openrouter-key>
OPENAI_BASE_URL=https://openrouter.ai/api/v1
GOS_EMBEDDING_MODEL=openrouter/openai/text-embedding-3-large
GOS_EMBEDDING_DIM=3072

OPENAI_API_KEY=<azure-api-key>
OPENAI_BASE_URL=https://YOUR-RESOURCE.services.ai.azure.com/openai/v1
# Must match your **deployment name** in Azure (not necessarily ``text-embedding-3-large``).
GOS_EMBEDDING_MODEL=openai/<your-deployment-name>
GOS_EMBEDDING_DIM=<vector-dimension-for-that-model>

GEMINI_API_KEY=<your-key>
GOS_EMBEDDING_MODEL=gemini/gemini-embedding-001
GOS_EMBEDDING_DIM=3072

Quick Start

Goal: install the package, pull the published skill libraries, build (or download) a graph workspace, then run retrieval from the shell.

Read next: DATA.md for every download flag and asset size; .env.example for embedding providers. After GoS works locally, use evaluation/README.md for benchmark runners and evaluation/skillsbench/README.md for Harbor-based SkillsBench.

Step 0: Install (once per machine)

Complete Installation above: clone, uv sync, cp .env.example .env, and set embedding (and optional LLM) keys. Indexing and retrieval load .env from the repo root when you use uv run gos ….

Step 1: Download skill libraries

The collections skills_200, skills_500, skills_1000, skills_2000 are directories of SKILL.md files on HuggingFace, not in git. They unpack to:

• data/skillsets/skills_200/ … data/skillsets/skills_2000/

./scripts/download_data.sh --skillsets

This tries each archive, skips directories that already have files, and logs [skip] if an archive is not yet on the Hub. Gated datasets: HF_TOKEN=hf_... ./scripts/download_data.sh --skillsets. Full reference (tasks, workspaces, selective flags): DATA.md.

Tiny smoke test without HuggingFace: index the built-in folder skills/ (only a few skills) with any --workspace path you like.

Step 2: Workspace directory layout (recommended for benchmarks)

--workspace is where GoS stores the indexed graph (vectors + graph storage). Use the same path for gos retrieve, gos status, and gos add.

For ALFWorld and SkillsBench defaults, keep this mapping (see evaluation/README.md and evaluation/skillsbench/graphskills_benchmark.py):

Step 3: Get a workspace (choose one path)

A. Build locally (needs embedding API; duration grows with library size):

mkdir -p data/gos_workspace
uv run gos index data/skillsets/skills_200 \
  --workspace data/gos_workspace/skills_200_v1 --clear

Use the matching pair for other sets (e.g. skills_1000 → data/gos_workspace/skills_1000_v1). Embedding model and dimension in .env must stay the same for later retrieval (see Configuration).

B. Download a prebuilt workspace (no gos index; must match the embedding used to build that archive):

./scripts/download_data.sh --workspace

Compatibility note: the legacy *_v1 archives were serialized by an undirected cleanup pipeline and are retained only for reproducing earlier results. The repaired typed-directed constructor rejects them because edge direction cannot be recovered safely. Build a fresh workspace from the downloaded skill library for current GoS retrieval and new experiments.

See DATA.md for which gos_workspace_skills_*_v1.tar.gz files exist on the Hub and how they map to data/gos_workspace/.

Step 4: Retrieve

uv run gos retrieve "parse binary STL file, calculate volume and mass" \
  --workspace data/gos_workspace/skills_200_v1 --max-skills 5

Step 5: Inspect or extend

uv run gos status --workspace data/gos_workspace/skills_200_v1
uv run gos add path/to/NEW_SKILL.md --workspace data/gos_workspace/skills_200_v1
uv run gos relink --workspace data/gos_workspace/skills_200_v1 \
  --concurrency 8 --checkpoint-every 10 --resume

gos relink writes three secret-free telemetry artifacts beside the graph:

• relink_progress.json: atomic resume ledger and cumulative counters;
• relink_events.jsonl: append-only attempt/checkpoint events with token, cost, timing, throughput, per-batch focus results, and failures;
• construction_report.json: final graph, provenance, configuration, aggregate usage, stage timing, and throughput summary.

The console checkpoint line also reports cumulative calls, input/output tokens, provider cost, failures, edges, and elapsed wall time. Logs never include API keys, request headers, prompts, raw skill documents, or model responses.

Step 6: What to run next

• End-to-end sanity check (retrieval + one Docker task): see the Minimal Verification collapsible in the Evaluation section.
• Paper benchmarks (ALFWorld, SkillsBench): evaluation/README.md (overview) and evaluation/skillsbench/README.md (Harbor / task generation).

Agent Integration

Claude Code (MCP Plugin)

GoS ships with a built-in MCP server that gives Claude Code direct access to the skill graph. When you open this project, Claude Code auto-discovers the server via .mcp.json — no manual setup.

Quick start:

uv sync                              # install deps (once)
cp .env.example .env                 # fill in API keys
./scripts/download_data.sh --workspace   # download prebuilt workspaces

Then open the project in Claude Code. The graph-of-skills MCP server is ready. Ask naturally:

"Find skills for processing 3D mesh files with GoS, then follow the skill instructions to complete the task."

Claude Code will call the GoS tools, retrieve relevant skills, and follow their instructions autonomously.

claude mcp add graph-of-skills -- uv run --directory /path/to/graph-of-skills gos-claude

# Or with an explicit workspace
claude mcp add graph-of-skills -- uv run --directory /path/to/graph-of-skills gos-claude \
  --workspace data/gos_workspace/skills_200_v1

Available tools:

See CLAUDE.md for the full integration guide, example workflows, and configuration.

Docker / Container (SkillsBench)

Inside a Docker container, an agent calls graphskills-query with a natural-language task description and receives a bounded skill bundle:

graphskills-query "parse binary STL file and calculate mass"

Each returned skill includes a Source: path the agent can open directly:

Source: /opt/graphskills/skills/mesh-analysis/SKILL.md

Set GOS_SKILLS_DIR to control path rewriting, so the same workspace can be indexed on a host and queried inside a container.

CLI Reference

Configuration

All runtime settings are driven by environment variables. See .env.example for the full template. Download paths and workspace layout on disk are documented in DATA.md.

Note: The embedding model at retrieval time must match the model used when the workspace was indexed.

Evaluation

Docs: Start with evaluation/README.md (all tracks, modes, env vars). For SkillsBench + Harbor only, use evaluation/skillsbench/README.md. Dataset files and scripts are described in DATA.md.

We evaluate GoS on two benchmarks:

For running these evaluations, we recommend routing the agent's chat / completion API through OpenRouter: use an OpenAI-compatible BASE_URL (for example https://openrouter.ai/api/v1) and the API key your runner documents. The GoS project's own evaluation testing is done mainly this way. Embeddings for indexing and retrieval are separate; configure them in .env as in .env.example (OpenRouter, direct OpenAI, Gemini, or Azure).

Benchmark data is hosted externally and not included in this repository:

./scripts/download_data.sh          # download all assets (~780 MB); options in DATA.md

Selective downloads and workspace rebuild steps: DATA.md.

uv run gos retrieve "unit tests with pytest" \
  --workspace data/gos_workspace/skills_200_v1 --max-skills 3

uv run python evaluation/skillsbench/graphskills_benchmark.py \
  --skillset-name skills_200 \
  --task dialogue-parser \
  --skip-allskills --skip-vectorskills \
  --output-root evaluation/skillsbench/generated_verify

cd evaluation/skillsbench
set -a && source ../../.env && set +a
harbor run --agent gemini-cli \
  --model gemini/gemini-3-flash-preview \
  --force-build \
  -p generated_verify/tasks_graph_skills/dialogue-parser \
  -o jobs/verify-sample

Repository Layout

graph-of-skills/
├── gos/                          # Core GoS package
│   ├── core/                     #   Engine, retrieval, parsing, schema
│   ├── interfaces/               #   CLI, MCP server, Claude Code plugin
│   └── utils/                    #   Configuration (pydantic-settings)
├── data/                         # Downloaded data (gitignored; see DATA.md)
│   ├── skillsets/                #   Skill libraries (skills_200, 500, 1000, 2000)
│   └── gos_workspace/            #   Indexed or prebuilt graph workspaces
├── evaluation/                   # See evaluation/README.md
│   ├── alfworld_run.py           #   ALFWorld benchmark runner
│   ├── skill.py                  #   SkillModule adapter for GoS
│   ├── analysis/                 #   Offline analyses — evaluation/analysis/README.md
│   └── skillsbench/              #   SkillsBench — evaluation/skillsbench/README.md
│       └── experiments/          #     Ablation conditions, batch configs, dashboard
├── skills/                       # Agent bootstrap skills for retrieval
├── scripts/                      # Utility scripts (data download, etc.)
├── tests/                        # Test suite
├── pyproject.toml                # Package definition & CLI entry points
├── .env.example                  # Environment variable template
├── DATA.md                       # Data & downloads
└── CLAUDE.md                     # Claude Code / MCP integration guide

Documentation

License

This project is licensed under the MIT License.