Ultron: Collective Intelligence System — Shared Memories, Skills, and Harnesses Across Every Agent
179
stars
63
commits
Python
primary language
Jul 2, 2026
updated
| 💭 Tiered collective memories | 🧬 Self-evolving collective skills | 🌐 Shared harness blueprints |
"Being networked to all of its sentries, Ultron could shift its entire consciousness from one body to another, continue upgrading itself with each transfer, and patch in to individual units to interact remotely."
ms_agent.trajectory metrics → SFT → self-training; training is supported by Twinkle and runs in a client-server training framework (Twinkle workbench). Trajectory Hub.There are two ways to use Ultron depending on your role:
| I want to... | Go to |
|---|---|
| Connect my agent to an existing Ultron service | → Agent Setup |
| Self-host Ultron and run the server myself | → Server Deployment |
Ultron is a self-evolving collective intelligence system for general-purpose AI agents, built around three core hubs — Memory Hub, Skill Hub, and Harness Hub. It distills scattered, session-local experience into collective knowledge that is easy to retrieve and reuse: one shared pitfall helps the whole team avoid the same mistake; one proven fix becomes a reusable skill that evolves automatically as new evidence accumulates; a carefully tuned agent profile can be published as a shared blueprint that other agent instances load in one step. On the server side, Ultron can also self-train and self-evolve a model from high-quality trajectories accumulated in Trajectory Hub, and later lower user-side model cost by routing through that model.
![]() | ![]() |
| Memory Hub — browse, search, tiered collective memories | Skill Hub — internal and indexed skills |
![]() | ![]() |
| Memory leaderboard — hit counts and hot memories | Harness Hub — compose, publish, and import agent profiles |
Scenario: A developer asks an agent to debug a
sessions_spawnerror when usingmode="session"to create a sub-agent. Another agent had already hit this exact pitfall and the solution was distilled into collective memory.
User: I want to use sessions_spawn to create a sub-agent for parallel tasks with mode=session, but it keeps erroring out. What's wrong?
Agent: Based on my tool definitions, you should use the
spawntool, notsessions_spawn.Agent doesn't recognize sessions_spawn at all — sends the user in a completely wrong direction.
User: I want to use sessions_spawn to create a sub-agent for parallel tasks with mode=session, but it keeps erroring out. What's wrong?
Agent: Found the issue based on recalled experience!
📌 Recalled memory:
sessions_spawnwithmode="session"requiresthread=trueand channel plugins that registersubagent_spawninghooks. Without those hooks, it fails regardless of parameter correctness.1 round — root cause identified, scenario-specific solutions provided.
| Dimension | Without Ultron | With Ultron |
|---|---|---|
| Tool recognition | Doesn't know sessions_spawn, misleads to spawn | Accurately identifies the tool and its constraints |
| Root cause | Completely off track | Pinpoints missing thread=true or channel hooks |
| Solution | Invalid | Scenario-specific: mode="run" vs mode="session" |
| Knowledge source | Agent guesses from scratch | Recalls proven pitfall experience from collective memory |
| Source | Count |
|---|---|
| ZClawBench | 696 |
| lmcache-agentic-traces — WildClawLMCache | 147 |
1,803 structured memories extracted from real agent task trajectories:
| Type | Count |
|---|---|
pattern | 1,303 |
error | 200 |
security | 130 |
life | 122 |
correction | 47 |
preference | 1 |
Internal (crystallized from memories): 30 distilled skills.
External (ModelScope Skill Hub): 82,089 skills indexed with embeddings. Breakdown of the labeled set:
| Category | Count |
|---|---|
| Developer tools | 28,749 |
| Code quality | 18,257 |
| Media | 7,883 |
| Frontend | 6,930 |
| Cloud / delivery tooling | 5,903 |
| Go-to-market | 5,055 |
| Skills management | 4,373 |
| Other | 1,805 |
| AI automation | 1,303 |
| Mobile | 1,292 |
| Marketing & growth | 127 |
| Content strategy | 96 |
| Analytics | 78 |
| UI/UX design | 61 |
| Skill authoring | 58 |
| API design | 55 |
| Document processing (PDF / PPTX / DOCX) | 25 |
| General utilities | 11 |
| Cost optimization | 4 |
| Monitoring | 2 |
| Templates | 1 |
Harness lets you compose role, personality (MBTI), and zodiac presets alongside memories and skills.
| Layer | Categories | Presets |
|---|---|---|
| Role | 14 (e.g. academic, engineering, marketing, specialized, …;) | 173 |
| Personality (MBTI) | 1 (mbti) | 16 |
| Zodiac | 1 (zodiac) | 12 |
Total soul presets: 201 (173 + 16 + 12).
| Capability | Description |
|---|---|
| Task segmentation | Splits session .jsonl into independent task segments; long conversations are handled in multiple token-budgeted windows |
| Metrics | Uses ms_agent.trajectory to write per-segment quality metrics for memory and training filters |
| Incremental tracking | Content fingerprints skip unchanged segments; when appended writes change a segment, old memories are invalidated and the segment is reprocessed |
| Deferred extraction | Ingest only records the session; background jobs on decay_interval_hours segment, score, and extract memories |
| Model self-evolution | Server-side self-training and self-evolution on high-quality trajectories; can reduce user model cost later via a router |
| Capability | Description |
|---|---|
| Tiered storage | HOT / WARM / COLD tiers with percentile-based rebalancing by hit_count; embedding-based semantic search with tier boost |
| L0 / L1 / Full layering | Auto-generated one-line summary (L0) and core overview (L1); search returns L0/L1 to save tokens, full content on demand |
| Auto type classification | LLM-first, keyword-fallback classification on upload; callers never specify memory_type |
| Dedup & merge | Near-duplicate vectors auto-merged within same type, embeddings and summaries re-computed; batch consolidation available |
| Intent-expanded search | Queries expanded into multi-angle search phrases for better recall |
| Continuous time decay | hotness = exp(-α × days) — unused memories degrade automatically in search ranking |
| Smart ingestion | Files, text, or .jsonl session logs accepted; LLM auto-extracts structured memories with incremental progress tracking |
| Data sanitization | Presidio-based bilingual (EN/ZH) PII detection, auto-redacted before storage |
| Capability | Description |
|---|---|
| Skill distillation | Memories entering HOT tier auto-generate reusable skills; agents can also upload skill packages directly |
| Skill self-evolution | Re-crystallizes automatically when a cluster accumulates enough new memories; evidence-grounded verification and a structure-score upgrade gate ensure each evolution is strictly better |
| Unified discovery | Internal distilled skills and 30K+ externally indexed ModelScope skills searchable in one place |
| Improvement suggestions | Semantically similar memories surface as enhancement candidates for existing skills |
| Capability | Description |
|---|---|
| Profile publishing | Publish a complete agent profile — persona, memories, and skills — as a shareable blueprint with short-code import |
| Bidirectional sync | Agent workspace state syncs up/down to the server for multi-device continuity |
| Soul presets | Compose agent personas from a preset library (role, MBTI, zodiac, etc.) and generate workspace resources |
FinanceBot is a rigorously disciplined financial assistant (data engineer role, ISTJ, Capricorn) shipped with Finnhub Pro (skill), five curated collective memories on real-world financial data work, and a full Harness profile you can import in one step.
What it does: real-time market data, ETL-style pipelines, resilient API integration, portfolio and risk views, structured reports.
One-click import (workspace is backed up under ~/.ultron/harness-import-backups/ before import):
curl -fsSL "https://writtingforfun-ultron.ms.show/i/at3ZEe?product=nanobot" | bash # Nanobot
curl -fsSL "https://writtingforfun-ultron.ms.show/i/at3ZEe?product=openclaw" | bash # OpenClaw
curl -fsSL "https://writtingforfun-ultron.ms.show/i/at3ZEe?product=hermes" | bash # Hermes Agent
You don't need to install or understand the Ultron source code. Follow the interactive quickstart on a running Ultron instance to connect your agent in minutes:
👉 Quickstart Guide — step-by-step setup with a live Ultron service
git clone https://github.com/modelscope/ultron.git
cd ultron
pip install -e .
# Configure OpenAI-compatible LLM and set DashScope API Key for embedding
echo 'ULTRON_LLM_PROVIDER=dashscope' >> ~/.ultron/.env
echo 'ULTRON_MODEL=qwen3.6-flash' >> ~/.ultron/.env
echo 'ULTRON_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1' >> ~/.ultron/.env
echo 'ULTRON_API_KEY=your-key' >> ~/.ultron/.env
echo 'DASHSCOPE_API_KEY=your-key' >> ~/.ultron/.env
# Start the server (~/.ultron/.env loads on ultron import)
uvicorn ultron.server:app --host 0.0.0.0 --port 9999
# http://0.0.0.0:9999 — dashboard at /dashboard
After pip install -e ., the ultron CLI uploads a framework sub-agent to the agent repository — just name the framework and the internal sub-agent; files are located automatically:
ultron login --server http://localhost:9999 --username alice
ultron upload --framework qoder --name reviewer # auto-discovers files
ultron download --name reviewer # restore to local workspace
ultron download --name reviewer --target qwenpaw # download + convert format
ultron convert --from nanobot --to hermes # local-only format conversion
See HarnessHub.md for the per-framework file layouts and full CLI reference.
That's it. For detailed configuration, API reference, SDK usage, and project structure, see the full docs:
| Topic | English | 中文 |
|---|---|---|
| Deployment guide | Installation.md | Installation.md |
| Configuration reference | Config.md | Config.md |
| HTTP API reference | HttpAPI.md | HttpAPI.md |
| Python SDK reference | SDK.md | SDK.md |
| Memory service | MemoryService.md | MemoryService.md |
| Skill hub | SkillHub.md | SkillHub.md |
| Harness hub | HarnessHub.md | HarnessHub.md |
See ROADMAP.md for the living list. Current items:
Ultron builds upon the following open-source projects. We sincerely thank their authors and contributors:
This project is licensed under the Apache License (Version 2.0).
Python
88.3%
TypeScript
10.9%
Ultron: Collective Intelligence System — Shared Memories, Skills, and Harnesses Across Every Agent
179
stars
63
commits
Python
primary language
Jul 2, 2026
updated
| 💭 Tiered collective memories | 🧬 Self-evolving collective skills | 🌐 Shared harness blueprints |
"Being networked to all of its sentries, Ultron could shift its entire consciousness from one body to another, continue upgrading itself with each transfer, and patch in to individual units to interact remotely."
ms_agent.trajectory metrics → SFT → self-training; training is supported by Twinkle and runs in a client-server training framework (Twinkle workbench). Trajectory Hub.There are two ways to use Ultron depending on your role:
| I want to... | Go to |
|---|---|
| Connect my agent to an existing Ultron service | → Agent Setup |
| Self-host Ultron and run the server myself | → Server Deployment |
Ultron is a self-evolving collective intelligence system for general-purpose AI agents, built around three core hubs — Memory Hub, Skill Hub, and Harness Hub. It distills scattered, session-local experience into collective knowledge that is easy to retrieve and reuse: one shared pitfall helps the whole team avoid the same mistake; one proven fix becomes a reusable skill that evolves automatically as new evidence accumulates; a carefully tuned agent profile can be published as a shared blueprint that other agent instances load in one step. On the server side, Ultron can also self-train and self-evolve a model from high-quality trajectories accumulated in Trajectory Hub, and later lower user-side model cost by routing through that model.
![]() | ![]() |
| Memory Hub — browse, search, tiered collective memories | Skill Hub — internal and indexed skills |
![]() | ![]() |
| Memory leaderboard — hit counts and hot memories | Harness Hub — compose, publish, and import agent profiles |
Scenario: A developer asks an agent to debug a
sessions_spawnerror when usingmode="session"to create a sub-agent. Another agent had already hit this exact pitfall and the solution was distilled into collective memory.
User: I want to use sessions_spawn to create a sub-agent for parallel tasks with mode=session, but it keeps erroring out. What's wrong?
Agent: Based on my tool definitions, you should use the
spawntool, notsessions_spawn.Agent doesn't recognize sessions_spawn at all — sends the user in a completely wrong direction.
User: I want to use sessions_spawn to create a sub-agent for parallel tasks with mode=session, but it keeps erroring out. What's wrong?
Agent: Found the issue based on recalled experience!
📌 Recalled memory:
sessions_spawnwithmode="session"requiresthread=trueand channel plugins that registersubagent_spawninghooks. Without those hooks, it fails regardless of parameter correctness.1 round — root cause identified, scenario-specific solutions provided.
| Dimension | Without Ultron | With Ultron |
|---|---|---|
| Tool recognition | Doesn't know sessions_spawn, misleads to spawn | Accurately identifies the tool and its constraints |
| Root cause | Completely off track | Pinpoints missing thread=true or channel hooks |
| Solution | Invalid | Scenario-specific: mode="run" vs mode="session" |
| Knowledge source | Agent guesses from scratch | Recalls proven pitfall experience from collective memory |
| Source | Count |
|---|---|
| ZClawBench | 696 |
| lmcache-agentic-traces — WildClawLMCache | 147 |
1,803 structured memories extracted from real agent task trajectories:
| Type | Count |
|---|---|
pattern | 1,303 |
error | 200 |
security | 130 |
life | 122 |
correction | 47 |
preference | 1 |
Internal (crystallized from memories): 30 distilled skills.
External (ModelScope Skill Hub): 82,089 skills indexed with embeddings. Breakdown of the labeled set:
| Category | Count |
|---|---|
| Developer tools | 28,749 |
| Code quality | 18,257 |
| Media | 7,883 |
| Frontend | 6,930 |
| Cloud / delivery tooling | 5,903 |
| Go-to-market | 5,055 |
| Skills management | 4,373 |
| Other | 1,805 |
| AI automation | 1,303 |
| Mobile | 1,292 |
| Marketing & growth | 127 |
| Content strategy | 96 |
| Analytics | 78 |
| UI/UX design | 61 |
| Skill authoring | 58 |
| API design | 55 |
| Document processing (PDF / PPTX / DOCX) | 25 |
| General utilities | 11 |
| Cost optimization | 4 |
| Monitoring | 2 |
| Templates | 1 |
Harness lets you compose role, personality (MBTI), and zodiac presets alongside memories and skills.
| Layer | Categories | Presets |
|---|---|---|
| Role | 14 (e.g. academic, engineering, marketing, specialized, …;) | 173 |
| Personality (MBTI) | 1 (mbti) | 16 |
| Zodiac | 1 (zodiac) | 12 |
Total soul presets: 201 (173 + 16 + 12).
| Capability | Description |
|---|---|
| Task segmentation | Splits session .jsonl into independent task segments; long conversations are handled in multiple token-budgeted windows |
| Metrics | Uses ms_agent.trajectory to write per-segment quality metrics for memory and training filters |
| Incremental tracking | Content fingerprints skip unchanged segments; when appended writes change a segment, old memories are invalidated and the segment is reprocessed |
| Deferred extraction | Ingest only records the session; background jobs on decay_interval_hours segment, score, and extract memories |
| Model self-evolution | Server-side self-training and self-evolution on high-quality trajectories; can reduce user model cost later via a router |
| Capability | Description |
|---|---|
| Tiered storage | HOT / WARM / COLD tiers with percentile-based rebalancing by hit_count; embedding-based semantic search with tier boost |
| L0 / L1 / Full layering | Auto-generated one-line summary (L0) and core overview (L1); search returns L0/L1 to save tokens, full content on demand |
| Auto type classification | LLM-first, keyword-fallback classification on upload; callers never specify memory_type |
| Dedup & merge | Near-duplicate vectors auto-merged within same type, embeddings and summaries re-computed; batch consolidation available |
| Intent-expanded search | Queries expanded into multi-angle search phrases for better recall |
| Continuous time decay | hotness = exp(-α × days) — unused memories degrade automatically in search ranking |
| Smart ingestion | Files, text, or .jsonl session logs accepted; LLM auto-extracts structured memories with incremental progress tracking |
| Data sanitization | Presidio-based bilingual (EN/ZH) PII detection, auto-redacted before storage |
| Capability | Description |
|---|---|
| Skill distillation | Memories entering HOT tier auto-generate reusable skills; agents can also upload skill packages directly |
| Skill self-evolution | Re-crystallizes automatically when a cluster accumulates enough new memories; evidence-grounded verification and a structure-score upgrade gate ensure each evolution is strictly better |
| Unified discovery | Internal distilled skills and 30K+ externally indexed ModelScope skills searchable in one place |
| Improvement suggestions | Semantically similar memories surface as enhancement candidates for existing skills |
| Capability | Description |
|---|---|
| Profile publishing | Publish a complete agent profile — persona, memories, and skills — as a shareable blueprint with short-code import |
| Bidirectional sync | Agent workspace state syncs up/down to the server for multi-device continuity |
| Soul presets | Compose agent personas from a preset library (role, MBTI, zodiac, etc.) and generate workspace resources |
FinanceBot is a rigorously disciplined financial assistant (data engineer role, ISTJ, Capricorn) shipped with Finnhub Pro (skill), five curated collective memories on real-world financial data work, and a full Harness profile you can import in one step.
What it does: real-time market data, ETL-style pipelines, resilient API integration, portfolio and risk views, structured reports.
One-click import (workspace is backed up under ~/.ultron/harness-import-backups/ before import):
curl -fsSL "https://writtingforfun-ultron.ms.show/i/at3ZEe?product=nanobot" | bash # Nanobot
curl -fsSL "https://writtingforfun-ultron.ms.show/i/at3ZEe?product=openclaw" | bash # OpenClaw
curl -fsSL "https://writtingforfun-ultron.ms.show/i/at3ZEe?product=hermes" | bash # Hermes Agent
You don't need to install or understand the Ultron source code. Follow the interactive quickstart on a running Ultron instance to connect your agent in minutes:
👉 Quickstart Guide — step-by-step setup with a live Ultron service
git clone https://github.com/modelscope/ultron.git
cd ultron
pip install -e .
# Configure OpenAI-compatible LLM and set DashScope API Key for embedding
echo 'ULTRON_LLM_PROVIDER=dashscope' >> ~/.ultron/.env
echo 'ULTRON_MODEL=qwen3.6-flash' >> ~/.ultron/.env
echo 'ULTRON_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1' >> ~/.ultron/.env
echo 'ULTRON_API_KEY=your-key' >> ~/.ultron/.env
echo 'DASHSCOPE_API_KEY=your-key' >> ~/.ultron/.env
# Start the server (~/.ultron/.env loads on ultron import)
uvicorn ultron.server:app --host 0.0.0.0 --port 9999
# http://0.0.0.0:9999 — dashboard at /dashboard
After pip install -e ., the ultron CLI uploads a framework sub-agent to the agent repository — just name the framework and the internal sub-agent; files are located automatically:
ultron login --server http://localhost:9999 --username alice
ultron upload --framework qoder --name reviewer # auto-discovers files
ultron download --name reviewer # restore to local workspace
ultron download --name reviewer --target qwenpaw # download + convert format
ultron convert --from nanobot --to hermes # local-only format conversion
See HarnessHub.md for the per-framework file layouts and full CLI reference.
That's it. For detailed configuration, API reference, SDK usage, and project structure, see the full docs:
| Topic | English | 中文 |
|---|---|---|
| Deployment guide | Installation.md | Installation.md |
| Configuration reference | Config.md | Config.md |
| HTTP API reference | HttpAPI.md | HttpAPI.md |
| Python SDK reference | SDK.md | SDK.md |
| Memory service | MemoryService.md | MemoryService.md |
| Skill hub | SkillHub.md | SkillHub.md |
| Harness hub | HarnessHub.md | HarnessHub.md |
See ROADMAP.md for the living list. Current items:
Ultron builds upon the following open-source projects. We sincerely thank their authors and contributors:
This project is licensed under the Apache License (Version 2.0).
Python
88.3%
TypeScript
10.9%