Maki

Maki is a Python framework for building multi-agent LLM applications. It supports multiple LLM backends (Ollama, OpenAI, Anthropic, HuggingFace), a plugin system with 16 built-in tools, a workflow engine with dependency resolution and parallel execution, and a distributed layer for serving agents over HTTP.

---

Architecture

The framework is organized into four layers:

• Public API — maki/__init__.py lazy-loads all exports on first access
• LLM Backends — MakiLLama, MakiOpenAI, MakiAnthropic, and HFBackend all implement the abstract LLMBackend contract
• Agent System — Agent composes PluginHandler and ReasoningEngine mixins; AgentManager orchestrates agents via WorkflowTask and WorkflowState
• Distributed Layer — AgentServer (FastAPI) exposes agents over HTTP; AgentProxy provides a remote-agent client with circuit-breaking; DistributedAgentManager mixes local and remote agents
• Infrastructure — Connector (SSRF-protected HTTP with connect-time IP pinning), shared data classes, typed exceptions, runtime config, and structured logging

The Plugin System sits alongside the Agent layer: plugins are loaded on demand and invoked automatically when the LLM emits a TOOL: directive or via native tool-calling APIs (Ollama, OpenAI, Anthropic).

---

Features

• MakiLLama — Ollama chat API with synchronous, streaming, async, and vision-capable workflows
• MakiOpenAI — OpenAI chat completions, including reasoning models (o3/o4)
• MakiAnthropic — Anthropic messages API (Claude Sonnet, Haiku, Opus)
• HFBackend — direct HuggingFace Transformers integration with quantization and device selection
• Agent — role-based agents with task execution, memory, reasoning, and plugin support; per-agent execution lock for concurrent safety
• AgentManager — multi-agent orchestration: sequential pipelines, collaborative tasks, and dependency-aware workflows with parallel batching and checkpoint/resume
• ConversationMemory — token-budgeted, pair-based conversation history shared by Agent (stateful mode) and ChatSession
• Native tool-calling for all backends (Ollama tools=, OpenAI, Anthropic tool use) with multi-round execution and self-correction
• 16 built-in plugins covering files, web content, search, trading, and memory
• Distributed agent serving: maki serve exposes any agent over HTTP; AgentProxy consumes remote agents transparently
• SSRF-protected HTTP connector with DNS pinning, error classification, and configurable timeouts
• Fail-closed plugin security: every plugin declares ALLOWED_METHODS; destructive methods require explicit opt-in
• Explicit logging setup and a typed exception hierarchy

---

Installation

curl -fsSL https://raw.githubusercontent.com/BowlOfData/maki/main/install.sh | bash

This clones the repo into ~/.maki, installs it into an isolated virtual environment, and symlinks the maki command into ~/.local/bin (override with the MAKI_INSTALL_DIR / MAKI_BIN_DIR env vars). Re-run it any time to update to the latest main.

From source

pip install -e .

For development tools:

pip install -e ".[dev]"

Some built-in plugins and backends rely on optional extras (defined in pyproject.toml):

• maki-framework[web] — feedparser, readability-lxml, html2text (web search / web-to-Markdown)
• maki-framework[trends] — pytrends (Google Trends)
• maki-framework[alpaca] — alpaca-py (market data, news, trading, streaming)
• maki-framework[ftp] — paramiko (FTP/SFTP)
• maki-framework[gui] — PySide6 (desktop GUI)
• maki-framework[openai] — openai (OpenAI backend)
• maki-framework[anthropic] — anthropic (Anthropic backend)
• maki-framework[distributed] — fastapi, uvicorn, pyyaml (agent server and proxies)
• maki-framework[distributed-redis] — redis (Redis workflow checkpoints)

Install everything with pip install -e ".[all]".

---

Configuration

Shared runtime defaults live in maki/config.py. All values are overridable via environment variables or a .env file (python-dotenv is supported).

---

Quick Start

Basic request

from maki import MakiLLama

llm = MakiLLama(model="gemma4:26b")
response = llm.chat("Explain recursion in one sentence.")
print(response.content)

Chat, streaming, and async

import asyncio
from maki import MakiLLama
from maki.objects import GenerationConfig

config = GenerationConfig(temperature=0.7, max_tokens=512)
llm = MakiLLama(model="gemma4:26b", config=config)

reply = llm.chat("Give me three project naming ideas.")
print(reply.content)

for chunk in llm.stream("Write a short haiku about testing"):
    print(chunk, end="", flush=True)

async def main():
    response = await llm.async_chat("Summarize the benefits of type hints.")
    print(response.content)

asyncio.run(main())

Stateful session

from maki import MakiLLama

llm = MakiLLama(model="gemma4:26b")
session = llm.session(system="You are a concise engineering assistant.")

session.say("We are building a release checklist.")
response = session.say("What should we verify before publishing a Python package?")
print(response.content)

Hosted backends

from maki import MakiOpenAI, MakiAnthropic

# OpenAI
llm = MakiOpenAI(model="gpt-4o")
response = llm.chat("What is the capital of France?")

# Anthropic
llm = MakiAnthropic(model="claude-sonnet-4-5")
response = llm.chat("Summarize this code in one sentence.")

---

Agents

Basic agent

from maki import MakiLLama
from maki.agents import Agent

llm = MakiLLama(model="gemma4:26b")
agent = Agent(
    name="Reviewer",
    maki_instance=llm,
    role="code reviewer",
    instructions="Focus on bugs, regressions, and missing validation.",
    stateful=True,
)

result = agent.execute_task("Review this design: a plugin system with file access.")
print(result)

Memory and reasoning helpers

agent.remember("repo", "maki")
print(agent.recall("repo"))

steps = agent.think_step_by_step("How should we structure plugin validation?")
subtasks = agent.decompose_task("Prepare this repository for a public release")

Streaming task execution

for chunk in agent.stream_task("Draft a short changelog entry."):
    print(chunk, end="", flush=True)

Long-running tasks with use_streaming

By default, execute_task sends one blocking HTTP request. For tasks that exceed the configured timeout (default 120 s), set use_streaming=True — the timeout then applies per chunk rather than to the whole response.

agent = Agent(
    name="Ranker",
    maki_instance=llm,
    role="content ranker",
    use_streaming=True,
)

result = agent.execute_task("Rank these 50 articles by relevance: ...")
print(result)

---

Agent Manager and Workflows

AgentManager coordinates multiple agents and can run collaborative or dependency-aware workflows.

from maki import MakiLLama
from maki.agents import AgentManager, WorkflowTask

llm = MakiLLama(model="gemma4:26b")
manager = AgentManager(llm)

manager.add_agent("Researcher", role="researcher")
manager.add_agent("Writer", role="writer")

workflow = [
    WorkflowTask(
        name="research",
        agent="Researcher",
        task="Find the main public-release risks for this repository.",
    ),
    WorkflowTask(
        name="summary",
        agent="Writer",
        task="Summarize the research into a release checklist.",
        dependencies=["research"],
    ),
]

results = manager.run_workflow(workflow)
print(results["summary"]["result"])

Supported manager patterns:

---

Distributed Layer

Serve any agent over HTTP with maki serve:

maki serve --config agent.yaml --host 127.0.0.1 --port 8100

# agent.yaml
name: MyAgent
model: gemma4:27b
role: assistant
plugins:
  - web_search
  - file_reader

Connect to a remote agent from another process:

from maki.distributed.proxy import AgentProxy

agent = AgentProxy(name="MyAgent", base_url="http://127.0.0.1:8100")
result = agent.execute_task("Summarize the latest AI news.")

DistributedAgentManager lets you mix local and remote agents in the same workflow.

---

Plugins

Built-in plugins are registered in maki/plugins/__init__.py:

Loading a plugin in an agent

from maki import MakiLLama
from maki.agents import Agent

llm = MakiLLama(model="gemma4:26b")
agent = Agent(name="ToolUser", maki_instance=llm, role="assistant")
agent.load_plugin("file_reader")

result = agent.execute_task(
    "Read the first lines of README.md and summarize them.",
    use_plugins=True,
)
print(result)

When use_plugins=True (or the backend supports native tool-calling), available plugin methods are advertised to the model and executed automatically. Destructive methods (file writes, trades, FTP deletes) require Agent(allow_dangerous_tools=True).

---

HuggingFace Backend

HFBackend runs models directly via HuggingFace Transformers — no Ollama required.

from maki import HFBackend

llm = HFBackend(model="mistralai/Mistral-7B-Instruct-v0.2", device="cuda")
response = llm.chat("Explain attention mechanisms.")
print(response.content)

Supports quantization and device selection (cpu, cuda, mps).

---

Public API

Top-level imports exposed by maki:

• MakiLLama, MakiOpenAI, MakiAnthropic, HFBackend
• LLMBackend, BackendType
• Agent, AgentManager
• GenerationConfig, LLMResponse, Message, ToolCall
• ConversationMemory, RateLimiter
• Connector, Utils
• config

All exports are lazy-loaded on first access.

---

Desktop App

The repository includes a PySide6/QML desktop shell (requires maki-framework[gui]):

maki-gui

---

Testing

pytest

888 tests covering backends, agents, workflows, plugins, connectors, distributed layer, and security-related behaviour. Tests marked @pytest.mark.network (requiring live external services) are excluded by default; run them explicitly with pytest -m network.

---

Contributing

Contributions are welcome: bug fixes, documentation improvements, new plugins, and feature suggestions all help move the project forward. Open an issue or submit a pull request on GitHub.

If you are interested in this line of research, consider joining Bowl of Data, an open-source AI research community.

---

License

MIT