KiwiQ AI Platform

Production-grade multi-agent orchestration platform — JSON-defined agents, multi-tier memory, and built-in observability.

Battle-tested on 200+ enterprise AI agents. Define complex multi-step AI workflows as Python/JSON graph schemas with 24+ reusable node types, multi-provider LLM support, human-in-the-loop interactions, web scraping, RAG pipelines, and versioned customer data management. Ships with 27+ ready-to-use workflow definitions.

This platform powered KiwiQ AI (Marketing AI Agents) in production and is now fully open-sourced.

---

Features

• SDK-first workflow engine — Define workflows as Python/JSON graph schemas (not a visual builder), compiled to LangGraph and executed via Prefect
• Multi-provider LLM support — OpenAI, Anthropic, Google Gemini, Perplexity, Fireworks, AWS Bedrock
• Human-in-the-Loop (HITL) — Pause workflows for human review, input, or approval with real-time WebSocket streaming
• 24+ reusable node types — LLM, routing, conditional branching, data transforms, scraping, code execution, sub-workflows, and more
• 27+ production workflow definitions — Content creation, diagnostics, lead scoring, deep research, playbook generation, and more included out of the box
• Multi-tier memory & state — PostgreSQL for relational state, MongoDB for versioned documents, Weaviate for vector search, Redis for caching
• RAG pipelines — Document ingestion, vector search (Weaviate), and retrieval-augmented generation
• Customer data management — Versioned document storage in MongoDB with CRUD workflow nodes
• Event-driven architecture — RabbitMQ-based event bus for async processing between services
• Observability — Real-time progress via RabbitMQ events, WebSocket streaming, Prefect dashboard, structured logging
• Web scraping — LinkedIn profiles/companies, web crawling, AI-powered search engines
• Sandboxed code execution — Run user-defined Python safely within workflows
• Billing & auth — Stripe integration, JWT authentication, role-based access control

Architecture Overview

┌─────────────────────────────────────────────────────────────┐
│                      FastAPI (kiwi_app)                      │
│  Auth │ Billing │ Workflow API │ RAG │ Data Jobs │ WebSocket │
└──────────────────────────┬──────────────────────────────────┘
                           │
              ┌────────────┼────────────┐
              ▼            ▼            ▼
        ┌──────────┐ ┌──────────┐ ┌──────────┐
        │ RabbitMQ │ │ Prefect  │ │  Redis   │
        │ (events) │ │ (orch)   │ │ (cache)  │
        └────┬─────┘ └────┬─────┘ └──────────┘
             │             │
             ▼             ▼
        ┌──────────────────────────┐
        │   Workflow Service       │
        │  ┌────────┐ ┌────────┐  │
        │  │LangGraph│ │ Nodes  │  │
        │  │ Engine  │ │Registry│  │
        │  └────────┘ └────────┘  │
        └──────────┬───────────────┘
                   │
     ┌─────────┬───┼───┬──────────┐
     ▼         ▼   ▼   ▼          ▼
┌────────┐┌───────┐┌────────┐┌──────────┐
│Postgres││MongoDB││Weaviate ││LLM APIs  │
│(state) ││(docs) ││(vector) ││(OpenAI…) │
└────────┘└───────┘└────────┘└──────────┘

Quick Start

Prerequisites

• Python 3.12
• Poetry
• Docker & Docker Compose

1. Clone and install

git clone https://github.com/kiwiq-ai/kiwiq-oss.git
cd kiwiq-oss
poetry install

2. Configure environment

cp .env.sample .env

Edit .env and fill in required values:

See .env.sample for the full list of configuration options.

3. Start services

# Development (all services including databases)
docker compose -f docker-compose-dev.yml up --build

4. Access the platform

• API docs: http://localhost:8000/docs
• Prefect dashboard: http://localhost:4201
• RabbitMQ management: http://localhost:15672
• RedisInsight: http://localhost:8001

Using with Claude Code

This repository ships with a comprehensive CLAUDE.md that provides Claude Code with full context about the project — architecture, commands, testing patterns, workflow development, and troubleshooting. To get started:

1. Install Claude Code
2. Run claude from the repo root

Claude Code will automatically pick up CLAUDE.md and can help with local setup, running tests, modifying services, building workflows, debugging, and navigating the codebase.

Docker Environments

Development (docker-compose-dev.yml)

Full local environment with all services containerized:

All data is persisted via named Docker volumes. Code changes are hot-reloaded via volume mounts.

Production (docker-compose.prod.yml)

Production-hardened environment with:

• Nginx + Certbot for SSL/TLS termination
• External managed databases (PostgreSQL, MongoDB) — not containerized
• Resource limits on all containers (CPU and memory caps)
• JSON-file logging with rotation
• Health checks on critical services
• Tuned for 4x concurrent workflow execution

Running Without Docker

To run services directly on your host:

# Set Python path
export PYTHONPATH=$(pwd):$(pwd)/services

# Start FastAPI server
poetry run uvicorn kiwi_app.main:app --host 0.0.0.0 --port 8000 --reload

# Start Prefect worker (separate terminal)
poetry run python services/workflow_service/services/worker.py

You'll need PostgreSQL, MongoDB, Redis, RabbitMQ, and Weaviate running separately (locally or hosted).

Repository Structure

kiwiq-oss/
├── libs/src/                        # Shared libraries
│   ├── db/                          #   PostgreSQL (SQLModel, Alembic migrations)
│   ├── mongo_client/                #   MongoDB client
│   ├── redis_client/                #   Redis client
│   ├── rabbitmq_client/             #   RabbitMQ event streaming
│   ├── weaviate_client/             #   Weaviate vector DB client
│   ├── global_config/               #   Global settings
│   └── global_utils/                #   Shared utilities
│
├── services/
│   ├── kiwi_app/                    # Core FastAPI application
│   │   ├── auth/                    #   Authentication & authorization
│   │   ├── billing/                 #   Stripe billing & credits
│   │   ├── workflow_app/            #   Workflow API, WebSockets, events
│   │   ├── data_jobs/               #   Data ingestion & RAG pipelines
│   │   └── rag_service/             #   RAG query endpoints
│   │
│   ├── workflow_service/            # Workflow engine
│   │   ├── registry/nodes/          #   24+ node implementations
│   │   ├── graph/                   #   GraphSchema → LangGraph builder
│   │   └── services/worker.py       #   Prefect worker entrypoint
│   │
│   ├── linkedin_integration/        # LinkedIn OAuth & API
│   └── scraper_service/             # Web scraping endpoints
│
├── standalone_test_client/          # Workflow SDK, 27+ workflow definitions
│   └── kiwi_client/
│       └── workflows/active/        #   Production workflow examples
│
├── tests/                           # 75+ unit & integration tests
├── docs/                            # 40+ pages of technical documentation
├── docker/                          # Dockerfiles & setup scripts
└── pyproject.toml                   # Poetry dependencies

Node Types

Workflows are composed from reusable nodes defined in Python/JSON. Key categories:

Included Workflows

The standalone_test_client/ ships with 27+ production workflow definitions — complete with graph schemas, LLM prompts, HITL test inputs, and runner scripts. These serve as both reference implementations and a starting point for building your own.

Content Studio (11 workflows)

Workflows for content creation across blog and LinkedIn:

Content Diagnostics (10 workflows)

AI visibility analysis, competitive intelligence, and content audits:

Sales & Investor (2 workflows)

Playbooks (2 workflows)

Labs & Utilities (3 workflows)

Each workflow folder includes:

• wf_*_json.py — Graph schema definition (Python/JSON)
• wf_llm_inputs.py — LLM prompts, output schemas, templates
• wf_testing/ — HITL inputs, runner script, test artifacts

See the workflow onboarding guide to get started.

Testing

# Run all tests
PYTHONPATH=$(pwd):$(pwd)/services poetry run pytest

# Unit tests only
PYTHONPATH=$(pwd):$(pwd)/services poetry run pytest -m unit

# Integration tests only
PYTHONPATH=$(pwd):$(pwd)/services poetry run pytest -m integration

# Specific test file
PYTHONPATH=$(pwd):$(pwd)/services poetry run pytest tests/unit/services/workflow_service/test_example.py -v

# With coverage
PYTHONPATH=$(pwd):$(pwd)/services poetry run pytest --cov=services --cov=libs --cov-report=term-missing

See tests/ for the full test suite (75+ test files).

Documentation

Database Migrations

# Run pending migrations
PYTHONPATH=$(pwd):$(pwd)/services poetry run alembic upgrade head

# Create a new migration
PYTHONPATH=$(pwd):$(pwd)/services poetry run alembic revision --autogenerate -m "description"

LLM Providers

Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch (git checkout -b feat/your-feature)
3. Make your changes with tests
4. Run the test suite (poetry run pytest)
5. Submit a pull request

Acknowledgements

The KiwiQ AI platform was designed, built from scratch and open-sourced by Raunak Bhandari, CTO and co-founder of KiwiQ AI, including the core platform, workflow engine, node system, infrastructure, shared libraries, many of the production workflows, and auxiliary systems such as scraping services and browser pools.

Gaurav Kumar and Anish Bharadwaj built and tested several of the marketing and deep market research focused workflows, helped with platform testing and productizing for our customers.

Disclaimer

This project is provided for educational and research purposes only. Scraping websites may violate their Terms of Service. The authors and contributors are not responsible for any misuse of this software. Users are solely responsible for ensuring their use complies with all applicable laws, regulations, and platform policies. Use at your own risk.

License

Apache 2.0 — see the LICENSE file for details.