[!IMPORTANT] shaide is in early access. The platform runs production workloads today, but this repository is young: interfaces, stack configuration and module layout may still change between releases. Pin a release rather than tracking main, and please open an issue if something breaks.

---

What is shaide?

shaide is a self-hosted AI platform that serves AI models at scale on your own Kubernetes clusters. The goal is to run many models side by side, each with multiple replicas, and route traffic across them.

Standing up enterprise AI infrastructure today means assembling a long list of moving parts - an inference engine, a serving orchestrator, a gateway, a model registry, storage, observability, and that is only the beginning. Each one has to be chosen, configured and glued to the next, component by component, then again for every environment. shaide ships that whole stack as one installable platform.

The entire platform is managed as infrastructure as code. Every layer - the internal registry, the gateway, model serving and the application layer - is defined as a Pulumi project in this repository, so your AI infrastructure is versioned and reproducible.

And it stays inside your perimeter. shaide is built for organisations whose data cannot leave their infrastructure: regulated industries, defence, public sector, or anyone who simply will not send prompts to a third-party API. There is nothing phoning home and no dependency on a vendor's cloud - including fully air-gapped installations with no internet access at all.

Why shaide

• Sovereign by design. Everything runs in your infrastructure. An internal OCI registry mirrors every container image and model weight, so a cluster can operate with no egress whatsoever.

• Installed by one command. Point the interactive terminal installer at a prepared cluster and it installs the entire platform through guided prompts. No Helm chart archaeology, no twelve READMEs to follow in order.

• Distributed multi-model serving. vLLM as the inference engine, llm-d for multi-instance orchestration. Serve generative and embedding models concurrently, each scaled independently across GPU nodes.

• Built for agent fleets. Multi-model routing, KV-cache-aware scheduling and inference-pool load balancing keep many concurrent agents served from one platform.

• Runs anywhere Kubernetes runs. AWS EKS, GCP GKE, Azure AKS and on-prem RKE2 - the same platform and the same installer across all of them.

• OpenAI-compatible API. Point any OpenAI-compatible SDK, agent framework or tool at your endpoint and change nothing but the base URL.

• Infrastructure as Code. Every layer is a Pulumi Go program, so deployments are reviewable, diffable and reproducible.

Quickstart

1. Prepare a cluster

shaide installs onto an existing Kubernetes cluster. Check it against the prerequisites:

→ Prerequisites

2. Choose your models

The installer ships with everything it deploys. The one input you supply is a model manifest listing the models to publish into the internal registry.

[!IMPORTANT] Supplying models.yaml by hand is a temporary step. Model selection moves into the installer in the next release, and this file will no longer be required.

mkdir -p /tmp/manifests
cat > /tmp/manifests/models.yaml <<'YAML'
models:
  - id: "openai/gpt-oss-20b"
    revision: "6cee5e81ee83917806bbde320786a8fb61efebee"
    harbor_project: "ai-models"
    harbor_name: "gpt-oss-20b"
    harbor_tag: "1.0.0"
YAML

3. Run the installer

Create a directory to hold the installer data and run the installer container:

STORAGE_PATH=<storage-path>
PULUMI_CONFIG_PASSPHRASE=<choose-a-passphrase>
HF_TOKEN=<your-huggingface-token>

mkdir -p "${STORAGE_PATH}"

docker run --rm -it \
  --network host \
  -e PULUMI_CONFIG_PASSPHRASE \
  -e HF_TOKEN \
  -e MODEL_MANIFEST_PATH=/manifests/models.yaml \
  -v "$HOME/.kube/config:/.kube/config:ro" \
  -v /tmp/manifests/models.yaml:/manifests/models.yaml:ro \
  --mount "type=bind,src=${STORAGE_PATH},dst=/var/shaide-installer" \
  ghcr.io/axem-solutions/shaide/installer:oss

→ Installer guide

4. Verify

Once the installer completes, check that the platform is serving:

curl https://<your-shaide-endpoint>/v1/models

Using the API

shaide exposes an OpenAI-compatible API, so existing clients work unchanged:

from openai import OpenAI

client = OpenAI(
    base_url="https://<your-shaide-endpoint>/v1",
    api_key="<your-api-key>",
)

response = client.chat.completions.create(
    model="<model-name>",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(response.choices[0].message.content)

Architecture

shaide is built in layers, each deployed as an independent Pulumi program:

Deployment proceeds in a fixed order:

1. Internal OCI registry       harbor
2. Gateway + Istio             infra/gateway-provider
3. Model serving               app_serving
4. Application layer           app_shaide
5. Optional add-ons            app_mcp, monitoring

The shaide components

shaide is developed across three open-source repositories:

Repository layout

├── app_serving/      Per-model LLM serving stacks (vLLM + llm-d)
├── app_shaide/       Application layer: shaide server and control panel
├── app_mcp/          MCP server datasources deployed into the shared gateway
├── harbor/           Internal OCI registry for images and model weights
├── infra/            In-cluster platform services: the shared gateway
├── installer/        Containerized interactive installer
├── monitoring/       Observability stack
└── pkg/              Shared Go module with the Pulumi deployment logic

Full technical documentation lives at docs.

Contributing

Contributions are welcome. See CONTRIBUTING.md for development setup, coding standards and the review process.

Security

Please do not report security vulnerabilities through public issues. See our security policy for how to disclose them responsibly.

License

Licensed under the Apache License 2.0. See LICENSE for details.

shaide Copyright 2026 axem solutions Kft.

This product includes software developed by third parties. See their respective licensing for additional copyright and licensing information.