A distributed, Redis-compatible key-value database.
AP by design · coordination-free · disk-native · Kubernetes-native

Website · Quickstart · Redis API · Design docs · Docker / k8s

---

A distributed key-value database server with a Redis-compatible API, written in Rust. AP by design (eventually consistent, coordination-free), disk-native via the ondaDB LSM engine — no in-memory dataset.

• Redis protocol: RESP2 + RESP3, strings / hashes / sets / sorted sets / lists / streams / pub-sub / JSON documents (coverage matrix)
• Convergent replication: hybrid logical clocks + per-element ORSWOT merges; concurrent SADDs on different nodes both survive, deletes never resurrect (data model)
• JSON documents (JSON.*): the full RedisJSON v2 surface stored as a per-path CRDT — concurrent editors on different nodes merge structurally instead of clobbering whole documents (JSON)
• Document comparison (DIFF.*, experimental): immutable snapshots, independent move/edit suggestions, replicated review decisions, three-way merges, and identity-preserving import (command reference, design)
• Protobuf schema store (PROTO.*): upload .proto schemas (compiled server-side), bind them to key prefixes, store validated messages with server-side field access, JSON projection, and field-level CRDT merge — two nodes writing different fields of the same message both win (protobuf, field merge)
• Dynamic replication: any node serves any key; a node that reads a remote key caches it and subscribes to its updates (replication)
• Bounded staleness: sequence-cursor resume + Merkle anti-entropy repair divergence within seconds (anti-entropy)
• Kubernetes-native: gossip membership (chitchat) with DNS-seeded discovery; nodes come and go (membership, k8s)
• Lua scripting: EVAL/EVALSHA with Redis-grade atomicity for scripts whose keys co-locate (hash tags {...}); script effects replicate, never the script (scripting)
• Distributed budgets (BG.*): escrow-based reserve/commit tokens with a hard never-overspend invariant — fail-closed under partition, crash, and split-brain (budgets)
• Kubernetes operator: MarekvsCluster CRD with safe one-node-at-a-time scale-down and ops/s-based autoscaling (operator)
• OS-less images: static binary in a FROM scratch container

📖 Documentation lives at marekvs.com (source in docs/, built by crates/docsgen and published to GitHub Pages). Lower-level design internals live in design/.

Quickstart

Everything runs through just:

just build          # debug build
just test           # unit + property tests (merge laws!)
just test-smoke     # end-to-end single node via redis-cli
just run            # single local node on :6379

just run-cluster    # local 3-node cluster on :6379/:6380/:6381

just docker-build   # FROM-scratch image
                    # plain cargo builds fall back to the ondadb git dep
                    # (github.com/yannick/ondadb)
just docker-test    # 3-node compose cluster + convergence tests
just apple-build    # same image via Apple's `container` CLI
just apple-test     # 3-node apple-container cluster + convergence tests

just ci             # fmt-check + clippy + tests

just k8s-apply      # example Kubernetes deployment (see k8s/README.md
just k8s-status     # for safe dynamic scale-up/down without data loss)
just operator-apply # CRD-based operator with autoscaling (design/12)

just bench          # benchmark vs KeyDB (both in docker) → bench/report.md
just bench-report   # re-render the report from accumulated results

Try it:

just run &
redis-cli set greeting hello
redis-cli get greeting
redis-cli sadd tags rust distributed redis
redis-cli smembers tags

Typed values that merge instead of clobber — two clients on two different nodes edit the same protobuf message concurrently, and both edits survive:

just run-cluster &            # 3 nodes on :6379 / :6380 / :6381

redis-cli -p 6379 proto.schema set acme/user.proto SOURCE '
  syntax = "proto3"; package acme;
  message User { string name = 1; int32 age = 2; repeated string tags = 3; }'
redis-cli -p 6379 proto.bind "user:" acme.User
redis-cli -p 6379 proto.setjson user:1 '{"name":"seed","age":1}'

# client A on node 1 and client B on node 2, at the same time:
redis-cli -p 6380 proto.setfield user:1 name ada     # ← client A
redis-cli -p 6381 proto.setfield user:1 age 42       # ← client B

redis-cli -p 6379 proto.getjson user:1
# {"name":"ada","age":42}      ← BOTH edits, on every node

Each field is its own CRDT record: same-field races are last-writer-wins, concurrent appends to a repeated field all survive in contiguous runs, and JSON documents (JSON.SET doc $.a … vs JSON.SET doc $.b …) merge the same way. Details: docs/protobuf.md, docs/json.md.

Run from published images (ghcr.io)

No checkout required — pull the FROM scratch image straight from the GitHub Container Registry (swap in your own owner/org if you run a fork). Full walkthrough (env vars, ports, verification): docs/quickstart.md.

Single node — Docker:

docker run -d --name marekvs -p 6379:6379 -p 9121:9121 \
  -e MAREKVS_NODE_ID=0 -e MAREKVS_REPLICAS_N=1 -e MAREKVS_DATA_DIR=/data \
  -v marekvs-data:/data ghcr.io/yannick/marekvs:latest
redis-cli -p 6379 ping     # PONG

Three nodes — Docker Compose: point deploy/compose.yaml at ghcr.io/yannick/marekvs:latest and docker compose up -d — three nodes on a fixed subnet seed off each other with MAREKVS_REPLICAS_N=2 (client ports 16379/16380/16381). From a checkout, just docker-up does this locally.

Single node — Apple container: Apple containers get their own IP instead of host ports, so read it back and connect directly:

container system start
container run -d --name marekvs -e MAREKVS_NODE_ID=0 -e MAREKVS_REPLICAS_N=1 \
  -e MAREKVS_ADVERTISE_IP=auto -e MAREKVS_DATA_DIR=/data ghcr.io/yannick/marekvs:latest
IP=$(container inspect marekvs | python3 -c 'import json,sys;print(json.load(sys.stdin)[0]["status"]["networks"][0]["ipv4Address"].split("/")[0])')
redis-cli -h "$IP" ping    # PONG

Three nodes — Apple container: with MAREKVS_ADVERTISE_IP=auto, only the seed must be known — start node 0, then seed nodes 1 and 2 off its IP (exactly what just apple-up automates via tests/apple_cluster.sh).

Configuration (environment)

Storage engine (ondaDB)

marekvs is disk-native — the memtable, the block cache and the OS page cache are its only memory tiers — so these are pinned explicitly rather than inherited, and they move with ondaDB releases. Defaults below are ondaDB 0.8.0's.

Reader memory is the one to watch: before ondaDB 0.7 it was unbounded and grew with total stored bytes rather than working set. marekvs_db_reader_resident_bytes against marekvs_db_reader_budget_bytes shows the headroom, and marekvs_db_bloom_skips_total / marekvs_db_sst_probes_total show whether bloom filters are actually filtering (the ratio should be high, not zero).

Compaction backlog is the second. ondaDB 0.8.0 paces writers against marekvs_db_compaction_debt_bytes and, at its own hard ceiling (8 GiB), blocks the commit outright — on a shard thread, which head-of-line blocks every key on that shard with no timeout anywhere above it. marekvs therefore refuses client writes with a MISCONF one step earlier, at MAREKVS_COMPACTION_DEBT_HIGH_BYTES, and says so on marekvs_db_compaction_write_stopped. Replication, anti-entropy and bootstrap keep applying throughout — refusing a merge is divergence, not backpressure.

MAREKVS_FINISH_COMPACTIONS_ON_CLOSE is a shutdown/restart trade, and the numbers are lopsided. Measured on 10M x 400 B records with ~2 GB of compaction debt outstanding: off (the default) closes in 0.58 s; on was still draining after 30 s and hit MAREKVS_CLOSE_TIMEOUT_SECS. Off, the restarted pod serves reads over a deeper L0, and because L0 files overlap every one of them is probed per point read — that is what you buy for the prompt shutdown. Turn it on only where a fully merged restart matters more, and only after checking the close fits terminationGracePeriodSeconds: 60 minus the ~9 s the SIGTERM handler already spends gossiping and draining. The close is bounded precisely so overrunning that budget costs a WAL replay instead of a SIGKILL landing mid-close.

Workspace layout

Consistency notes (read before production use)

marekvs is AP: reads on one connection see that connection's writes and never go backward, but two clients on two nodes can briefly observe different values. Lists are per-element LWW registers: two nodes pushing to the same position can drop one push — a bounded, per-collision loss, not a whole-list clobber. INCR/DECR/INCRBY/DECRBY are PN counters: concurrent increments across nodes are never lost (an explicit SET resets). Lua scripts are atomic per node/shard when all KEYS share a partition — they are NOT a distributed lock primitive; see 11-lua-scripting.md. See 00-overview.md.