Tellstone is an ultra‑high‑performance, cloud‑native in‑memory key/value store written entirely in Go. It speaks two protocols over TCP — a compact custom binary protocol and a Redis‑compatible (RESP2) protocol — on top of a shared-nothing (SN) storage engine with optional TTL eviction, at‑rest encryption, and write-ahead log persistence.

       +---------------------------------------------+
       |             Your K8s Cluster                |
       |                                             |
       |  [App Pod] --( binary :9988 / RESP :6379 )->|
       |                                             |
       |     +---------------------------------+     |
       |     |        TELLSTONE CORE           |     |
       |     |  (N Shards, each a goroutine +  |     |
       |     |   sync.RWMutex + map[string]Item)|    |
       |     |  FNV-1a hash → O(1) dispatch    |     |
       |     +---------------------------------+     |
       +---------------------------------------------+

Why Tellstone?

Many managed databases (PostgreSQL, MySQL, …) become bottlenecks under high‑frequency workloads. Tellstone offers a lean, modern, memory‑efficient buffer that:

• Zero‑Copy Binary Protocol – Direct binary messages avoid text parsing / Protobuf overhead.
• Redis‑Compatible – An optional RESP2 listener lets you drive Tellstone with redis-cli, redis-benchmark, memtier_benchmark, and existing Redis client libraries (GET/SET/PING/DEL).
• Shared-Nothing Engine – N independent shards, each containing one map[string]Item plus a sync.RWMutex. Keys are pinned to a shard via FNV-1a hashing so the lock is almost never contended. No cross-shard coordination, no channel round-trips, no per-request allocations.
• Configurable TTL Eviction – An active timing‑wheel (chronometer) evicts expired keys in O(1); lazy eviction on read backs it up.
• Optional At‑Rest Encryption – ChaCha20‑Poly1305, off by default.
• Write-Ahead Log Persistence – Per-shard append-only WAL for crash recovery. SET and DEL operations are persisted (deletes as tombstones) and replayed on restart. Zero-allocation on the hot path (Write = 0 allocs/op). Disabled by default.
• Metrics, Tracing & Audit Logging – Built‑in Prometheus exporter, optional OpenTelemetry tracing, and an opt-in structured audit trail for security events.

Core Architecture

For a detailed description of the package structure, request flow, and design decisions, see ARCHITECTURE.md.

---

Getting Started

Install

Debian / Ubuntu (APT)

curl -fsSL https://saxy.github.io/tellstone-apt/saxy-keyring.gpg \
  | sudo gpg --dearmor -o /usr/share/keyrings/saxy-keyring.gpg

echo "deb [signed-by=/usr/share/keyrings/saxy-keyring.gpg] \
  https://saxy.github.io/tellstone-apt stable main" \
  | sudo tee /etc/apt/sources.list.d/saxy-tellstone.list > /dev/null

sudo apt update && sudo apt install tellstone

macOS (Homebrew)

brew tap Saxy/tellstone-tap
brew install --cask Saxy/tellstone-tap/tellstone

Binary Downloads

Pre-built binaries are available on the Releases page for Linux, macOS, and Windows (amd64 and arm64).

Build from Source

Requires Go 1.26+ and optionally task (go‑task):

task build          # → ./bin/tellstone   (or: go build -o bin/tellstone ./cmd/tellstone)

Run

task run            # binary protocol on 127.0.0.1:9988
task run:resp       # binary on :9988  +  Redis-compatible RESP on :6379

Or run the binary directly with flags / environment variables:

./bin/tellstone --addr 127.0.0.1:9988 --enable-resp --resp-addr 127.0.0.1:6379
TSD_ADDR=127.0.0.1:9988 TSD_ENABLE_RESP=true ./bin/tellstone

If a previous run got killed uncleanly and left a server stuck on a port (address already in use), find and stop it with:

task kill                          # checks :19988, :6379, :6060 and any bin/tellstone process
task kill PORTS="9988" NAME=myapp  # override the ports/name to search for

Works on Linux and macOS (lsof/pgrep/pkill, no OS-specific tooling).

Configuration

Every option is available as a flag and an environment variable.

Runtime tuning (environment only): TSD_GC_PERCENT (default -1, GC off for a zero‑GC hot path), TSD_MEM_LIMIT_BYTES (soft heap ceiling), TSD_ENABLE_PROFILING (serves pprof on 127.0.0.1:6060).

At-rest encryption takes its 32-byte key from exactly one source. --encryption-key carries it base64-encoded, because process arguments and environment variables are NUL-terminated and roughly 1 in 8 random 32-byte keys contains a 0x00 byte that cannot survive them:

tellstone --enable-encryption --encryption-key "$(openssl rand -base64 32)"

--encryption-key-file carries the same key as raw, unencoded bytes, which suits a mounted Kubernetes Secret or a vault-agent-rendered file. Every byte of the file is significant, so it must be exactly 32 bytes with no trailing newline:

# umask 077 in a subshell so the file is created 0600; a later chmod would leave
# the key world-readable in between.
(umask 077; head -c 32 /dev/urandom > /etc/tellstone/key)
tellstone --enable-encryption --encryption-key-file /etc/tellstone/key

The key is read once at startup; rotating it requires a restart. Enabling encryption without either source is refused at startup rather than silently falling back to plaintext.

Deprecated: --encryption-key previously used the value as raw text, so a literal 32-character key such as 0123456789abcdef0123456789abcdef was accepted. That form still works and logs a warning at startup, but it is insecure and will be removed in the next major release. Re-encode an existing key with base64 < <keyfile>, or move it to --encryption-key-file. The two forms are never ambiguous: a base64 key is 44 characters (43 unpadded), while a 32-character value can only decode to 24 bytes.

When TLS is enabled, Tellstone watches the parent directories of the certificate, key, and optional client CA. Valid replacements are applied after a 500 ms debounce; existing TLS connections continue uninterrupted. Directory watching supports atomic file renames and Kubernetes projected Secret updates.

By default, both listeners require TLS from the first byte. Setting --resp-starttls keeps only the RESP listener plaintext until a client sends STARTTLS; Tellstone replies +OK in plaintext and then requires an immediate TLS 1.3 handshake on the same socket. STARTTLS is allowed before AUTH so credentials need not cross plaintext. The command must not be pipelined with any other plaintext bytes. The binary listener always retains implicit TLS, and each RESP upgrade loads the latest rotated certificate configuration. Once a RESP connection is handed to the TLS state machine — on accept for implicit TLS, or after a STARTTLS acceptance — it must complete the handshake within 10 seconds; the listener closes it at the deadline even if the client sends nothing further, so stalled connections cannot pile up.

---

Using Tellstone

Redis‑compatible (RESP) — easiest

Start with task run:resp, then use any Redis client:

redis-cli -p 6379 PING            # PONG
redis-cli -p 6379 SET foo bar     # OK
redis-cli -p 6379 GET foo         # "bar"
redis-cli -p 6379 SET k v EX 60   # OK (60s TTL)
redis-cli -p 6379 DEL foo         # (integer) 1

Supported commands today: PING, GET, SET (with EX/PX), DEL, AUTH, COMMAND, ROLE (CREATE/SETUSER/DELUSER/DELETE/LIST/GETUSER), ACL (SETUSER/DELUSER/LIST/LOG). Unknown commands return a -ERR reply without dropping the connection. STARTTLS is additionally available when --resp-starttls is enabled.

Authentication & RBAC

Start with --require-pass for a single shared password, or --rbac-config for per-user authentication with role-based access control (supersedes --require-pass):

# policy.yaml — loaded at startup and hot-reloaded on SIGHUP.
# Passwords are bcrypt hashes, e.g. of "adminsecret" / "alicepw"; a password
# is required unless the user is explicitly marked nopass.
roles:
  - name: admin
    rules: ["+@all", "~*"]
  - name: readonly
    rules: ["+get", "~*"]
users:
  - name: admin
    role: admin
    password: "$2a$10$pcaKkTfRy.KSdNUgKszYYedE7L32P9fSEG3x1phq0EbjeYkn5WpEi"
  - name: alice
    role: readonly
    password: "$2a$10$sslrTYVwaIaA7O1lhokY2OgnojP5bB8YJ/o2MXaFP1v49lG8fqJYK"
default_role: readonly    # least privilege: fallback for users without an explicit role

Generate a password hash with htpasswd -nbBC 10 "" PASSWORD | tr -d ':\n' or mkpasswd -m bcrypt PASSWORD (bcrypt, cost 10, $2a$10$...). Do not use openssl passwd — it emits SHA-512-crypt, not bcrypt. ROLE SETUSER accepts a raw >password and bcrypt-hashes it server-side, so runtime-created users need no tooling.

./bin/tellstone --rbac-config policy.yaml --enable-resp
redis-cli AUTH admin adminsecret                 # +OK
redis-cli ROLE CREATE operator +get '~users:*'   # +OK (runtime roles)
redis-cli ROLE SETUSER bob operator '>bobpw'     # +OK
redis-cli ROLE GETUSER bob                       # bob / operator / 1

Roles are user-defined; a role's rules are Redis-style tokens: +cmd / -cmd grant or revoke one command, +@cat / -@cat a whole category, and ~prefix whitelists a key namespace (an empty list or ~* allows every key). - rules override + rules. The built-in categories:

A ready-to-run policy file ships with the role example at cmd/example/role/policy.yaml.

Unauthenticated data commands return -NOAUTH; commands a user's role does not grant return -NOPERM. The native binary client offers the same via client.AuthUser and RoleCreate / RoleSetUser (see cmd/example/role). The ACL command family (ACL SETUSER / ACL DELUSER / ACL LIST / ACL LOG) manages the same policy store through a Redis-flavored alias, driven end-to-end in cmd/example/acl.

Federated auth (OAuth / OIDC)

Connections can present an OIDC id_token (a signed JWT) as their credential instead of a password. The token's claims map to a role through the policy file's oauth.rules (claims + match pattern + role, first match wins), and that role is pinned to the connection like any other. Requires --rbac-config — a token can only map to a role through the policy. The server needs no client secret: verification is signature + issuer + audience against the provider's public JWKS.

./bin/tellstone --rbac-config policy.yaml --oauth-provider google --oauth-client-id 1234.apps.googleusercontent.com
redis-cli AUTH <id_token>              # +OK — claims map to a role via oauth.rules

Supported presets: google, stackit; set --oauth-issuer for any other OIDC provider. How the provider pipeline, claim matching, and fail-closed semantics work is documented in internal/oauth/README.md. A runnable client example lives in cmd/example/oauth.

Audit logging

Opt-in structured audit trail via --enable-audit. Each selected audit event is written as one JSON line carrying "level": "AUDIT", so log aggregators can separate it from operational INFO/WARN/ERROR output without custom parsing:

./bin/tellstone --enable-audit --audit-log-path /var/log/tellstone --audit-events all

--audit-log-path is stdout (the default) or a directory. In directory mode the server creates rotating files named <unix-timestamp>_<dir-fingerprint>_tsd.log with 0600 permissions, rotating to a fresh file once a file reaches 50 MiB (history is never truncated or renamed). When --enable-encryption is set, every record is sealed with the crypto engine before it is flushed. The engine is a no-op when audit logging is disabled, so the hot path pays nothing.

--audit-events selects which event types are logged (default auth,acl; all enables every type). connect, disconnect, and command are high-volume and must be opted into explicitly:

{"event":"acl_deny","level":"AUDIT","msg":"command denied by rbac policy","protocol":"binary","remote_addr":"127.0.0.1:51642","command":"SET","key":"config:key","user":"reader","time":"2026-08-05T10:40:32.908569649+02:00"}

Native binary protocol (Go client)

The native protocol is the fastest path. Use the bundled client in internal/network:

import "github.com/Saxy/Tellstone/internal/network"

c, _ := network.Dial("127.0.0.1:9988", 2*time.Second)
defer c.Close()

scratch := make([]byte, 4096)              // reusable response buffer (zero-alloc)
c.Set([]byte("hello"), []byte("world"), 0, scratch)   // ttlMs=0 → no expiry
val, _ := c.Get([]byte("hello"), scratch)             // val == "world"

A runnable example lives in cmd/example/client.

---

Benchmarks

Methodology matters. A naive local benchmark runs the load generator on the same cores as the server's event loops, so the two contend for CPU and the latency tail balloons. All tasks below pin the server and the load generator to disjoint core sets (taskset) so the numbers reflect the server, not scheduler contention. For absolute comparisons, run the load generator on a separate host.

Native binary protocol

task bench:native       # pinned: server cpu0-15, generator cpu16-31

Redis‑compatible (RESP) via memtier

task bench:resp                 # latency run (pipeline=1)
task bench:resp:pipeline        # throughput ceiling (pipeline=16)
task bench:resp:hits            # preload then read-heavy (realistic ~100% hit rate)
task bench:resp:correctness     # preload then read back — proves GET returns what SET stored

Override workload knobs on the command line, e.g.:

task bench:resp PIPELINE=16 DURATION=30 CONNS=50 RATIO=1:4 KEYSPACE=1000000

You can point memtier_benchmark/redis-benchmark at :6379 directly and run the identical command against Redis, Dragonfly, Valkey (or --protocol=memcache_text against memcached) for an apples‑to‑apples comparison.

Reference results

memtier_benchmark — 100k requests, 256-byte values, --ratio=1:10 (1:10 read:write), pipeline 10, uniform random keys, preloaded key set. All four systems tested with identical parameters on the same hardware.

In-memory database benchmarks are highly sensitive to the underlying network infrastructure. To provide an honest and comprehensive view of Tellstone's performance, we categorize our results into two distinct scenarios: Cloud SDN Constraints (simulating a standard production microservice mesh) and Raw Engine Capabilities (eliminating the network stack to test core architectural limits).

Methodology: memtier_benchmark executed from a separate client VM against a remote target VM via a virtual Software-Defined Network. 100k requests, 256-byte values, --ratio=1:10, pipeline=10. At higher concurrency levels (16_64 and 60_128), all engines run directly into the physical Packets Per Second (PPS) ceiling imposed by the cloud provider's virtual switches, capping out at roughly 850K ops/s. Tellstone successfully saturates the cloud network infrastructure, operating at the absolute limit of the hardware while maintaining lower or equivalent tail latencies compared to Redis, Valkey, and Dragonfly.

Small (4 threads, 16 clients)

Medium (16 threads, 64 clients)

Large (60 threads, 128 clients)

Tellstone leads throughput across all three environments (up to 10.5% faster than Redis, 20% faster than Valkey, 20% faster than Dragonfly at small scale). Its p50 latency is consistently the lowest or near-lowest at every concurrency level.

Native binary protocol

Throughput with the native binary protocol (no pipelining, read-heavy):

Numbers are environment-specific; reproduce with task bench:resp and the benchmark/benchmark.sh script.

Bare‑metal benchmarks (localhost, no network overhead)

Same memtier_benchmark parameters (256 B values, --ratio=1:10, pipeline 10, uniform random keys) but executed on a dedicated bare‑metal server (Intel Xeon Platinum 8580, 56 cores, 118 GB RAM, Debian). The load generator and server share the same machine, with taskset pining the server process to the requested core set. This isolates raw engine throughput and latency from any cloud‑SDN constraints.

500K requests per client, 4 clients per memtier thread. Server taskset -c 0-N, memtier runs unpinned.

Small (4 CPUs)

Medium (16 CPUs)

Large (32 CPUs)

On bare metal Tellstone scales nearly linearly with available cores — 11.5x Redis at 32 CPUs — while Redis and Valkey flatline around 1 M ops/s regardless of core count. Dragonfly scales well (6.6x) but Tellstone maintains a ~1.75x lead at every level, with the lowest p50 latency across the board (0.06 – 0.08 ms).

---

Development

task test           # go test ./...
task test:race      # go test -race ./...
task vet            # go vet ./...
task check          # vet + race tests (run before committing)
task fmt            # format

Continuous Integration

Pull requests and pushes to main trigger the CI workflow:

• Build — go build ./...
• Vet — go vet ./...
• Test — go test ./...
• Race tests — go test -race ./...

Benchmarks are not run automatically on every push due to resource constraints. Run them locally with task bench:native or task bench:resp:precise.

Observability

• Metrics: task run:resp with --enable-metrics exposes Prometheus text at http://<metrics-addr>/metrics (default :9100).
• Audit logging: --enable-audit writes structured security events (see Audit logging); run with --audit-events all to capture every event type.

Profiling

Two independent workflows, both built on the stock Go toolchain (pprof / trace). Neither assumes a specific core count, OS, or machine — every variable below is overridable on the CLI, so the same commands work on a laptop, a CI runner, or a dedicated benchmarking host.

1) Profile a package's benchmarks directly — no server involved, good for isolating one function (e.g. the storage engine or the RESP parser):

task profile:pkg                                          # ./internal/storage/..., all benchmarks
task profile:pkg PKG=./internal/resp/... BENCH=BenchmarkParseGet
task profile:view FILE=tmp/profile/cpu.out                # opens the CPU profile in the browser
task profile:view FILE=tmp/profile/mem.out ARGS=-alloc_space

2) Profile the running server under real load, generated from a second terminal:

task run:profiling                    # foreground server, RESP + live pprof on :6060

# in a second terminal, generate load, e.g.:
task bench:resp:pipeline
# or: ./bin/benchmark -addr 127.0.0.1:19988 -c 32 -n 1000000 -read-ratio 0.95 -skew 1.5

# while load is running, pull a profile and open it in the browser:
task profile:live                     # CPU, 30s sample (default)
task profile:live KIND=heap
task profile:live KIND=mutex
task profile:live KIND=block
task profile:live:trace               # execution trace, opened via `go tool trace`

go tool pprof -http starts a local web server and opens your default browser automatically. On a headless/remote host, set PORT=<port> and open http://<host>:<port> yourself (e.g. via an SSH tunnel), or browse the raw index at http://127.0.0.1:6060/debug/pprof/ directly.

---

Contributing

Contributions are welcome — especially around networking, replication, persistence, and RESP command coverage. See CONTRIBUTING.md for the full guide (DCO sign-off required, core principles, workflow).

---

“A contest of focus. Keep yours made of steel.” — Tellstone