Glider

Vector search with S3 as the source of truth.

Website and interactive storage simulation

Glider is a single-node vector database. Every acknowledged write is durable in S3-compatible object storage before the server answers; local RAM and SSD only speed up reads and never hold the only copy. It serves nearest-neighbor search with metadata filters over an HTTP/JSON API, with a Python client and an MCP memory server for AI agents.

Quickstart

Start a server with a local directory as storage (data is lost when the container stops):

docker run --rm -p 8080:8080 \
  -e GLIDER_DATA_DIR=/var/lib/glider/data ghcr.io/omerfeyzioglu/glider:latest

Create a collection, write two points and query them:

curl -sS localhost:8080/v1/collections -H 'content-type: application/json' \
  -d '{"name":"demo","dimensions":3}'

curl -sS localhost:8080/v1/collections/demo/write -H 'content-type: application/json' \
  -d '{"upsert":[{"id":1,"vector":[0,0,0],"metadata":{"color":"red"}},{"id":2,"vector":[1,1,1]}]}'

curl -sS localhost:8080/v1/collections/demo/query -H 'content-type: application/json' \
  -d '{"vector":[1,1,0.9],"k":2,"include_metadata":true}'

The same with the dependency-free Python client:

# pip install "git+https://github.com/omerfeyzioglu/glider#subdirectory=clients/python"
from glider_client import Client

client = Client("http://localhost:8080")
client.create_collection("docs", dimensions=3, metric="cosine")
docs = client.collection("docs")
docs.upsert([{"id": 1, "vector": [0, 0, 1], "metadata": {"lang": "en"}}])
print(docs.query([0, 0, 1], k=1, include_metadata=True))

Open http://localhost:8080/console to browse collections and run queries in the browser.

For S3 or MinIO storage, the Docker Compose demo and building from source, see installation; every setting is listed in configuration.

Features

• Durable on S3. Each write batch becomes one immutable log object, created conditionally before it is acknowledged; nothing is overwritten in place.
• Crash-safe takeover. A restarted server waits out the old writer's lease, fences it at the object store and replays the log; no operator step is needed after a crash.
• Retry-safe writes. Every write carries a request ID; resending it within the retained 128-commit window returns the original outcome instead of applying the write twice. Supply and keep the ID before sending so a lost response can be retried (retry contract).
• Collections. One server creates and serves many collections, each with its own dimension and metric (squared Euclidean, Manhattan or cosine).
• Metadata filters. Equality, set, existence, numeric and logical filters; exact: true answers any filter exhaustively.
• Clustered ANN that builds itself. The server clusters a collection in the background at 250,000 rows and rebuilds the clusters as it grows.
• SSD cache. A local block cache warms in the background so warm queries need no remote reads; losing it loses no data.
• Operations. Prometheus metrics, a status endpoint, bearer-token authentication, and glider-admin for backup and restore.

Agent memory (MCP)

The Python client includes glider-mcp, an MCP server that gives agents (Claude Code, Claude Desktop, Cursor and other MCP clients) durable remember, recall and forget tools backed by Glider:

docker run -d --name glider -p 8080:8080 -v glider-data:/var/lib/glider \
  -e GLIDER_DATA_DIR=/var/lib/glider/data ghcr.io/omerfeyzioglu/glider:latest
pip install "glider-client[mcp] @ git+https://github.com/omerfeyzioglu/glider#subdirectory=clients/python"
claude mcp add glider -e GLIDER_COLLECTION=memory -- glider-mcp

The client guide covers other MCP clients and settings.

Performance

1,000,000 SIFT vectors (128 dimensions, k=10) in AWS S3 Standard, served from a c7g.2xlarge (8 vCPU, 16 GiB) in eu-central-1 while four writers and four readers run concurrently:

The run at revision 35f9b46 lost no acknowledged writes and returned equal results after cache loss and backup restore. Methodology and raw results are in the benchmark report and BENCHMARKS.md.

Many small collections on one server, same instance and S3 Standard: 10,000 collections of 1,000 vectors (10,000,000 vectors) were created and loaded in 22 minutes with the server killed (SIGKILL) halfway; every tenant was verified with no lost or duplicated write. Warm queries over 64 active collections ran at 2,457 per second with a p95 of 18.5 ms and recall@10 of 1.0; a cold tenant opened from S3 and answered its first query in 716 ms (p95). Idle collections close, so they cost no S3 requests (details).

Architecture

• Write: a single committer publishes each batch as one conditional log object, then acknowledges it.
• Maintenance: idle time seals the log into immutable packs and indexes, then publishes a new root generation that names them.
• Read: queries route to candidate blocks, read bounded byte ranges from S3 or the SSD cache, and rerank full vectors.

The architecture guide adds an AWS deployment pattern; DESIGN.md specifies formats, invariants and recovery.

Documentation

Limitations

• Single node: no replication, sharding or standby; after a crash, the next server waits for the writer lease (10 s by default) before taking over.
• Filtered queries are approximate and may return fewer than k results unless they use exact: true or the collection's resident filter.
• Cache loss can lower approximate recall as well as increase latency; acknowledged data remains durable. Use exact: true for exhaustive results.
• A collection's dimension and metric are fixed when it is created.
• The server speaks plain HTTP; terminate TLS in a reverse proxy.
• The measured 1M-vector collection opens from S3 in a few seconds; larger collections have no fixed open-time guarantee.

Contributing

Contributions are welcome. See CONTRIBUTING.md for the workflow and the checks CI runs.

License

Licensed under either of Apache License, Version 2.0 or MIT license at your option. Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this project, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.