Supabase ETL

High-performance Postgres replication, written in Rust.
Embed it as a library or run it as a standalone binary.

A project by Supabase.

Documentation · First Pipeline · Standalone Replicator · Examples · Issues

[!NOTE] Supabase ETL is under active development. APIs and setup steps may change before the first stable release.

Supabase ETL is a high-performance Postgres replication engine written in Rust. Embed it in your Rust application or run it as a standalone binary. For each published table, it performs an initial sync of existing rows, then replicates changes to a built-in or custom destination.

This repository contains the framework, destination modules, standalone replicator, examples, and documentation. For the managed product in the Supabase Dashboard—including availability, pricing, and operations—use the Supabase Pipelines documentation.

How It Works

flowchart LR
    Postgres["Postgres publication"] --> Sync["Initial sync"]
    Sync --> Replication["Ongoing replication"]
    Replication --> Destination["Destination"]

Supabase ETL replicates each table in two phases:

1. Initial sync: Copy the existing rows selected by the publication, then catch up changes that occurred while the copy was running.
2. Ongoing replication: Capture subsequent inserts, updates, deletes, and truncates, then deliver those changes as ordered events.

Across both phases, a store persists checkpoints, schemas, destination metadata, and table state so replication can recover safely after a restart. Copy and change data capture (CDC) are replication paths, not customer-visible phases. Initial sync uses the copy path followed by CDC catch-up; ongoing replication uses the CDC path after the table is ready.

Start Here

ETL is installed from Git while we prepare for a crates.io release:

[dependencies]
etl = { git = "https://github.com/supabase/etl" }
tokio = { version = "1", features = ["full"] }

Why Supabase ETL?

• Library or standalone binary: Embed the engine in a Rust application or run the ready-made replicator.
• High performance, small footprint: Run one Rust process without Kafka, Flink, Debezium, or another coordination service.
• Postgres-native selection: Use publications to select tables, columns, rows, and operation types.
• Flexible by design: Implement custom destinations and durable stores with typed Rust APIs.
• Recovery-aware: Persist checkpoints and table state for safe restarts and at-least-once delivery.

Destinations

ClickHouse is the fastest way to start locally: cargo x init runs it, and cargo x setup replicator configures it by default. BigQuery is the most mature cloud destination. See the Destinations reference for maturity and limitations, and the etl-examples guide for runnable examples.

Requirements

Supabase ETL supports PostgreSQL 14 through 18. PostgreSQL 15 or newer is recommended for column and row publication filters. PostgreSQL 16 or newer is required when the replication connection points at a physical read replica.

The source must use wal_level = logical, and the replication user needs the REPLICATION role. See Configure Postgres for the complete setup and production guidance.

Development

cargo x init              # Docker, databases, migrations
cargo x setup api && cargo x run api
cargo x setup replicator && cargo x seed && cargo x run replicator

See DEVELOPMENT.md to start the replicator or API, pick a destination, and run tests. The workspace uses Rust 1.95.0 from rust-toolchain.toml.

Contributing

Contributions are welcome. Before proposing a new destination, start a discussion or issue so maintainership and long-term demand can be evaluated first.

Report suspected vulnerabilities privately according to SECURITY.md.

License

Apache-2.0. See LICENSE.