The Quack Client/Server Protocol for DuckDB

Quack is released as a pre-release extension and is currently experimental. If you encounter any issues, please file them via GitHub.

The quack extension adds a client-server protocol to DuckDB. With this extension, DuckDB can act as both a server and a client to communicate over a network. For more details, please see the announcement page, the blog post and the documentation.

Usage Example

The Quack extensions autoinstalls and autoloads on first use. You can also install and load it manually using:

INSTALL quack;
LOAD quack;

Start Quack on one DuckDB instance, the server, using:

CALL quack_serve('quack:localhost', token = 'super_secret');
CREATE TABLE hello AS FROM VALUES ('world') v(s);

And talk to this server from another instance:

CREATE SECRET (TYPE quack, TOKEN 'super_secret');
ATTACH 'quack:localhost' AS remote;
FROM remote.hello;

This should show the content of the remote table hello on the client side.

Starting the server from a secret

The server can take its token from a quack secret instead of the token parameter. Without arguments, quack_serve uses the default secret; secret = 'name' picks a specific one. When the chosen secret is scoped to a concrete endpoint, that endpoint is what the server listens on:

CREATE SECRET s1 (TYPE quack, TOKEN 'super_secret', SCOPE 'quack:localhost:9494');
CALL quack_serve(secret = 's1');   -- listens on quack:localhost:9494 with token 'super_secret'

CREATE SECRET (TYPE quack, TOKEN 'super_secret');
CALL quack_serve();                -- default secret, listens on quack:localhost

If no secret matches and no token is given, quack_serve generates a random token and returns it in the auth_token column. To keep that token across restarts, ask for it to be persisted:

CALL quack_serve(create_secret_if_not_exists = true);

This writes the token to a persistent default secret (__default_quack, scoped to quack:), so the next quack_serve - and any client on the machine - reuses the same token. It only does so when there is nothing to reuse yet: if a secret was named with secret =, or a default secret already matches, nothing is written.

Connecting through a secret

The client side mirrors this. ATTACH takes a SECRET name, and a bare quack: path takes its endpoint from that secret, so both ends of the connection can be described by the same secret:

CREATE SECRET s1 (TYPE quack, TOKEN 'super_secret', SCOPE 'quack:localhost:9494');
ATTACH 'quack:' AS remote (TYPE quack, SECRET s1);   -- connects to quack:localhost:9494

Without a SECRET name the default secret is used: the one whose scope matches the path, or - when the path names no host - the only quack secret there is. A path given explicitly always wins over the scope of the secret, and a secret scoped to nothing more specific than quack: leaves the default host in place.

We can also copy data from client to server:

-- on client
CREATE TABLE remote.hello2 AS FROM VALUES ('world2') v(s);

-- on server
FROM hello2;

Development

Managing dependencies

DuckDB extensions uses VCPKG for dependency management. Enabling VCPKG via the provided makefile target:

make setup-vcpkg
export VCPKG_TOOLCHAIN_PATH=`pwd`/vcpkg/scripts/buildsystems/vcpkg.cmake

Build steps

Now to build the extension, run:

make

The main binaries that will be built are:

./build/release/duckdb
./build/release/test/unittest
./build/release/extension/quack/quack.duckdb_extension

• duckdb is the binary for the duckdb shell with the extension code automatically loaded.
• unittest is the test runner of duckdb. Again, the extension is already linked into the binary.
• quack.duckdb_extension is the loadable binary as it would be distributed.

Running the tests

Different tests can be created for DuckDB extensions. The primary way of testing DuckDB extensions should be the SQL tests in ./test/sql. These SQL tests can be run using:

make test