Copyright Debezium Authors. Licensed under the Apache License, Version 2.0.

Debezium Examples

This repository contains multiple examples for using Debezium, e.g. configuration files, Docker Compose files, OpenShift templates.

Getting Started

For getting started please check the tutorial example.

Examples

• AI RAG: How to implement Retrieval-Augmented Generation (RAG) using Debezium Server to stream Postgres changes, compute embeddings, and store them in a Milvus vector database for use by a local Ollama-served LLM
• Apache Pulsar: Streaming Postgres database changes to Apache Pulsar
• Audit Logs: Building Audit Logs with Change Data Capture
• Cache Invalidation: How Debezium can be used to invalidate items in the JPA 2nd level cache after external data changes
• Camel - pipelines: Building an Apache Camel pipeline that captures Postgres database changes
• Camel - Kafka Connect: How to use the Camel Kafka Connect component with Debezium
• Cloud Events: How to use cloud events defined in Json with Debezium
• CQRS: How to implement Command Query Responsibility Segregation (CQRS) pattern with Debezium
• Data Replication: Kafka-less data replication between databases using Debezium Server, Debezium Operator, and Debezium Platform with the JDBC sink
• Database Activity Monitoring: How to use Debezium for comprehensive database activity logging and analysis
• Debezium - End-to-end demo: End-to-end demo using MySQL as database and Kafka Connect
• Debezium - Failover: How Debezium recovers after a database failure
• Debezium - Monitoring: Monitoring a Debezium instance
• Debezium - Auto-Creation of Topics: Auto-creating Debezium change data topics
• Debezium - Outbox Pattern: Implement the "outbox pattern", an approach for letting services communicate in an asynchronous and reliable fashion
• Debezium Management Platform - Streaming to Apache Kafka: How to use the debezium-platform to create a data pipeline to stream changes from a PostgreSQL database to Apache Kafka broker in kubernetes cluster.
• Debezium Management Platform - Streaming to Amazon Kinesis on AWS: How to run the Debezium Management Platform on a k3s cluster on AWS EC2 and stream changes from Amazon RDS for PostgreSQL to an Amazon Kinesis stream
• Debezium - Python Connect-Mode: How to embed Debezium into a Python application using Connect-mode with no JSON serialization overhead
• Debezium - Saga Pattern: How to implement the Saga pattern for realizing distributed transactions across multiple microservices
• Debezium - Testing: How to implement an integration test for your CDC set-up using Testcontainers
• Debezium on Plain Kafka Containers: Running Debezium with official Apache Kafka and Confluent Connect container images (instead of pre-baked Debezium images)
• Debezium Connect - Using Kafka with SSL enabled: How to configure Debezium Connect to use a Kafka instance with SSL enabled
• Debezium Server - Using Google Cloud Pub/Sub: How to deploy Debezium Server using Postgres, MongoDB, and MySQL as data sources and Google Cloud Pub/Sub as a destination
• Debezium Server - Using Amazon Kinesis: How to deploy Debezium Server using Postgres, MongoDB, and MySQL as data sources and Amazon Kinesis as a destination
• Debezium Server - Using storage for offset and schema history: How to deploy Debezium Server using MySQL as data sources, Redis and storage for offset and schema history, and Google Cloud Pub/Sub emulator as a destination
• Debezium Server - Using custom topic naming policy: How to deploy Debezium Server using a custom topic naming policy
• Debezium Server - Operator - Using Kafka: How to stream changes from a PostgreSQL database into Apache Kafka with Debezium Server deployed in a Kubernetes cluster
• Debezium Server - Operator - Using Kafka: How to stream changes from a PostgreSQL database into Google Cloud Pub/Sub with Debezium Server deployed in a Kubernetes cluster
• Apicurio Registry: How to use Debezium with Apicurio Registry for schema management, supporting Avro and JSON converters with MySQL
• Graphql: How to build a GraphQL Subscription on top of Debezium Change Events
• HTTP Signaling and Notification: How to create custom signaling and notification channels for Debezium connectors
• Infinispan - Standalone: How to use a standalone Infinispan cluster that will be used for buffering in-progress transactions by the Debezium Oracle connector
• Infinispan - Distributed Caching: How to combine Debezium and Infinispan for an CQRS-style application design
• JPA Aggregations: How to materialize consistent aggregates using a PoC-level Hibernate ORM extension
• Using multiple databases: How to capture and stream change events from multiple databases such as MySQL and PostgreSQL
• JSON Logging: This example uses Logstash json_event pattern for log4j
• Kafka KStreams - Using Kafka Connect MongoDB: How to use kstreams topologies and the Kafka Connect MongoDB sink connector
• Kafka KStreams - Foreign Key Joins: How two Debezium change data topics can be joined via Kafka Streams
• Kafka KStreams - WebSockets: How to use KStreams and stream the merged events to a client using WebSockets
• Machine Learning - TensorFlow: Image classification with Debezium and TensorFlow
• Machine Learning - K-means: Iris classification using streaming k-means and Apache Flink
• Kinesis: How to stream changes from MySQL database running on a local machine to an Amazon Kinesis stream
• KSQL: Querying Debezium change data events with KSQL
• Neo4j - Relational to graph: How to stream a relational Postgres schema into a Neo4j graph using the Neo4jCudConverter SMT, a split step (Kafka Streams or ksqlDB), and the official Neo4j Kafka sink connector
• Podman: How to run the Debezium using Podman.
• Postgres - Failover slots: How to use Postgres 17 failover replication slots with Debezium
• Postgres - Kafka signal: How to stream data into the signal topic
• Postgres - TOAST Column Values: Dealing With Postgres TOAST Column Values
• SQL Server - Replication: How to deploy the topology of services to stream from SQL Server read-only replica
• SQL Server - Streaming to Oracle: How to capture events from a SQL Server database and stream them into an Oracle database using the Debezium JDBC sink connector
• MongoDB - Streaming to PostgresSQL: How to capture events from a MongoDB database and stream them to a PostgresSQL database
• Milvus - Streaming to PostgreSQL (pgvector): How to capture changes from a Milvus vector database and stream them into PostgreSQL with pgvector using the Debezium JDBC sink connector
• MySQL - Streaming to PostgresSQL and Elasticsearch: How to capture events from a MySQL database and stream them to a PostgresSQL and Elasticsearch database
• Quarkus Native Image with Debezium: superfast CDC with Debezium and Quarkus
• Quarkus Debezium Extension Quick Start: superfast CDC with Debezium Extension for Quarkus
• MongoDB - Failover: Demo showing how Debezium works for mongodb when primary replica fails.