📅 Dallas/Fort Worth Postgres · April 14, 2026

Real-Time Data Streaming
with PostgreSQL
and Debezium

Capture every INSERT, UPDATE & DELETE straight from the WAL —
sub-second latency, zero schema changes required.

HV

Hector Ventura

Lead Software Engineer · Debezium Contributor

McKinney, Texas — hectorvent.dev

01 / 10

Speaker

About Me

HV

Hector Ventura

📍 McKinney, Texas

Current Role

Lead Software Engineer at Consensus Cloud Solutions
Co-founder at Alphnology
20+ years in backend systems, distributed services & cloud-native apps

Tech Stack

Open Source & Community

⚡ Creator of Floci — fast MIT-licensed AWS service emulator
🔷 Quarkus 1,000 Contributors · Active Debezium & Eclipse Vert.x contributor
🎤 Active in Dallas/Fort Worth Postgres & Java Dominicano
🎓 Former Linux Admin instructor (2007–2012) · CS @ UASD · Java tutor on Wyzant

02 / 10

Overview

What We'll Cover

01

Change Data Capture (CDC)

What it is, why it matters, and how it replaces slow polling approaches

02

PostgreSQL WAL & Logical Decoding

How Postgres exposes changes via the Write-Ahead Log using pgoutput

03

Debezium Deep Dive

Architecture, connectors, and WAL-to-JSON/Avro event conversion

04

Setup: Docker & Kafka Connect

Live configuration of Postgres logical replication + Debezium connector

05

Schema Evolution

Handling downstream schema changes safely with Schema Registry strategies

06

Real-World Use Cases

Cache invalidation, search indexing, microservices sync, audit logging

03 / 10

Foundations

Change Data Capture

"A set of software design patterns used to determine and track data changes to enable downstream action."

✓Sub-second latency — no polling delays, reads the transaction log
✓No schema changes — no updated_at columns required
✓Lossless — captures all operations, even during downtime
→Enables event-driven architectures at the data layer
✗Polling misses fast insert+delete within the same interval

Captured Operations

INSERT New row added → full after state

UPDATE Row changed → before + after state

DELETE Row removed → before state (tombstone)

Why Not Just Poll?

✗Misses deletes & same-interval round-trips
✗High read load on the database at scale
✗Requires an updated_at column on every table

04 / 10

PostgreSQL Internals

WAL & Logical Decoding

Production — Kafka Connect

🐘

PostgreSQL

Write-Ahead Log

pgoutput

🔄

Debezium

Kafka Connect

JSON / Avro

📨

Apache Kafka

topic per table

consume

⚙️

Consumers

any downstream

Demo — Embedded Engine (no Kafka)

🐘

PostgreSQL

Write-Ahead Log

pgoutput

📦

Your Application

single process

Debezium Engine Consumer Logic

Enable logical replication — postgresql.conf

# postgresql.conf
wal_level             = logical
max_replication_slots = 4
max_wal_senders       = 4

Create publication & replication slot

-- Grant replication to the connector user
CREATE PUBLICATION dbz_pub
  FOR ALL TABLES;

SELECT pg_create_logical_replication_slot(
  'debezium', 'pgoutput'
);

05 / 10

Platform

Debezium Overview

🔓

Open Source

Apache-licensed, built on Kafka Connect. Production-ready with thousands of global deployments.

🐘

PostgreSQL Connector

Uses pgoutput — built-in since Postgres 10, no extra plugins needed.

📸

Snapshot Mode

Initial snapshot of all existing rows, then streams every subsequent change in real time.

🗂️

Topic Per Table

Each table maps to its own Kafka topic: server.schema.table

🛡️

At-Least-Once

Replication slots ensure zero event loss. Offsets in Kafka guarantee delivery even after restarts.

🔗

Schema Registry

Integrates with Confluent Schema Registry (Avro / JSON Schema) for safe schema evolution.

06 / 10

Hands-On

Setup: Docker & Kafka Connect

1

Start Kafka + Zookeeper

# docker-compose.yml
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:7.6
  kafka:
    image: confluentinc/cp-kafka:7.6
    environment:
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092

2

Start Debezium Connect

connect:
  image: debezium/connect:2.7
  environment:
    BOOTSTRAP_SERVERS: kafka:9092
    GROUP_ID: "1"
    CONFIG_STORAGE_TOPIC: connect_configs
    OFFSET_STORAGE_TOPIC: connect_offsets

3

Register PostgreSQL Connector

# POST /connectors
{
  "name": "pg-connector",
  "config": {
    "connector.class":
      "...postgresql.PostgresConnector",
    "database.hostname": "postgres",
    "plugin.name": "pgoutput"
  }
}

4

Sample Change Event (INSERT)

{
  "op": "c",
  "before": null,
  "after": {
    "id": 42,
    "email": "hector@example.com",
    "created_at": 1713139200000
  },
  "source": { "table": "users" }
}

07 / 10

Real World

Use Cases

🗄️

Cache Invalidation

Automatically expire or refresh Redis / Memcached entries the moment a database row changes — no more stale cache bugs or manual TTL juggling.

🔍

Search Index Sync

Keep Elasticsearch or OpenSearch perfectly in sync with your Postgres data in real time — no batch re-index jobs needed.

🔗

Microservices Sync

Propagate domain events across service boundaries reliably. Each service consumes only the tables relevant to its bounded context.

📋

Audit Logging

Capture a complete, tamper-proof history of every data change — before and after state — without touching any application code.

08 / 10

Hands-On

Live Demo — Real-Time Fraud Detection

Architecture (Embedded — No Kafka)

🐘

PostgreSQL

WAL · pgoutput

CDC events

📦

Java App

single process

Debezium Engine Fraud Detector

Transfer Event — captured fields

📅 transferred_at TIMESTAMPTZ

📍 location VARCHAR

🌐 latitude / longitude DOUBLE

💰 amount NUMERIC(15,2)

🏦 account_id VARCHAR

💱 currency CHAR(3)

Source Code & Run

🐙 github.com/hectorvent/debezium-postgres

$ docker compose up -d # start postgres $ mvn exec:java # start CDC engine + simulator

Fraud Scenarios — what you'll see

💸

Scenario A — High Amount

ACC-FRAUD-001 sends $50,000 from New York

HIGH AMOUNT t + 5s

✈️

Scenario A' — Impossible Travel

Same account appears in Tokyo just 5 seconds later
10,838 km at ~7,800,000 km/h

IMPOSSIBLE TRAVEL t + 10s

🌍

Scenario B — London → Sydney in 3s

ACC-FRAUD-002 in London, then Sydney 3 seconds later
16,993 km — physically impossible

IMPOSSIBLE TRAVEL t + 22s

🔥

Scenario C — Double Hit

ACC-FRAUD-003 sends $12,000 from Miami,
then appears in Dubai 5 seconds later

HIGH AMOUNT IMPOSSIBLE TRAVEL t + 37s

09 / 10

Q & A

Thanks for joining — let's talk Postgres, Debezium & event-driven data!

🌐 hectorvent.dev 📖 Debezium Docs 🐘 PG Logical Decoding 🐙 github.com/hectorvent

PostgreSQL Debezium Kafka CDC Event-Driven WAL pgoutput

10 / 10

Real-Time Data Streaming with PostgreSQL and Debezium

About Me

What We'll Cover

Change Data Capture (CDC)

PostgreSQL WAL & Logical Decoding

Debezium Deep Dive

Setup: Docker & Kafka Connect

Schema Evolution

Real-World Use Cases

Change Data Capture

Captured Operations

Why Not Just Poll?

WAL & Logical Decoding

Debezium Overview

Open Source

PostgreSQL Connector

Snapshot Mode

Topic Per Table

At-Least-Once

Schema Registry

Setup: Docker & Kafka Connect

Use Cases

Cache Invalidation

Search Index Sync

Microservices Sync

Audit Logging

Live Demo — Real-Time Fraud Detection

Scenario A — High Amount

Scenario A' — Impossible Travel

Scenario B — London → Sydney in 3s

Scenario C — Double Hit

Real-Time Data Streaming
with PostgreSQL
and Debezium