talk-data.com talk-data.com

Y

Speaker

Yingjun Wu

1

talks

CEO, RisingWave

Bio from: Data Streaming Lakehouse Tour (Paris)

Filtering by: Apache Iceberg Paris Community Meetup #2 ×

Filter by Event / Source

Talks & appearances

Showing 1 of 11 activities

Search activities →

At first, streaming Postgres changes into Iceberg feels like a no-brainer. You spin up Debezium or Kafka Connect, point it at Iceberg, and it all looks boringly straightforward. The surprise comes once the pipeline hits production. Replication slots vanish and start filling up WAL space, LSNs don't line up and cause duplicates or gaps, and Iceberg sinks fail in ways that push back all the way to your primary database. Then you throw in schema changes, backfills, and compaction, and suddenly the \"boring\" pipeline becomes a source of late-night firefights. In this talk, I'll share real stories from running Postgres to Iceberg CDC pipelines in production. We'll look at the unexpected problems that show up, why they happen, and the strategies that actually helped keep things stable. If you've ever thought of Postgres -> Iceberg as just plumbing, this session will show you why it's not so boring after all.