In this talk we will look into the details of how Kleinanzeigen, a leader in classifieds business in Germany, built a data migration system using Apache Kafka and Debezium that migrated millions of users' data from a legacy to a new platform and allowed bi-directional data sync between them in real time. We will also discover how the system allowed user's data to be updated on both platforms (partially, under certain conditions) while keeping the entire system in sync. Finally, we will learn how the system leveraged a logical clock to implement a custom synchronization algorithm that helped avoid infinite update loops between the platforms.
talk-data.com
Topic
debezium
2
tagged
Activity Trend
2
peak/qtr
2020-Q1
2026-Q1
Top Events
Data pipeline troubleshooting: Root cause analysis with lineage metadata
1
Berlin Open Source Data Infrastructure Meetup - November 2023
1
Debezium, Apache Kafka®, and an Acyclic Synchronization Algorithm
1
Data Berlin Midsummer Meetup - Speeding up Analytics
1
Debezium, Apache Kafka®, and an Acyclic Synchronization Algorithm
1
Top Speakers
Filtering by:
MD Sayem Ahmed
×
In this talk we will look into the details of how Kleinanzeigen, a leader in classifieds business in Germany, built a data migration system using Apache Kafka and Debezium that migrated millions of users' data from a legacy to a new platform and allowed bi-directional data sync between them in real time. We will also discover how the system allowed user's data to be updated on both platforms (partially, under certain conditions) while keeping the entire system in sync. Finally, we will learn how the system leveraged a logical clock to implement a custom synchronization algorithm that helped avoid infinite update loops between the platforms.