How to move data from thousands of SQL databases to data lake with no impact on OLTP? We'll explore the challenges we faced while migrated legacy batch data flows to event-based architecture. A key challenge for our data engineers was the multi-tenant architecture of our backend, meaning that we had to handle the same SQL schema on over 15k databases. We'll present the journey employing Debezium, Azure Event Hub, Delta Live tables and the extra tooling we had to put in place.
talk-data.com
M
Speaker
Marco Santoni
1
talks
Data Engineering Manager
TeamSystem
Marco Santoni is a data engineering leader currently serving as Data Engineering Manager at TeamSystem, a leading tech company that provides products for businesses and professionals to manage finance, operations, and compliance. Before joining TeamSystem, Marco led the Data Platform team at Flowe, a digital banking startup, where he focused on building modern data infrastructure and fostering a data-driven culture.
Beyond his professional roles, Marco is an active contributor to the Italian Python community. He co-hosts the podcast Intervista Pythonista, which features bi-weekly interviews with developers and researchers in the Python ecosystem. He also helps organize local meetups and conferences, including PyCon Italia and Py4AI, promoting knowledge sharing and community growth.
Marco holds a degree from ETH Zurich and maintains a personal blog where he shares insights on data engineering, productivity, and the evolving tech landscape.
Bio from: Big Data LDN 2025
Filter by Event / Source
Talks & appearances
1 activities · Newest first