talk-data.com

Join us for an unmissable evening of insight, discussion, and lively debate at The High Performance Data and AI Debate, hosted by Chris Tabb — a unique Big Data London special running from 6:00–8:00 PM. This fast-paced, interactive event brings together some of the brightest minds in data and AI to tackle the most pressing questions shaping the future of teams, architecture, and products in an AI-first world.

The evening kicks off at 6:00 PM with a welcome and free drinks. Then, across three rapid-fire 20-minute debates, our expert panels will explore:

• AI & Data – Teams (Chair: Eevamaija Virtanen)

Mehdi Ouazza, Paul Rankin, Jesse Anderson, Hugo Lu

• AI & Data – Architecture (Chair: Adi Polak)

Chris Freestone, David Richardson, Nick White, Karl Ivo Sokolov

• AI & Data – Products (Chair: Jai Parmar)

Kelsey Hammock, Jean-Georges (jgp) Perrin, Taylor McGrath, Jon Cooke

Refuel with free pizza at 6:50 PM, then stay for the Town Hall Debate, where all speakers return to the stage for an open-floor Q&A — your chance to challenge their ideas, share perspectives, and shape the conversation.

Expect fresh perspectives, healthy disagreement, and practical takeaways you can bring back to your organisation. Whether you’re leading a data team, designing cutting-edge architectures, or building AI-powered products, this is your space to engage with the people shaping what’s next.

Get ready for the first Data Engineer Things meetup of 2025!!! 🥳🥳

We're super excited to be hosted by the Information Lab at their chic offices in St. Pauls and boy do we have some cool talks! We're still looking for ONE Speaker so if you know anyone who wants to chat data platforms, let us know!

🗣️ Robert Vadai - Data Engineer at Merlin Entertainment 💡 Scaling similarity: running algorithmically complex computation at scale in Spark for ELT Robert's is a Data Engineer and Big Data Architect, and most recently built an anti-money laundering analytics engine with scalable optimisations implemented in Spark. This has recursive machine learning models to process 500m transactions a day. Currently, Robert is building data platforms where he focusses on advancing a data product mindset, a platform anyone can use, Data Governance and Security

🗣️ Sanchit Agarwal - Data Engineer@BJSS 💡 Building a Data Platform in AWS, dbt, Iceberg Glue and Athena: a data engineer's perspective

🗣️ Hugo Lu - CEO@Orchestra 💡 Data Platforms without Boundaries: a crash course in Data Architecture and building platforms for scale on AWS, GCP and Azure

18:00 - 18:45 - Drinks, Food and Networking 18:45 - 19.15 - Scaling similarity: running algorithmically complex computation at scale in Spark for ELT 19:15 - 19.30 - Break 19.30 - 20:00 - Building a Data Platform in AWS, dbt, Iceberg Glue and Athena: a data engineer's perspective 20:00 - 20:15 - Data Platforms without Boundaries 20:15 onwards - Drinks, Networking and Pub

Sign-up if: you want to meet other folks in the data space and talk about real data engineering work :) Not just theory!

Really looking forward to seeing as many of you there as possible! DET Team

Summary The core task of data engineering is managing the flows of data through an organization. In order to ensure those flows are executing on schedule and without error is the role of the data orchestrator. Which orchestration engine you choose impacts the ways that you architect the rest of your data platform. In this episode Hugo Lu shares his thoughts as the founder of an orchestration company on how to think about data orchestration and data platform design as we navigate the current era of data engineering.

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data managementIt’s 2024, why are we still doing data migrations by hand? Teams spend months—sometimes years—manually converting queries and validating data, burning resources and crushing morale. Datafold's AI-powered Migration Agent brings migrations into the modern era. Their unique combination of AI code translation and automated data validation has helped companies complete migrations up to 10 times faster than manual approaches. And they're so confident in their solution, they'll actually guarantee your timeline in writing. Ready to turn your year-long migration into weeks? Visit dataengineeringpodcast.com/datafold today to learn how Datafold can automate your migration and ensure source to target parity. As a listener of the Data Engineering Podcast you clearly care about data and how it affects your organization and the world. For even more perspective on the ways that data impacts everything around us don't miss Data Citizens® Dialogues, the forward-thinking podcast brought to you by Collibra. You'll get further insights from industry leaders, innovators, and executives in the world's largest companies on the topics that are top of mind for everyone. In every episode of Data Citizens® Dialogues, industry leaders unpack data’s impact on the world, from big picture questions like AI governance and data sharing to more nuanced questions like, how do we balance offense and defense in data management? In particular I appreciate the ability to hear about the challenges that enterprise scale businesses are tackling in this fast-moving field. The Data Citizens Dialogues podcast is bringing the data conversation to you, so start listening now! Follow Data Citizens Dialogues on Apple, Spotify, YouTube, or wherever you get your podcasts.Your host is Tobias Macey and today I'm interviewing Hugo Lu about the data platform and orchestration ecosystem and how to navigate the available optionsInterview IntroductionHow did you get involved in building data platforms?Can you describe what an orchestrator is in the context of data platforms?There are many other contexts in which orchestration is necessary. What are some examples of how orchestrators have adapted (or failed to adapt) to the times?What are the core features that are necessary for an orchestrator to have when dealing with data-oriented workflows?Beyond the bare necessities, what are some of the other features and design considerations that go into building a first-class dat platform or orchestration system?There have been several generations of orchestration engines over the past several years. How would you characterize the different coarse groupings of orchestration engines across those generational boundaries?How do the characteristics of a data orchestrator influence the overarching architecture of an organization's data platform/data operations?What about the reverse?How have the cycles of ML and AI workflow requirements impacted the design requirements for data orchestrators?What are the most interesting, innovative, or unexpected ways that you have seen data orchestrators used?What are the most interesting, unexpected, or challenging lessons that you have learned while working on data orchestration?When is an orchestrator the wrong choice?What are your predictions and/or hopes for the future of data orchestration?Contact Info MediumLinkedInParting Question From your perspective, what is the biggest thing data teams are missing in the technology today?Closing Announcements Thank you for listening! Don't forget to check out our other shows. Podcast.init covers the Python language, its community, and the innovative ways it is being used. The AI Engineering Podcast is your guide to the fast-moving world of building AI systems.Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.Links OrchestraPrevious Episode: Overview Of The State Of Data OrchestrationCronArgoCDDAGKubernetesData MeshAirflowSSIS == SQL Server Integration ServicesPentahoKettleDataVoloNiFiPodcast EpisodeDagstergRPCCoalescePodcast EpisodedbtDataHubPalantirThe intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA

Join us at RVUs' STUNNING new Riverside Headquarters for the inaugural Data Engineer Things SUMMER BASH. An evening of networking, drinks, food and talks from some brilliant speakers from the Data Engineer community, including:

• Sid Dawara (Head of Data Engineering at RVU)- War stories and Data Architecture from the RVU Journey: from Chaos to Clarity.
• Ties Boukema (Data Lead at Dawn Capital) - How to build a data stack for venture capital

Event Schedule 6:30 - 7:00 : food, drinks, networking

7:00 - 7:20: Sid Dawara (Head of Data Engineering at RVU)- War stories and Data Architecture from the RVU Journey

7:20 - 7:40: Break

7:40 - 8:00 : Ties Boukema (Data Lead at Dawn Capital) How to build a data stack for venture capital

8:00-8:30: Q&A

8:30 onwards: Beverages

Join the guestlist 👉to secure your spot as spaces are limited.

This gathering of London’s Data Engineer Things community is organised by Hugo Lu (Orchestra) and sponsored by RVU

About us: Data Engineer Things (DET), founded by Xinran Waibel, is an online community and publication dedicated to curating learning resources for data engineers. The London chapter is focused on bringing together like-minded data engineers and practitioners to share experiences, and speak about high-quality real-world data engineer projects, and of course, socialise.

Talk submission:

If you'd like to submit a talk (or help us organise!) please reach out to Hugo or Tim Webster on Linkedin. Similar to the Data Engineer Things Medium blog, we recommend submitting talks on some of the topics below, which are educational and beneficial to the wider community!

Speaking about relevant professional experiences is also recommended: Project ideas: Interesting data engineering projects you've done (ideally in a professional capacity) ---> Tech stacks: Building a batch/streaming pipeline on AWS\, for example ---> Engineering process: Integrating CICD into data engineering dev cycle ---> Overlooked topics: e.g. how to design systems for high data quality Which could be interspersed with stories about: ---> Career switching: Transitioning into DE from another field ---> Interviews: Preparing for data engineering interviews

Resources: Slack: http://join.det.life Medium: https://blog.det.life Youtube Channel: https://www.youtube.com/@data-engineer-things/streams Website: https://www.dataengineerthings.org/aboutus/

Join Snowflake London User Group & DET community in Holborn at MMC HQ for an evening of networking, drinks, food and talks from some brilliant speakers from the Data Engineer community, including:

Anna Aleshko (Data Engineer and Founder of Data Engineers London)- How we power insights using Snowflake for a £7bn business: Data Architecture and Data Products at Waitrose [SURPRISE] - The ultimate combination: how to run dbt-Core on Snowflake Container Services

Alex Spanos (Head of Data @Tarabut) - Automating Data Workflows for ML in Middle East Open Banking

🤝Sponsored by Snowflake, Orchestra and MMC Ventures

Event Schedule 6:30 - 7:00 : food, drinks, networking

7:00 - 7:20: Anna Aleshko, Data Engineer - How we power insights for a £7bn business: Data Architecture and Data Products at Waitrose

7:20 - 7:40: [SURPRISE] - The ultimate combination: how to run dbt-Core on Snowflake Container Services + Q&A

7:40 - 8:00 : Alex Spanos (Head of Data @Tarabut) - Automating Data Workflows for ML in Middle East Open Banking

8:00-8:30: Q&A

8:30 onwards: drinks and pub

Join the guestlist 👉to secure your spot as spaces are limited.

This gathering of London’s Data Engineer Things community is organised by Hugo Lu (Orchestra) and sponsored by Snowflake and MMC Ventures

About DET: Data Engineer Things (DET), founded by Xinran Waibel, is an online community and publication dedicated to curating learning resources for data engineers. The London chapter is focused on bringing together like-minded data engineers and practitioners to share experiences, and speak about high-quality real-world data engineer projects, and of course, socialise.

Talk submission: If you'd like to submit a talk (or help us organise!) please reach out to Hugo or Tim Webster on Linkedin. Similar to the Data Engineer Things Medium blog, we recommend submitting talks on some of the topics below, which are educational and beneficial to the wider community!

Speaking about relevant professional experiences is also recommended:

Project ideas: Interesting data engineering projects you've done (ideally in a professional capacity)

---> Tech stacks: Building a batch/streaming pipeline on AWS, for example

---> Engineering process: Integrating CICD into data engineering dev cycle

---> Overlooked topics: e.g. how to design systems for high data quality

Which could be interspersed with stories about:

---> Career switching: Transitioning into DE from another field

---> Interviews: Preparing for data engineering interviews

DET Resources: Slack: http://join.det.life

Medium: https://blog.det.life

Youtube Channel: https://www.youtube.com/@data-engineer-things/streams

Website: https://www.dataengineerthings.org/aboutus/

Agenda

6:30 PM: food, drinks, networking

7:00 PM: Anna Aleshko, Data Engineer - How we power insights for a £7bn business: Data Architecture and Data Products at Waitrose

7:20 PM: [SURPRISE] - The ultimate combination: how to run dbt-Core on Snowflake Container Services + Q&A

7:40 PM: Dario Morelli - Lessons from Truelayer - Building the Data & Analytics Team for the UK's largest Open Banking provider

8:00 PM: Q&A

8:30 PM: drinks and pub

Speakers

Anna Aleshko - Waitrose (Data Engineer)

https://www.linkedin.com/in/anna-aleshko;

Dario Morelli - Truelayer (previous) (Head of Data Analytics)

https://www.linkedin.com/in/morellidario/;

Surprise Speaker

https://docs.snowflake.com/en/developer-guide/snowpark-container-services/overview;

Partner

Data Engineering Things (https://blog.det.life/)

Snowflake User Groups

Complete your event RSVP here: https://usergroups.snowflake.com/events/details/snowflake-london-presents-london-snowflake-user-group-x-data-engineering-things-community-meetup-at-mmc-hq-holborn/.

Join us in Holborn at MMC HQ for an evening of networking, drinks, food and talks from some brilliant speakers from the Data Engineer community, including:

• Anna Aleshko (Data Engineer and Founder of Data Engineers London)- How we power insights using Snowflake for a £7bn business: Data Architecture and Data Products at Waitrose
• [SURPRISE] - The ultimate combination: how to run dbt-Core on Snowflake Container Services
• Alex Spanos (Head of Data @Tarabut) - Automating Data Workflows for ML in Middle East Open Banking

Event Schedule 6:30 - 7:00 : food, drinks, networking

7:00 - 7:20: Anna Aleshko, Data Engineer - How we power insights for a £7bn business: Data Architecture and Data Products at Waitrose

7:20 - 7:40: [SURPRISE] - The ultimate combination: how to run dbt-Core on Snowflake Container Services + Q&A

7:40 - 8:00 : Alex Spanos (Head of Data @Tarabut) - Automating Data Workflows for ML in Middle East Open Banking

8:00-8:30: Q&A

8:30 onwards: drinks and pub

Join the guestlist 👉to secure your spot as spaces are limited.

This gathering of London’s Data Engineer Things community is organised by Hugo Lu (Orchestra) and sponsored by Snowflake and MMC Ventures

About us: Data Engineer Things (DET), founded by Xinran Waibel, is an online community and publication dedicated to curating learning resources for data engineers. The London chapter is focused on bringing together like-minded data engineers and practitioners to share experiences, and speak about high-quality real-world data engineer projects, and of course, socialise.

Talk submission:

If you'd like to submit a talk (or help us organise!) please reach out to Hugo or Tim Webster on Linkedin. Similar to the Data Engineer Things Medium blog, we recommend submitting talks on some of the topics below, which are educational and beneficial to the wider community!

Speaking about relevant professional experiences is also recommended: Project ideas: Interesting data engineering projects you've done (ideally in a professional capacity) ---> Tech stacks: Building a batch/streaming pipeline on AWS\, for example ---> Engineering process: Integrating CICD into data engineering dev cycle ---> Overlooked topics: e.g. how to design systems for high data quality Which could be interspersed with stories about: ---> Career switching: Transitioning into DE from another field ---> Interviews: Preparing for data engineering interviews

Resources: Slack: http://join.det.life Medium: https://blog.det.life Youtube Channel: https://www.youtube.com/@data-engineer-things/streams Website: https://www.dataengineerthings.org/aboutus/