For years, data engineering was a story of predictable "pipelines": move data from point A to point B. But AI just hit the reset button on our entire field. Now, we're all staring into the void, wondering what's next. While the fundamentals haven't changed, data remains challenging in the traditional areas of data governance, data management, and data modeling, which still present challenges. Everything else is up for grabs. This talk will cut through the noise and explore the future of data engineering in an AI-driven world. We'll examine how team structures will evolve, why agentic workflows and real-time systems are becoming non-negotiable, and how our focus must shift from building dashboards and analytics to architecting for automated action. The reset button has been pushed. It's time for us to invent the future of our industry.

For years, data engineering was a story of predictable pipelines: move data from point A to point B. But AI just hit the reset button on our entire field. Now, we're all staring into the void, wondering what's next. While the fundamentals haven't changed, data remains challenging in the traditional areas of data governance, data management, and data modeling, which still present challenges. Everything else is up for grabs.

This talk will cut through the noise and explore the future of data engineering in an AI-driven world. We'll examine how team structures will evolve, why agentic workflows and real-time systems are becoming non-negotiable, and how our focus must shift from building dashboards and analytics to architecting for automated action. The reset button has been pushed. It's time for us to invent the future of our industry.

Practicing analytics well takes more than just tools and tech. It requires data modeling practices that unify and empower all teams within analytics, from engineers to analysts. This is especially true as AI becomes a part of analytics. Without a governed data model that provides consistent data interpretation, AI tools are left to guess. Join panelists Joe Reis, Sarah Levy, Harry Gollop, Rob Hulme, Shachar Meir, and Guy Fighel, as they share battle-tested advice on overcoming conflicting definitions and accurately mapping business intent to data, reports and dashboards at scale. This panel is for data & analytics engineers seeking a clear framework to capture business logic across layers, and for data leaders focused on building a reliable foundation for Gen AI.

For years, data engineering was a story of predictable pipelines: move data from point A to point B. But AI just hit the reset button on our entire field. Now, we're all staring into the void, wondering what's next. While the fundamentals haven't changed, data remains challenging in the traditional areas of data governance, data management, and data modeling, which still present challenges. Everything else is up for grabs.

This talk will cut through the noise and explore the future of data engineering in an AI-driven world. We'll examine how team structures will evolve, why agentic workflows and real-time systems are becoming non-negotiable, and how our focus must shift from building dashboards and analytics to architecting for automated action. The reset button has been pushed. It's time for us to invent the future of our industry.

In this discussion, I sit down with data veterans Remco Broekmans and Marco Wobben to explore why so many data projects fail. They argue that the problem isn't the technology, but a fundamental misunderstanding of communication, culture, and long-term strategy.The conversation goes deep into the critical shift from being a "hardcore techie" to focusing on translating business needs into data models. They use the classic "involved party" data modeling pattern as a prime example of how abstract IT jargon creates a massive disconnect with the business.Marco shares a fascinating (and surprising) case study of the Dutch Railroad organization, which has been engaged in an 18-year information modeling "program" - not a project - to manage its immense complexity. This sparks a deep dive into the cultural and work-ethic differences between the US and Europe, contrasting the American short-term, ROI-driven "project" mindset with the European capacity for long-term, foundational "programs".Finally, they tackle the role of AI. Is it a silver bullet or just the latest shiny object? They conclude that AI's best use is as an "intern" or "assistant", a tool to brainstorm, ask questions, and handle initial prototyping, but never as a replacement for the deep, human-centric work of understanding a business.Timestamps:00:00 - Introduction01:09 - Marco Wobben introduces his 25-year journey in information modeling.01:56 - Remco Broekmans reintroduces himself and his focus on the communication aspect of data.03:22 - The progression from hardcore techie to focusing on communication over technology.08:16 - Why is communication in data and IT projects so difficult? 09:49 - The "Involved Party" Problem: A perfect example of where IT communication goes wrong with the business.13:35 - The essence of IT is automating the communication that happens on the business side.18:39 - Discussing a client with 20,000 distinct business terms in their information model.21:55 - The story of the Dutch Railroad's 18-year information modeling program that reduced incident response from 4 hours to 2 seconds.27:25 - Project vs. Program: A key mindset difference between the US and Europe.34:18 - The danger of chasing shiny new tools like AI without getting the fundamentals right first.39:55 - Where does AI fit into the world of data modeling? 43:34 - Why you can't trust AI to be the expert, especially with specialized business jargon.47:18 - The role of risk in trusting AI, using a self-driving car analogy.53:27 - Cultural differences in work pressure and ethics between the US and the Netherlands.59:29 - Why personality and communication skills are more important than a PhD for data modelers.01:03:38 - What is the purpose of an AI-run company with no human benefit? 01:11:21 - Using AI as an instructive tool to improve your own skills, not just to get an answer.01:14:12 - How AI can be used as a "sidekick" to ask dumb questions and help you think.01:18:00 - Where to find Marco and Remco online

Madison Schott joins me to chat about about her journey from working as an analytics engineer to creating content about dbt, SQL, data modeling, and more.

John Giles joins me to discuss his new book, "The Data Elephant in the Board Room," conceptual modeling, the unreasonable effectiveness of data modeling patterns, and more.

In data modeling - and pretty much anything else - do you choose "old school" or "new school"? In other words, do you move slow and methodically or fast?

Remco Broekmans and I chat about data modeling and the business, Data Vault, and using AI to accelerate data modeling.

Jamie Davidson (Chief Product Officer at Omni, Former VP of Product at Looker) joins me to chat about "modern" data modeling, going from a startup to Google and back to a startup, and much more.

Omni: https://omni.co/

It's 2025! We made it! ;)

In this podcast, I rant about why data modeling matters more than ever, AI, and why humans will seek out "human" things in 2025 and beyond.

❤️ Your support means a lot. Please like and rate this podcast on your favorite podcast platform.

🤓 My works:

📕Fundamentals of Data Engineering: https://www.oreilly.com/library/view/fundamentals-of-data/9781098108298/

🎥 Deeplearning.ai Data Engineering Certificate: https://www.coursera.org/professional-certificates/data-engineering

🔥Practical Data Modeling: https://practicaldatamodeling.substack.com/

🤓 My SubStack: https://joereis.substack.com/

It's December 31, 2024. Gordon Wong and I wrap up 2024 and chat about what we're excited about in 2025 in data and otherwise.

❤️ If you like my podcasts, please like and rate it on your favorite podcast platform.

🤓 My works:

📕Fundamentals of Data Engineering: https://www.oreilly.com/library/view/fundamentals-of-data/9781098108298/

🎥 Deeplearning.ai Data Engineering Certificate: https://www.coursera.org/professional-certificates/data-engineering

🔥Practical Data Modeling: https://practicaldatamodeling.substack.com/

🤓 My SubStack: https://joereis.substack.com/

Matt Housley and I have a LONG chat about working in consulting, leaving your job, AI, the job market, our thoughts on what's coming in 2025, and much more.

❤️ If you like my podcasts, please like and rate it on your favorite podcast platform.

🤓 My works:

📕Fundamentals of Data Engineering: https://www.oreilly.com/library/view/fundamentals-of-data/9781098108298/

🎥 Deeplearning.ai Data Engineering Certificate: https://www.coursera.org/professional-certificates/data-engineering

🔥Practical Data Modeling: https://practicaldatamodeling.substack.com/

🤓 My SubStack: https://joereis.substack.com/

This morning, a great article came across my feed that gave me PTSD, asking if Iceberg is the Hadoop of the Modern Data Stack?

In this rant, I bring the discussion back to a central question you should ask with any hot technology - do you need it at all? Do you need a tool built for the top 1% of companies at a sufficient data scale? Or is a spreadsheet good enough?

Link: https://blog.det.life/apache-iceberg-the-hadoop-of-the-modern-data-stack-c83f63a4ebb9

❤️ If you like my podcasts, please like and rate it on your favorite podcast platform.

🤓 My works:

📕Fundamentals of Data Engineering: https://www.oreilly.com/library/view/fundamentals-of-data/9781098108298/

🎥 Deeplearning.ai Data Engineering Certificate: https://www.coursera.org/professional-certificates/data-engineering

🔥Practical Data Modeling: https://practicaldatamodeling.substack.com/

🤓 My SubStack: https://joereis.substack.com/

People often ask me for career advice. In a tough job market where people are sending out thousands of resumes and hearing nothing back, I notice a lot of people have weak networks and are unknown to the companies they're applying to. This results in lots of frustration and disappointment for job seekers.

Is there a better way? Yes. People need to know who you are. Obscurity is your enemy.

Also, the name of the Friday show changed because I can't seem to keep things to five minutes ;)

My works:

📕Fundamentals of Data Engineering: https://www.oreilly.com/library/view/fundamentals-of-data/9781098108298/

🎥 Deeplearning.ai Data Engineering Certificate: https://www.coursera.org/professional-certificates/data-engineering

🔥Practical Data Modeling: https://practicaldatamodeling.substack.com/

🤓 My SubStack: https://joereis.substack.com/

Let's do things the right way, not just the fast way.

My works:

📕Fundamentals of Data Engineering: https://www.oreilly.com/library/view/fundamentals-of-data/9781098108298/

🎥 Deeplearning.ai Data Engineering Certificate: https://www.coursera.org/professional-certificates/data-engineering

🔥Practical Data Modeling: https://practicaldatamodeling.substack.com/

🤓 My SubStack: https://joereis.substack.com/

I speak at a lot of conferences, and I've lost track of how many questions I've answered. Since conferences are top of mind for me right now, here are some tips for asking good (and bad) questions of speakers.

My works:

📕Fundamentals of Data Engineering: https://www.oreilly.com/library/view/fundamentals-of-data/9781098108298/

🎥 Deeplearning.ai Data Engineering Certificate: https://www.coursera.org/professional-certificates/data-engineering

🔥Practical Data Modeling: https://practicaldatamodeling.substack.com/

🤓 My SubStack: https://joereis.substack.com/

I've seen a TON of horror stories with tech debt and code migrations. It's estimated that 15% to 60% of every dollar in IT spend goes toward tech debt (that's a big range, I know). Regardless, most of this tech debt will not be paid down without a radical change in how we do things. Might AI be the Hail Mary we need to pay down tech debt? I don't see why not...

My works:

📕Fundamentals of Data Engineering: https://www.oreilly.com/library/view/fundamentals-of-data/9781098108298/

🎥 Deeplearning.ai Data Engineering Certificate: https://www.coursera.org/professional-certificates/data-engineering

🔥Practical Data Modeling: https://practicaldatamodeling.substack.com/

🤓 My SubStack: https://joereis.substack.com/

For decades, siloed data modeling has been the norm: applications, analytics, and machine learning/AI. However, the emergence of AI, streaming data, and “shifting left" are changing data modeling, making siloed data approaches insufficient for the diverse world of data use cases. Today's practitioners must possess an end-to-end understanding of the myriad techniques for modeling data throughout the data lifecycle. This presentation covers "mixed model arts," which advocates converging various data modeling methods and the innovations of new ones.

Speaker: Joe Reis Author Nerd Herd

Read the blog to learn about the latest dbt Cloud features announced at Coalesce, designed to help organizations embrace analytics best practices at scale https://www.getdbt.com/blog/coalesce-2024-product-announcements

For decades, data modeling has been fragmented by use cases: applications, analytics, and machine learning/AI. This leads to data siloing and “throwing data over the wall.”

With the emergence of AI, streaming data, and “shifting left" are changing data modeling, these siloed approaches are insufficient for the diverse world of data use cases. Today's practitioners must possess an end-to-end understanding of the myriad techniques for modeling data throughout the data lifecycle. This presentation covers "mixed model arts," which advocates converging various data modeling methods and the innovations of new ones.