It's Friday! Matt Housley and I catch up to discuss the aftermath of AWS re:Invent and why the industry’s obsession with AI Agents might be premature. We also dive deep into the hardware wars between Google and NVIDIA , the "brain-damaged" nature of current LLMs , and the growing "enshittification" of the internet and platforms like LinkedIn. Plus, I reveals some details about my upcoming "Mixed Model Arts" project.

Matt Housley joins me for our monthly round-up of topics. This time, there's danger everywhere - The AI Bubble, how vibe coding is evolving, AI slop, and more.

It's all about acquisitions, acquisitions, acquisitions! Matt Housley joins me to tackle the biggest rumor in the data world this week: the potential acquisition of dbt Labs by Fivetran. This news sparks a wide-ranging discussion on the inevitable consolidation of the Modern Data Stack, a trend we predicted as the era of zero-interest-rate policy ended. We also talk about financial pressures, vendor exposure to the rise of AI, the future of data tooling, and more.

Matt Housley joins me for our monthly chat. This time, we discuss my article The Pedantic Layer, fraud, and more.

Matt Housley joins me to chat about whether it matters that AI is PhD level, clanker content (the new term for AI slop), a retrospective on Fundamentals of Data Engineering, and much more.

Tired of spending money on data courses you never finish? Here are 7 essential books that will actually boost your analytical skills, with no subscription required! Plus, make sure to tune in till the end as one lucky listener will get a free book from this list! Get the books here! DISCLAIMER: Some of the links in this video are affiliate links, meaning if you click through and make a purchase, I may earn a commission at no extra cost to you. Storytelling with Data by Cole Nussbaumer Knaflic 👉 https://amzn.to/3ZYHhsG Ace the Data Science Interview by Nick Singh and Kevin Huo 👉 https://amzn.to/3XZ9IaB Moneyball by Michael Lewis 👉 https://amzn.to/44fy4OD The StatQuest Illustrated Guide To Machine Learning by Josh Starmer 👉 https://amzn.to/40hRgu2 Fundamentals of Data Engineering by Joe Reis and Matt Housley 👉 https://amzn.to/3W84K8K Data Science for Business by Foster Provost and Tom Fawcett 👉 https://amzn.to/4k7jkaD The Big Book of Dashboards by Steve Wexler, Jeffrey Shaffer, and Andy Cotgreave 👉 https://amzn.to/462GJVj 💌 Join 10k+ aspiring data analysts & get my tips in your inbox weekly 👉 https://www.datacareerjumpstart.com/newsletter 🆘 Feeling stuck in your data journey? Come to my next free "How to Land Your First Data Job" training 👉 https://www.datacareerjumpstart.com/training 👩‍💻 Want to land a data job in less than 90 days? 👉 https://www.datacareerjumpstart.com/daa 👔 Ace The Interview with Confidence 👉 https://www.datacareerjumpstart.com/interviewsimulator ⌚ TIMESTAMPS 00:16 Book 1: The Big Book of Dashboards 02:52 Book 2: Data Science for Business 04:38 Book 3: Fundamentals of Data Engineering 06:05 Book 4: The StatQuest Illustrated Guide To Machine Learning 07:52 Book 5: Moneyball 10:09 Book 6: Ace the Data Science Interview 11:24 Book 7: Storytelling With Data I've interviewed some of these awesome data authors! Check out these episodes! Stats You Need to Know as a Data Analyst (w/ StatQuest) 👉 https://datacareerpodcast.com/episode/105-do-you-have-to-be-good-at-statistics-to-be-a-data-analyst-w-statquest-josh-starmer-phd How to Ace The Data Science & Analytics Interview w/ Nick Singh 👉 https://datacareerpodcast.com/episode/74-how-to-ace-the-data-science-analytics-interview-w-nick-singh Meet The Woman Who Changed Data Storytelling Forever (Cole Knaflic) 👉 https://datacareerpodcast.com/episode/142-meet-the-woman-who-changed-data-storytelling-forever-cole-knafflic

🔗 CONNECT WITH AVERY 🎥 YouTube Channel: https://www.youtube.com/@averysmith 🤝 LinkedIn: https://www.linkedin.com/in/averyjsmith/ 📸 Instagram: https://instagram.com/datacareerjumpstart 🎵 TikTok: https://www.tiktok.com/@verydata 💻 Website: https://www.datacareerjumpstart.com/ Mentioned in this episode: Join the last cohort of 2025! The LAST cohort of The Data Analytics Accelerator for 2025 kicks off on Monday, December 8th and enrollment is officially open!

To celebrate the end of the year, we’re running a special End-of-Year Sale, where you’ll get: ✅ A discount on your enrollment 🎁 6 bonus gifts, including job listings, interview prep, AI tools + more

If your goal is to land a data job in 2026, this is your chance to get ahead of the competition and start strong.

👉 Join the December Cohort & Claim Your Bonuses: https://DataCareerJumpstart.com/daa https://www.datacareerjumpstart.com/daa

Tony Baer, Matt Housley, and Juan Sequeda and I recap our thoughts on Data Day Texas 2025.

Matt Housley and I have a LONG chat about working in consulting, leaving your job, AI, the job market, our thoughts on what's coming in 2025, and much more.

❤️ If you like my podcasts, please like and rate it on your favorite podcast platform.

🤓 My works:

📕Fundamentals of Data Engineering: https://www.oreilly.com/library/view/fundamentals-of-data/9781098108298/

🎥 Deeplearning.ai Data Engineering Certificate: https://www.coursera.org/professional-certificates/data-engineering

🔥Practical Data Modeling: https://practicaldatamodeling.substack.com/

🤓 My SubStack: https://joereis.substack.com/

The Data Engineer's role amidst the rise of big data, cloud computing, and AI-driven analytics has shifted. This panel chat explores the ever-changing landscape of essential skills and the automation of outdated ones. With a myriad of architectural options available, we'll dissect how organizations navigate the complexities to tailor solutions to their specific needs. Let's unravel the intricacies of building scalable data systems, pinpointing common breakpoints and strategies for efficient scaling. Come along as we delve into in constructing the foundation of the data-driven future.

Matt Housley hangs out at my house, and we have a random chat about all sorts of stuff - fads in data, data and ML engineering, tech hubs, and more. If you want a glimpse into the sorts of chats that Matt and I have all the time, here you go.

As Joe Reis recently opined, if you want to know what’s next in data engineering, just look at the software engineer. The MDS-in-a-box pattern has been a game changer for applying software engineering principles to local data development– improving the ability to share data, collaborate on modeling work and data analysis the same way we build and share open source tooling.

This panel brings together experts in data engineering, data analytics and software engineering to explore the current state of the pattern, pieces that remain missing today and how emerging tools and data engineering testing capabilities can refine the transition from local development to production workflows.

Speakers: Matt Housley, CTO, Halfpipe Systems; Mehdi Ouazza, Developer Advocate, MotherDuck; Sung Won Chung, Solutions Engineer, Datafold; Louise de Leyritz, Host, The Data Couch podcast

Register for Coalesce at https://coalesce.getdbt.com

Matt Housley hangs out at my house and we chat about Data NIMBYism and gatekeeping. What is that? Listen and find out.

ABOUT THE TALK: Hybridization approaches such as Lambda and Kappa architecture are powerful tools for combining the most useful characteristics of batch and real time systems. Implementation and management of these architectures is not for the faint of heart, but the last several years have seen a wave of SaaS platforms and managed services that deliver hybrid capabilities with a greatly reduced operational burden. This talk details the anticipated impact of these hybrid technologies on future data stacks, and touches on the mythical “one database to rule them all.”

ABOUT THE SPEAKER: Matt Housley holds a Ph.D. in mathematics and is co-author of the bestselling O’Reilly book Fundamentals of Data Engineering.

ABOUT DATA COUNCIL: Data Council (https://www.datacouncil.ai/) is a community and conference series that provides data professionals with the learning and networking opportunities they need to grow their careers.

Make sure to subscribe to our channel for the most up-to-date talks from technical professionals on data related topics including data infrastructure, data engineering, ML systems, analytics and AI from top startups and tech companies.

FOLLOW DATA COUNCIL: Twitter: https://twitter.com/DataCouncilAI LinkedIn: https://www.linkedin.com/company/datacouncil-ai/

Summary Data engineering is a difficult job, requiring a large number of skills that often don’t overlap. Any effort to understand how to start a career in the role has required stitching together information from a multitude of resources that might not all agree with each other. In order to provide a single reference for anyone tasked with data engineering responsibilities Joe Reis and Matt Housley took it upon themselves to write the book "Fundamentals of Data Engineering". In this episode they share their experiences researching and distilling the lessons that will be useful to data engineers now and into the future, without being tied to any specific technologies that may fade from fashion.

Announcements

Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. Go to dataengineeringpodcast.com/linode today and get a $100 credit to launch a database, create a Kubernetes cluster, or take advantage of all of their other services. And don’t forget to thank them for their continued support of this show! Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Push information about data freshness and quality to your business intelligence, automatically scale up and down your warehouse based on usage patterns, and let the bots answer those questions in Slack so that the humans can focus on delivering real value. Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. Prefect is the modern Dataflow Automation platform for the modern data stack, empowering data practitioners to build, run and monitor robust pipelines at scale. Guided by the principle that the orchestrator shouldn’t get in your way, Prefect is the only tool of its kind to offer the flexibility to write code as workflows. Prefect specializes in glueing together the disparate pieces of a pipeline, and integrating with modern distributed compute libraries to bring power where you need it, when you need it. Trusted by thousands of organizations and supported by over 20,000 community members, Prefect powers over 100MM business critical tasks a month. For more information on Prefect, visit dataengineeringpodcast.com/prefect today. Your host is Tobias Macey and today I’m interviewing Joe Reis and Matt Housley about their new book on the Fundamentals of Data Engineering

Interview

Introduction How did you get involved in the area of data management? Can you explain what possessed you to write such an ambitious book? What are your goals with this book? What was your process for determining what subject areas to include in the book?

How did you determine what level of granularity/detail to use for each subject area?

Closely linked to what subjects are necessary to be effective as a data engineer is the concept of what that title encompasses. How have the definitions shifted over the past few decades?

In your experiences working in industry and researching for the book, what is the prevailing view on what data engineers do? In the book you focus on what you term the "data lifecycle engineer". What are the skills and background that are needed to be successful in that role?

Any discussion of technological concepts and how to build systems tends to drift toward specific tools. How did you balance the need to be agnostic to speci

Data engineering has grown rapidly in the past decade, leaving many software engineers, data scientists, and analysts looking for a comprehensive view of this practice. With this practical book, you'll learn how to plan and build systems to serve the needs of your organization and customers by evaluating the best technologies available through the framework of the data engineering lifecycle. Authors Joe Reis and Matt Housley walk you through the data engineering lifecycle and show you how to stitch together a variety of cloud technologies to serve the needs of downstream data consumers. You'll understand how to apply the concepts of data generation, ingestion, orchestration, transformation, storage, and governance that are critical in any data environment regardless of the underlying technology. This book will help you: Get a concise overview of the entire data engineering landscape Assess data engineering problems using an end-to-end framework of best practices Cut through marketing hype when choosing data technologies, architecture, and processes Use the data engineering lifecycle to design and build a robust architecture Incorporate data governance and security across the data engineering lifecycle