At the beginning of this year, Cityblock Health was afforded a unique opportunity: to rebuild their existing dbt project from scratch.

Launched in mid-2019, the legacy project had grown organically into a tangled mess of 1800+ models, with further development becoming more and more difficult.

Faced with the challenge of retroactively imposing order on the existing project, their leadership gave them the opportunity to start fresh instead.

They jumped at the chance, and began applying many of the lessons they learned at Coalesce 2022 to set the new project up for success: - SQL linting, with SQLFluff - YAML linting, with yamllint - dbt best practices, with dbt-checkpoint and dbt-project-evaluator

As a result, this core project has become the model for multi-project collaboration at Cityblock. Rather than a single monolithic project, the new state features a collection of smaller projects, each governed by a high bar for code quality.

Speakers: Katie Claiborne, Staff analytics engineer, Cityblock Health; Nathaniel Burren, Analytics Engineer, Cityblock Health

Register for Coalesce at https://coalesce.getdbt.com

Join Snowflake & Deloitte as they discuss how organizations can become exponential enterprise ready through the power of the Snowflake Data Cloud and dbt Cloud's ability to write, test, and ship reliable data in quick time. This session updates you on what your organization needs to do to become exponential enterprise ready. This session also shares examples of organizations that have already made the successful transformation and why they are winning in the market with dbt, Deloitte and Snowflake.

Speakers: Mathew Zele, Cloud & ISV Lead, Snowflake; Vivek Pradhan Lead Partner - Data and AI Platforms , Deloitte; Sagar Kulkarni Partner Sales Engineer, Snowflake

Register for Coalesce at https://coalesce.getdbt.com/

Lauren Benezra has been volunteering with a local cat rescue since 2018. She recently took on the challenge of rebuilding their data stack from scratch, replacing a Jenga tower of incomprehensible Google Sheets with a more reliable system backed by the Modern Data Stack. By using Airtable, Airbyte, BigQuery, dbt Cloud and Census, her role as Foster Coordinator has transformed: instead of digging for buried information while wrangling cats, she now serves up accurate data with ease while... well... wrangling cats.

Viewers will learn that it's possible to run an extremely scalable and reliable stack on a shoestring budget, and will come away with actionable steps to put Lauren's hard-won lessons into practice in their own volunteering projects or as the first data hire in a tiny startup.

Speakers: Lauren Benezra, Senior Analytics Engineer, dbt Labs

Register for Coalesce at https://coalesce.getdbt.com/

Bluecore works with the largest retail brands around the world to engage shoppers and keep them coming back. In this talk, you’ll learn how the team at Bluecore went about creating, scaling, and maturing an analytics data warehouse in BigQuery to orchestrate 10,000+ models every 30 minutes without bankrupting the company.

Speakers: Adam Whitaker, Analytics Lead, bluecore; Nicole Dallar-Malburg, Analytics Engineer, Bluecore

Register for Coalecse at https://coalesce.getdbt.com/

Does career growth feel like a black box? Does the question "where do you want to be in five years" fill you with existential dread? Data people come in all shapes and sizes, from backgrounds and disciplines, which can make it hard to see a clear growth path through the industry. In this session, Kasey Mazza at Hubspot explains how you can explore the whole ecosystem of data roles without feeling overwhelmed.

As the great Natasha Bedingfield once said,” the rest is still unwritten.” While a blank canvas ahead of you can be scary, it's also full of promise and opportunity. You'll leave this session with actionable next steps on how to be in control of your career growth and your destiny.

Speaker: Kasey Mazza, Manager, Analytics Engineering, HubSpot

Register for Coalesce at https://coalesce.getdbt.com/

Surfline uses Segment to collect product analytics events to understand how surfers use their forecasts and live surf cameras across 9000+ surf spots worldwide. An open source tool was developed to define and manage product analytics event schemas using JSON schema which are used to build dbt staging models for all events.

With this solution, the data team has more time to build intermediate and mart models in dbt, knowing that our staging layer fully reflects Surfline’s product analytics events. This presentation is a real-life example on how schemas (or data contracts) can be used as a medium to build consensus, enforce standards, improve data quality, and speed up the dbt workflow for product analytics.

Speaker: Greg Clunies, Senior Analytics Engineer, Surfline

Register for Coalesce at https://coalesce.getdbt.com/

Summary

The primary application of data has moved beyond analytics. With the broader audience comes the need to present data in a more approachable format. This has led to the broad adoption of data products being the delivery mechanism for information. In this episode Ranjith Raghunath shares his thoughts on how to build a strategy for the development, delivery, and evolution of data products.

Announcements

Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. You specify the customer traits, then Profiles runs the joins and computations for you to create complete customer profiles. Get all of the details and try the new product today at dataengineeringpodcast.com/rudderstack You shouldn't have to throw away the database to build with fast-changing data. You should be able to keep the familiarity of SQL and the proven architecture of cloud warehouses, but swap the decades-old batch computation model for an efficient incremental engine to get complex queries that are always up-to-date. With Materialize, you can! It’s the only true SQL streaming database built from the ground up to meet the needs of modern data products. Whether it’s real-time dashboarding and analytics, personalization and segmentation or automation and alerting, Materialize gives you the ability to work with fresh, correct, and scalable results — all in a familiar SQL interface. Go to dataengineeringpodcast.com/materialize today to get 2 weeks free! As more people start using AI for projects, two things are clear: It’s a rapidly advancing field, but it’s tough to navigate. How can you get the best results for your use case? Instead of being subjected to a bunch of buzzword bingo, hear directly from pioneers in the developer and data science space on how they use graph tech to build AI-powered apps. . Attend the dev and ML talks at NODES 2023, a free online conference on October 26 featuring some of the brightest minds in tech. Check out the agenda and register today at Neo4j.com/NODES. This episode is brought to you by Datafold – a testing automation platform for data engineers that finds data quality issues before the code and data are deployed to production. Datafold leverages data-diffing to compare production and development environments and column-level lineage to show you the exact impact of every code change on data, metrics, and BI tools, keeping your team productive and stakeholders happy. Datafold integrates with dbt, the modern data stack, and seamlessly plugs in your data CI for team-wide and automated testing. If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold Your host is Tobias Macey and today I'm interviewing Ranjith Raghunath about tactical elements of a data product strategy

Interview

Introduction How did you get involved in the area of data management? Can you describe what is encompassed by the idea of a data product strategy?

Which roles in an organization need to be involved in the planning and implementation of that strategy?

order of operations:

strategy -> platform design -> implementation/adoption platform implementation -> product strategy -> interface development

managing grain of data in products team organization to support product development/deployment customer communications - what questions to ask? requirements gathering, helping to understand "the art of the possible" What are the most interesting, innovative, or unexpected ways that you have seen organizations approach data product strategies? What are the most interesting, unexpected, or challenging lessons that you have learned while working on

Video highlights from Coalesce 2023 in San Diego. Coalesce is a conference for data practitioners, by data practitioners.

Coalesce is hosted by dbt Labs and crafted in collaboration with partners to advance the practice of the analytics engineering industry. Anyone that seeks new tips, tools, or techniques for more impactful data work is welcome to attend.

Sign up for our next Coalesce: https://coalesce.getdbt.com/

Mark, Cris and Marisa welcome colleague David Fieldhouse to Inside Economics to talk about the run up in long-term bond yields and mortgage rates, as well as the outlook for the consumer. The team discusses why the 10-year yield has breached 5% for the first time since before the GFC and what it might mean for the economy if rates stay higher for longer. They also consider the possibility of the yield curve reverting into positive territory soon. Mark tries to help Cris overcome his fears of a consumer-led recession and the team considers what a Phillies World Series win might mean for the economy.   Follow Mark Zandi @MarkZandi, Cris deRitis @MiddleWayEcon, and Marisa DiNatale on LinkedIn for additional insight.

Questions or Comments, please email us at [email protected]. We would love to hear from you.    To stay informed and follow the insights of Moody's Analytics economists, visit Economic View.

In this conversation with Tristan recorded at Coalesce 2023, Kasey Mazza, an analytics engineering manager on the RevOps team at HubSpot, discusses the roles of data analysts and analytics engineers, the importance of building internal data communities, and the evolving landscape of data teams.  Watch Kasey's Coalescse 2023 presentation The career growth software development lifecycle. For full show notes and to read 6+ years of back issues of the podcast's companion newsletter, head to https://roundup.getdbt.com.  The Analytics Engineering Podcast is sponsored by dbt Labs.

Uncover the transformative power of decentralising data with host Jason Foster and Nachiket Mehta, Head of Data and Analytics Engineering at Wayfair. They delve into Wayfair's journey from a centralised data approach to a decentralised model, the challenges they encountered, the value created, and how it has led to faster and more informed decision-making. Explore how Wayfair is utilising domain models, data contracts, and a platform-based approach to empower teams and enhance overall performance, and learn how decentralisation can revolutionise your organisation's data practices and drive business growth.

In this episode, Avery shows you how you can become a Data Analyst in just 90 days.

Join him as he debunks the myths, provides practical advice, and inspires you to achieve the seemingly impossible.

You got this. Tune in and level up your data career today!

🤝 Ace your data analyst interview with the interview simulator

📩 Get my weekly email with helpful data career tips

📊 Come to my next free “How to Land Your First Data Job” training

🏫 Check out my 10-week data analytics bootcamp

Timestamps:

(05:44) - Stop thinking small

(06:40) - Speed is key

(15:12) - Portfolio, experience, shadowing, learn from others

Connect with Avery:

📺 Subscribe on YouTube

🎙Listen to My Podcast

👔 Connect with me on LinkedIn

📸 Instagram

🎵 TikTok

Mentioned in this episode: Join the last cohort of 2025! The LAST cohort of The Data Analytics Accelerator for 2025 kicks off on Monday, December 8th and enrollment is officially open!

To celebrate the end of the year, we’re running a special End-of-Year Sale, where you’ll get: ✅ A discount on your enrollment 🎁 6 bonus gifts, including job listings, interview prep, AI tools + more

If your goal is to land a data job in 2026, this is your chance to get ahead of the competition and start strong.

👉 Join the December Cohort & Claim Your Bonuses: https://DataCareerJumpstart.com/daa https://www.datacareerjumpstart.com/daa

Seemingly straightforward data sets are seldom as simple as they initially appear. And, many an analysis has been tripped up by erroneous assumptions about either the data itself or about the business context in which that data exists. On this episode, Michael, Val, and Tim sat down with Viyaleta Apgar, Senior Manager of Analytics Solutions at Indeed.com, to discuss some antidotes to this very problem! Her structured approach to data discovery asks the analyst to outline what they know and don't know, as well as how any biases or assumptions might impact their results before they dive into Exploratory Data Analysis (EDA). To Viyaleta, this isn't just theory! She also shared stories of how she's put this into practice with her business partners (NOT her stakeholders!) at Indeed.com. For complete show notes, including links to items mentioned in this episode and a transcript of the show, visit the show page.

With the surge in big data and AI, organizations can rapidly create data products. However, the effectiveness of their analytics and machine learning models depends on the data's quality. Delta Lake's open source format offers a robust lakehouse framework over platforms like Amazon S3, ADLS, and GCS. This practical book shows data engineers, data scientists, and data analysts how to get Delta Lake and its features up and running. The ultimate goal of building data pipelines and applications is to gain insights from data. You'll understand how your storage solution choice determines the robustness and performance of the data pipeline, from raw data to insights. You'll learn how to: Use modern data management and data engineering techniques Understand how ACID transactions bring reliability to data lakes at scale Run streaming and batch jobs against your data lake concurrently Execute update, delete, and merge commands against your data lake Use time travel to roll back and examine previous data versions Build a streaming data quality pipeline following the medallion architecture

Throughout the past year, we've seen AI go from a nice-to-have, to a must-have in almost every large organization’s boardroom. There’s been more and more focus deploy AI  by leadership teams, and as a result, there's never been more pressure on the data team to deliver with AI. However, as the pressure to deliver with AI grows, the need to build safe and trustworthy experiences has also never been more important. But how do we balance between innovation and building these trustworthy experiences? How do you make responsible AI practical? Who should we get into the room when scoping safe AI use-cases?  Beena Ammanath is an award- winning senior technology executive with extensive experience in AI and digital transformation. Her career has spanned leadership roles in e-commerce, finance, marketing, telecom, retail, software products, service, and industrial domains. She is also the author of the ground breaking book, Trustworthy AI. Beena currently leads the Global Deloitte AI Institute and Trustworthy AI/ Ethical Technology at Deloitte. Prior to this, she was the CTO-AI at Hewlett Packard Enterprise. A champion for women and multicultural inclusion in technology and business, Beena founded Humans for AI, a 501c3b non-profit promoting diversity and inclusion in AI. Her work and contributions have been acknowledged with numerous awards and recognition such as 2016 Women Super Achiever Award from World Women’s Leadership Congress and induction into WITI’s 2017 Women in Technology Hall of Fame. Beena was honored by UC Berkeley as 2018 Woman of the Year for Business Analytics, by the San Francisco Business Times as one of the 2017 Most Influential Women in Bay Area and by the National Diversity Council as one of the Top 50 Multicultural Leaders in Tech. In the episode, Beena and Adel delve into the core principles of trustworthy AI, the interplay of ethics and AI in various industries, how to make trustworthy AI practical, who are the primary stakeholders for ensuring trustworthy AI, the importance of AI literacy when promoting responsible and trustworthy AI, and a lot more. Links mentioned in the Show Trustworthy AI by Beena AmmanathDeloitte AI InstituteHumans for AIData Literacy by Design, with Valerie Logan, CEO of the Data Lodge[Course] Implementing AI Solutions in Business[Webinar - October 19th 2023] Building a Capability Roadmap for AI

Summary

Building streaming applications has gotten substantially easier over the past several years. Despite this, it is still operationally challenging to deploy and maintain your own stream processing infrastructure. Decodable was built with a mission of eliminating all of the painful aspects of developing and deploying stream processing systems for engineering teams. In this episode Eric Sammer discusses why more companies are including real-time capabilities in their products and the ways that Decodable makes it faster and easier.

Announcements

Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. You specify the customer traits, then Profiles runs the joins and computations for you to create complete customer profiles. Get all of the details and try the new product today at dataengineeringpodcast.com/rudderstack This episode is brought to you by Datafold – a testing automation platform for data engineers that finds data quality issues before the code and data are deployed to production. Datafold leverages data-diffing to compare production and development environments and column-level lineage to show you the exact impact of every code change on data, metrics, and BI tools, keeping your team productive and stakeholders happy. Datafold integrates with dbt, the modern data stack, and seamlessly plugs in your data CI for team-wide and automated testing. If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold You shouldn't have to throw away the database to build with fast-changing data. You should be able to keep the familiarity of SQL and the proven architecture of cloud warehouses, but swap the decades-old batch computation model for an efficient incremental engine to get complex queries that are always up-to-date. With Materialize, you can! It’s the only true SQL streaming database built from the ground up to meet the needs of modern data products. Whether it’s real-time dashboarding and analytics, personalization and segmentation or automation and alerting, Materialize gives you the ability to work with fresh, correct, and scalable results — all in a familiar SQL interface. Go to dataengineeringpodcast.com/materialize today to get 2 weeks free! As more people start using AI for projects, two things are clear: It’s a rapidly advancing field, but it’s tough to navigate. How can you get the best results for your use case? Instead of being subjected to a bunch of buzzword bingo, hear directly from pioneers in the developer and data science space on how they use graph tech to build AI-powered apps. . Attend the dev and ML talks at NODES 2023, a free online conference on October 26 featuring some of the brightest minds in tech. Check out the agenda and register today at Neo4j.com/NODES. Your host is Tobias Macey and today I'm interviewing Eric Sammer about starting your stream processing journey with Decodable

Interview

Introduction How did you get involved in the area of data management? Can you describe what Decodable is and the story behind it?

What are the notable changes to the Decodable platform since we last spoke? (October 2021) What are the industry shifts that have influenced the product direction?

What are the problems that customers are trying to solve when they come to Decodable? When you launched your focus was on SQL transformations of streaming data. What was the process for adding full Java support in addition to SQL? What are the developer experience challenges that are particular to working with streaming data?

How have you worked to address that in the Decodable platform and interfaces?

As you evolve the technical and product direction, what is your heuristic for balancing the unification of interfaces and system integration against the ability to swap different components or interfaces as new technologies are introduced? What are the most interesting, innovative, or unexpected ways that you have seen Decodable used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Decodable? When is Decodable the wrong choice? What do you have planned for the future of Decodable?

Contact Info

esammer on GitHub LinkedIn

Parting Question

From your perspective, what is the biggest gap in the tooling or technology for data management today?

Closing Announcements

Thank you for listening! Don't forget to check out our other shows. Podcast.init covers the Python language, its community, and the innovative ways it is being used. The Machine Learning Podcast helps you go from idea to production with machine learning. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email [email protected]) with your story. To help other people find the show please leave a review on Apple Podcasts and tell your friends and co-workers

Links

Decodable

Podcast Episode

Understanding the Apache Flink Journey Flink

Podcast Episode

Debezium

Podcast Episode

Kafka Redpanda

Podcast Episode

Kinesis PostgreSQL

Podcast Episode

Snowflake

Podcast Episode

Databricks Startree Pinot

Podcast Episode

Rockset

Podcast Episode

Druid InfluxDB Samza Storm Pulsar

Podcast Episode

ksqlDB

Podcast Episode

dbt GitHub Actions Airbyte Singer Splunk Outbox Pattern

The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA Sponsored By: Neo4J:

NODES 2023 is a free online conference focused on graph-driven innovations with content for all skill levels. Its 24 hours are packed with 90 interactive technical sessions from top developers and data scientists across the world covering a broad range of topics and use cases. The event tracks: - Intelligent Applications: APIs, Libraries, and Frameworks – Tools and best practices for creating graph-powered applications and APIs with any software stack and programming language, including Java, Python, and JavaScript - Machine Learning and AI – How graph technology provides context for your data and enhances the accuracy of your AI and ML projects (e.g.: graph neural networks, responsible AI) - Visualization: Tools, Techniques, and Best Practices – Techniques and tools for exploring hidden and unknown patterns in your data and presenting complex relationships (knowledge graphs, ethical data practices, and data representation)

Don’t miss your chance to hear about the latest graph-powered implementations and best practices for free on October 26 at NODES 2023. Go to Neo4j.com/NODES today to see the full agenda and register!Rudderstack:

Introducing RudderStack Profiles. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. You specify the customer traits, then Profiles runs the joins and computations for you to create complete customer profiles. Get all of the details and try the new product today at dataengineeringpodcast.com/rudderstackMaterialize:

You shouldn't have to throw away the database to build with fast-changing data. Keep the familiar SQL, keep the proven architecture of cloud warehouses, but swap the decades-old batch computation model for an efficient incremental engine to get complex queries that are always up-to-date.

That is Materialize, the only true SQL streaming database built from the ground up to meet the needs of modern data products: Fresh, Correct, Scalable — all in a familiar SQL UI. Built on Timely Dataflow and Differential Dataflow, open source frameworks created by cofounder Frank McSherry at Microsoft Research, Materialize is trusted by data and engineering teams at Ramp, Pluralsight, Onward and more to build real-time data products without the cost, complexity, and development time of stream processing.

Go to materialize.com today and get 2 weeks free!Datafold:

This episode is brought to you by Datafold – a testing automation platform for data engineers that finds data quality issues before the code and data are deployed to production. Datafold leverages data-diffing to compare…

Inflation was front and center in this week’s podcast. Mark and Marisa (yes, she’s back and winning the stats game again) hosted a wonderful cast of colleagues to talk over the September CPI report, the European inflation experience, which is similar to that here in the U.S., and given recent grim events in the Middle East, energy prices. The bottom line is inflation continues to head in the right direction, but not in a straight line.   Follow Mark Zandi @MarkZandi, Cris deRitis @MiddleWayEcon, and Marisa DiNatale on LinkedIn for additional insight.

Questions or Comments, please email us at [email protected]. We would love to hear from you.    To stay informed and follow the insights of Moody's Analytics economists, visit Economic View.

All cloud architects need to know how to build data platforms that enable businesses to make data-driven decisions and deliver enterprise-wide intelligence in a fast and efficient way. This handbook shows you how to design, build, and modernize cloud native data and machine learning platforms using AWS, Azure, Google Cloud, and multicloud tools like Snowflake and Databricks. Authors Marco Tranquillin, Valliappa Lakshmanan, and Firat Tekiner cover the entire data lifecycle from ingestion to activation in a cloud environment using real-world enterprise architectures. You'll learn how to transform, secure, and modernize familiar solutions like data warehouses and data lakes, and you'll be able to leverage recent AI/ML patterns to get accurate and quicker insights to drive competitive advantage. You'll learn how to: Design a modern and secure cloud native or hybrid data analytics and machine learning platform Accelerate data-led innovation by consolidating enterprise data in a governed, scalable, and resilient data platform Democratize access to enterprise data and govern how business teams extract insights and build AI/ML capabilities Enable your business to make decisions in real time using streaming pipelines Build an MLOps platform to move to a predictive and prescriptive analytics approach

Join Avery Smith and Jordan Temple in this episode of the Data Career Podcast.

Jordan shares his insights on breaking into the data industry with a non-technical background, leveraging skills in Power BI and the power of a solid online presence.

Tune in now to gain inspiration and guidance for your data career journey.

Connect with Jordan Temple:

🤝 Connect on Linkedin

🤝 Ace your data analyst interview with the interview simulator

📩 Get my weekly email with helpful data career tips

📊 Come to my next free “How to Land Your First Data Job” training

🏫 Check out my 10-week data analytics bootcamp

Timestamps: (07:02) - Excel + Power BI

(12:20) - Landing a job with a DM

(15:10) - Hybrid > Remote

Connect with Avery:

📺 Subscribe on YouTube

🎙Listen to My Podcast

👔 Connect with me on LinkedIn

📸 Instagram

🎵 TikTok

Mentioned in this episode: Join the last cohort of 2025! The LAST cohort of The Data Analytics Accelerator for 2025 kicks off on Monday, December 8th and enrollment is officially open!

To celebrate the end of the year, we’re running a special End-of-Year Sale, where you’ll get: ✅ A discount on your enrollment 🎁 6 bonus gifts, including job listings, interview prep, AI tools + more

If your goal is to land a data job in 2026, this is your chance to get ahead of the competition and start strong.

👉 Join the December Cohort & Claim Your Bonuses: https://DataCareerJumpstart.com/daa https://www.datacareerjumpstart.com/daa

Send us a text Part 2 : Sam Torres is the Chief Digital Officer and co-founder of The Gray Dot Company,  Gray Dot Company is a consulting firm that specializes in search engine optimization.  Sam outlines expertise in complex digital analytics and consumer insights data. 00:39 The Gray Dot Company06:44 Using Data10:18 Website Design10:18 Website Design13:08 Ethical Marketing14:53 Biggest Marketing SuprisesLinkedIn: linkedin.com/in/samantha-torres-seo Website: https://thegray.company, https://legendarypodcasts.com/sam-torres/ Want to be featured as a guest on Making Data Simple?  Reach out to us at [email protected] and tell us why you should be next.  The Making Data Simple Podcast is hosted by Al Martin, WW VP Technical Sales, IBM, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun.  Want to be featured as a guest on Making Data Simple? Reach out to us at [email protected] and tell us why you should be next. The Making Data Simple Podcast is hosted by Al Martin, WW VP Technical Sales, IBM, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun.