ABOUT THE TALK: After two years, three rounds of funding, and hundreds of new employees — Whatnot’s modern data stack has come from not existing to processing tens of millions of events across hundreds of different event types each day.

How does their small (but mighty!) team keep up? This talk explores data contracts — it covers the use of Interface Definition Language (Protobuf) to serve as the source of truth for event definitions, govern event construction in production, automatically generate DBT models in the data warehouse.

ABOUT THE SPEAKER: Zack Klein is a software engineer at Whatnot, where he thoroughly enjoys building data products and narrowly avoiding breaking production each day. Previously, he worked on big data platforms at Blackstone and HBO.

ABOUT DATA COUNCIL: Data Council (https://www.datacouncil.ai/) is a community and conference series that provides data professionals with the learning and networking opportunities they need to grow their careers.

Make sure to subscribe to our channel for the most up-to-date talks from technical professionals on data related topics including data infrastructure, data engineering, ML systems, analytics and AI from top startups and tech companies.

FOLLOW DATA COUNCIL: Twitter: https://twitter.com/DataCouncilAI LinkedIn: https://www.linkedin.com/company/datacouncil-ai/

ABOUT THE TALK: Ever noticed the manifest.json file that dbt puts into your target folder? This little file contains rich information about your dbt project that enables numerous fun use cases! These include complex deployment configurations, quality enforcement, and streamlined development workflows. This talk will go over what the manifest is and how it is produced, along with case studies of how the manifest is used across the community and in Squarespace’s data pipelines.

ABOUT THE SPEAKER: Aaron Richter is a software developer with a passion for all things data. His work involves making sure data is clean and accessible, and that the tools to access it are at peak performance. Aaron is currently a data engineer at Squarespace, where he supports the company’s analytics platform. Previously, he built the data warehouse at Modernizing Medicine, and worked as a data science advocate at Saturn Cloud.

ABOUT DATA COUNCIL: Data Council (https://www.datacouncil.ai/) is a community and conference series that provides data professionals with the learning and networking opportunities they need to grow their careers.

Make sure to subscribe to our channel for the most up-to-date talks from technical professionals on data related topics including data infrastructure, data engineering, ML systems, analytics and AI from top startups and tech companies.

FOLLOW DATA COUNCIL: Twitter: https://twitter.com/DataCouncilAI LinkedIn: https://www.linkedin.com/company/datacouncil-ai/

ABOUT THE TALK: If you squint, the data warehouse-centric analytics stack can look a lot like mainframe-era centralized control: shared distributed compute, some sandbox space if you are lucky, and no way to work locally that's equivalent. And it's been getting expensive! Yet the major vendors appear to be converging on features. Is this "The End of History" for OLAP? Your laptop is incredibly powerful and tragically underutilized. It has better I/O than servers from just a few years back, and with a solid analytic engine like DuckDB, it can handle surprising workloads.

In this talk Nicholas Ursa will share what makes DuckDB fast and show how far single node scale-up can take you. He will also show some ideas Motherduck is testing that blur the line between local and remote systems and shift some of that centralized control back to users.

ABOUT THIS SPEAKER: Nicholas Ursa is an Engineer at Motherduck. He led data engineering teams at The New York Times and better.com after cutting his teeth in ad tech.

ABOUT DATA COUNCIL: Data Council (https://www.datacouncil.ai/) is a community and conference series that provides data professionals with the learning and networking opportunities they need to grow their careers.

Make sure to subscribe to our channel for the most up-to-date talks from technical professionals on data related topics including data infrastructure, data engineering, ML systems, analytics and AI from top startups and tech companies.

FOLLOW DATA COUNCIL: Twitter: https://twitter.com/DataCouncilAI LinkedIn: https://www.linkedin.com/company/datacouncil-ai/

ABOUT THE TALK Learn all about cost and performance optimization in Snowflake. This talk deep dive's into Snowflake’s architecture & billing model, covering key concepts like virtual warehouses, micro-partitioning, the lifecycle of a query and Snowflake’s two-tiered cache. It then goes in depth on the most important optimization strategies, like virtual warehouse configuration, table clustering and query writing best practices. Throughout the talk, code snippets and other resources are shared to help you get the most out of Snowflake.

ABOUT THE SPEAKERS Niall Woodward and Ian Whitestone are the co-founders at SELECT, a tool to help Snowflake users optimize their Snowflake cost & performance.

Niall Woodward has been well known in the data community for creating and contributing to open source packages.

Ian Whitestone previously led data teams at Shopify and Capital One. At Shopify, Ian spearheaded the efforts to reduce their data warehouse spend by over 50%.

ABOUT DATA COUNCIL: Data Council (https://www.datacouncil.ai/) is a community and conference series that provides data professionals with the learning and networking opportunities they need to grow their careers.

Make sure to subscribe to our channel for the most up-to-date talks from technical professionals on data related topics including data infrastructure, data engineering, ML systems, analytics and AI from top startups and tech companies.

FOLLOW DATA COUNCIL: Twitter: https://twitter.com/DataCouncilAI LinkedIn: https://www.linkedin.com/company/datacouncil-ai