Unlike transforming staged data to marts, ingesting data into staging requires robustness to data volume and type changes, schema evolution, and data drift. Especially when performing change data capture (CDC) on large databases (~100 tables to a database), we’ll ideally reinforce our dbt models with automatic:

• Mapping of dynamic columns and data types between the source and the target stag
• evolution of stage table schemas at pace with incoming data, including for nested data structures
• parsing and flattening of any arrays and JSON structs in the data.

Manually performing these tasks for data at scale is a tall order due to the many permutations with which CDC data can deviate. Waiting to implement them in mart transformation models is potentially detrimental to the business, as well as doesn’t reduce the complexity. Santona Tuli shares learnings from integrating dbt Core into high-scale data ingestion workloads, including trade-offs between ease-of-use and scale.

Speaker: Santona Tuli, Head/Director of Data, Upsolver

Register for Coalesce at https://coalesce.getdbt.com

Join us for story hour, as we follow two intrepid analytics engineers working in a large dbt project as they go on a journey to meshify their dbt project, with help from a ✨special guest✨

Along the way, learn about dbt-meshify - a new CLI tool to automate the creation of model governance and cross-project lineage features in your dbt project. dbt-meshify refactors your code for you, helping you add model contracts, versions, groups, access, cross-project lineage, and more -- all in a matter of minutes! No bespoke YAML writing needed.

Speakers: Grace Goheen, Product Manager, dbt Labs; Nicholas Yager, Principal Analytics Engineer, HubSpot; Dave Connors, DX, dbt Labs

Register for Coalesce at https://coalesce.getdbt.com

The Guild team migrated to Snowflake and dbt for their data warehousing needs and immediately saw the benefits of standardizing model structure, DRYer logic, data lineage ,and automated testing on Pull Requests.

But leveraging dbt didn’t solve everything. Pain points around maintaining model logic, handling historical data, and integrating data from over 60 source systems meant that analysts still struggled to provide a unified view of the business. The team knew that they needed to level up their processes and modeling again, and chose to adopt Data Vault (DV).

Brandon and Rebecca take you behind the scenes of this decision to explain the benefits of Data Vault. They highlight DV’s ability to handle complex data integration requirements while remaining agile and demonstrate that it complements other modern data concepts like domain-driven design and data mesh.

Attendees learn what Data Vault is, when it can be a key component of a successful data strategy, and instances where it’s not the right fit. Walk away with practical tips to successfully transition based on a real-world implementation.

Guild transformed their data warehouse; you can too!

Speakers: Brandon Taylor, Senior Data Architect, Guild; Rebecca Di Bari Staff Data Engineer , Guild

Register for Coalesce at https://coalesce.getdbt.com

In the dbt Labs on dbt series, you get a behind-the-scenes look at how dbt Labs uses data. You’ll learn how dbt Labs thinks about the role of data, how data developers collaborate with business leaders, and the technical decisions we’ve made in our own dbt project.

In this session, Brandon Thomson, Analytics Lead at dbt Labs, digs deeper into the technical details of the Campaign 360, a powerful marketing analytics asset used by every member of the marketing team. You'll learn about the technical decisions made during the build of this product and explore the finished asset.

Join this session to learn about dbt Labs' journey and leave with ideas that you can implement in your dbt project today.

Speaker: Brandon Thomson, Analytics Lead, dbt Labs

Register for Coalesce at https://coalesce.getdbt.com

The final session of Coalesce 2023! Join us for Community Awards and closing thoughts. We recognize 10 individuals who have contributed significantly to the success of the dbt Community. Some of them will be with us in person, and some others will be tuning in online. All Community Award recipients are recognized and presented with a unique, never-before-seen swag item!

After the awards, we end with closing thoughts and announce the location for Coalesce 2024!

Speakers: Amada Echeverría, Global Community Lead, dbt Labs; Tristan Handy, CEO, dbt Labs

Register for Coalesce at https://coalesce.getdbt.com/

With increased adoption of dbt and ever growing data volumes, data teams are increasingly looking to make their dbt deployments more efficient. The problem? Practitioners aren’t equipped with the tools and strategies to do this well. In this talk, Ian Whitestone and Niall Woodward of SELECT share a variety of dbt cost and performance optimization best practices, diving into the actual strategies they’ve deployed with numerous companies to give practitioners immediately actionable techniques they can apply to their own projects.

Speakers: Ian Whitestone, Co-founder, SELECT; Niall Woodward, Co-Founder, SELECT

Register for Coalesce at https://coalesce.getdbt.com

In this session, you'll learn tips and tricks from La-Z-Boy on how to modernize infrastructure and tackle complex retail and manufacturing data problems using dbt and Fivetran. Hear from a data leader about their experience and how you can replicate their success. Start your session off with some comfortable data movements to make sure you can extract as much information as possible from our loaded session on data transformation.

Speakers: Alex Hauer, Lead Product Marketing Manager, Fivetran; Selwyn Samuel, Director of Data Analytics & Enterprise Architecture, La-Z-Boy

Register for Coalesce at https://coalesce.getdbt.com

In the dbt Labs on dbt series, you get a behind-the-scenes look at how dbt Labs uses data. You’ll learn how dbt Labs thinks about the role of data, how data developers collaborate with business leaders, and the technical decisions we’ve made in our own dbt project.

In this session, Elize Papineau, Senior Data Engineer at dbt Labs, digs deeper into the technical details of the cost optimization project at dbt Labs. You'll learn how the team leveraged query tags in dbt to make model performance monitoring possible, the process for analyzing model performance, the implementation of warehouse specific configurations at the model level, and how the team measures the effectiveness of optimizations and translates it into cost savings.

Watch to learn about dbt Labs' journey and leave with ideas that you can implement in your dbt project today.

Speaker: Elize Papineau, Sr. Data Engineer, dbt Labs

Register for Coalesce at https://coalesce.getdbt.com

Two public servants at the City of Boston and the State of California are tasked with improving data services and data engineering practices within their respective governments. As part of the modernization process, they are adopting dbt and associated tools within their respective teams.

This session discusses the similarities and differences between the implementations of dbt, and how some of the constraints and challenges of working in government shape both the technical and social design of data services. The speakers will reflect on successes, challenges, and lessons learned about adopting modern data tooling in state and local governments.

Speakers: Jenna Jordan, Data Engineer, City of Boston; Ian Rose, Senior Data Engineer, California Office of Data and Innovation; Laurie Merrell, Senior Analytics Engineer, Jarvus Innovations

This session addresses the problem of how to leverage dbt Cloud to support domain user development for a migration from a centralized analytics environment towards a distributed data mesh analytics environment.

Speaker: Holly Burch, Data Architect, Sharp HealthCare

Register for Coalesce at https://coalesce.getdbt.com

Prioritizing the right sales opportunities is pivotal for any SaaS company's growth, but what happens after your initial success? Jessica Zhang, Data Science Manager at Notion, traces Notion's footsteps from its foundational days to its present-day lead scoring techniques. Learn how modern tools like dbt, Census, and Snowflake enable the Notion team to iterate quickly. More than a journey, this session is a lesson on evolving a data science model in response to changing business assumptions and fresh user insights.

Speakers: Jessica Zhang, Data Science Manager, Notion; Jeff Sloan, Sr. Data Community Advocate, Census

Register for Coalesce at https://coalesce.getdbt.com

In this session, Jeremy Cohen, product manager at dbt Labs, does an in-depth walk-through of the new dbt Cloud releases shared on-stage during the Keynote & Product Spotlight. This session focuses on changes to the dbt Cloud IDE and new ways to develop with dbt.

Wherever you write code, see the future of dbt development in action.

Speakers: Jeremy Cohen, Product Manager, dbt Labs; Greg McKeon, Product Manager, dbt Labs

Register for Coalesce at https://coalesce.getdbt.com

In the rapidly evolving landscape of data technology, the integration of Snowflake and dbt has revolutionized the creation and management of data applications. Now, developers can harness their combined capabilities to build superior, scalable, and sophisticated data applications.

With Snowflake’s cloud-based architecture, developers can access boundless storage, computing, and seamless data sharing. Additionally, Snowpark Python enables the performance of data transformation, analytics, and algorithmic functions within Snowflake, presenting developers with a new realm of opportunities. Incorporating dbt further enhances the synergy, allowing developers to streamline data workflows in an agile, model-driven environment.

This session covers how the Snowflake and dbt partnership can pave the way toward building better, future-proof data applications that cater to the dynamic needs of businesses in the digital era.

Speaker: Tarik Dwiek, Head of Technology and Application Partners, Snowflake

Register for Coalesce at https://coalesce.getdbt.com

Now that you can write models in Python, a new world of possibility has opened up. In this session, Christopher Marland introduces you to Snowpark and how it integrates with dbt, before demonstrating a real-world use case where Python transformations outperform SQL, starting from raw data and moving through to a completed analysis.

This talk is ideal for people who are familiar with PySpark but new to dbt, or who are experienced dbt users and curious about taking advantage of their new Pythonic superpowers from inside of a familiar development environment.

Speaker: Christopher Marland, Snowflake Solutions Architect, Aimpoint Digital

Register for Coalesce at https://coalesce.getdbt.com

Active metadata is the thread that helps weave your data mesh. In this session, Austin Kronz, Director of Data Strategy at Atlan and Sean Rober, Head of Data at Avenue One, discuss how Atlan and dbt are central to how Avenue One enhances their data and analytics ecosystem, brings their product to market, and positions an already fast-paced start-up for scale.

Speakers: Austin Kronz, Director of Data Strategy, Atlan; Sean Rober, Head of Data, Avenue One

Register for Coalesce at https://coalesce.getdbt.com

Data contracts have been much discussed in the community of late, with a lot of curiosity around how to approach this concept in practice and how it might enable shift-left developer-first governance and data quality. For organizations adopting dbt while also dealing with non-dbt data that is upstream of the warehouse, it can be challenging to understand how to apply data contracts uniformly across a fragmented stack. We are calling this harmonizing layer the Control Plane for Data - powered by the common thread across these systems: metadata.

In this talk, Shirshanka Das, CTO of Acryl Data and founder of the DataHub Project describes how you can use data contracts and DataHub to make your dbt centered stack more reliable - as well as other use cases that can help build a simpler, more flexible data stack.

Speaker: Shirshanka Das, CTO, Acryl Data

Register for Coalesce at https://coalesce.getdbt.com

With the dbt Semantic Layer, you can define metrics alongside your dbt models, and access them from any integrated analytics tool.

The Semantic Layer has been totally revamped. It’s now powered by MetricFlow, allowing for more complex metric definition and a significantly improved querying experience. Join the dbt Labs product team as they dive into what’s new, what’s different, and what it means for you.

Speakers: Nick Handel, Director, Product Management, dbt Labs; Roxi Pourzand, Product Manager, dbt Labs

Register for Coalesce at https://coalesce.getdbt.com

Learn how ClickUp uses dbt, dbt packages, and Snowflake to save on storage and compute costs using Slim CI and how they empower a data warehouse centric culture across Sales, Marketing, Product Growth, Finance, and RevOps all while maintaining one monolithic dbt build job.

Speaker: Michael Revelo, Data Platform Lead , ClickUp

Register for Coalesce at https://coalesce.getdbt.com

The Fivetran Analytics Engineering team strives to make its dbt packages multi-faceted enough and functional for the majority of data teams, balancing flexibility against ease of implementation. This means that the packages should get you 80% of the way there out of the box, but you're not alone as you cover the final 20%!

Over the years, the team has further developed its understanding of the nuances of analyses and developed different methods for folks to easily tweak packages to their liking. This session will discuss passthrough columns, union macros, and overriding package models, and will teach you how to use these features to make Fivetran's packages work for you or even leverage the same patterns in your own work.

Speaker: Jamie Rodriguez, Senior Analytics Engineer, Fivetran

Register for Coalesce at https://coalesce.getdbt.com

A significant proportion of dbt Cloud users do not have a dbt CI job set up. Among those who do, many don’t leverage powerful functionality like state comparison and deferral to implement Slim CI, likely causing teams to miss errors and building unnecessary tables. Setting up Slim CI in dbt Cloud can be especially challenging for larger-scale data organizations who have multiple data environments, git branches, and targets. Watch this session to learn how you can build and evolve a strong, lasting data environment using Slim CI.

Speakers: Leo Folsom, Solutions Engineer, Datafold

Register for Coalesce at https://coalesce.getdbt.com