talk-data.com
Activities & events
| Title & Speakers | Event |
|---|---|
|
How to use CI/CD pipelines for dbt and deploy dbt Cloud using Infrastructure as Code + sneak peak into dbt Fusion
2025-07-01 · 19:15
Koen Graat
– Principal Data Engineer
@ Xebia Data
Are you tired of manual and failing dbt deployments? This talk explores how CI/CD and IaC can revolutionize your data transformation workflows, enhancing collaboration and data quality within your dbt projects. Learn the core concepts of CI/CD, including automated testing and deployment pipelines, I will guide you through building a CI/CD pipeline for dbt, triggering it with code changes and running comprehensive tests. Next to that we will dive into Infrastructure as Code (IaC) and how it automates dbt Cloud deployments using tools like Terraform. You will gain practical knowledge for automating dbt Cloud resources, projects, and environments. As a bonus we will do a sneak peek into the recently announced dbt Fusion engine. |
|
|
Armand Duijen
– Data Engineer
@ Studyportals
,
Katie Scheitzer
– Senior Analytics Engineer
@ Studyportals
We strive for our dbt project to be ready by 9am for our stakeholders. Should be easy, right? Except that our dbt project consists of around 450 dbt models and over 30 sources. Some of those sources are ready as early as midnight but some as late as 4am, and in total our project takes around 4 hours to run. Join as us we walk through the evolution of our dbt run setup, from one selector, to a set of parallel commands, to today's setup -- a dynamic lineage in Airflow which runs models when and only when the upstream source is ready. It's finished when the Tableau datasource is refreshed and our stakeholders can start their day with the latest data. |
|