Discover the transformation of Airflow at GoDaddy: from its initial deployment on-prem to its migration to the cloud, and finally to a Single Pane Orchestration Model. This evolution has streamlined our Data Platform and improved governance. Our experience will be beneficial for anyone seeking to optimize their Airflow implementation and simplify their orchestration processes. History and Use-cases Design, Organization decisions, and Governance: Examining the decision-making process and governance structure. Migration to Cloud:Process of transitioning Airflow from on-premises to the cloud. Data Processing engines used with Airflow for Data Processing. Challenges: Obstacles faced during and after migration and how they were overcome. *Demonstrating how Airflow can be integrated with a central Glue Catalog and Data Lake Mesh model. Single Pane Orchestration (PAAS) and custom re-usable Github Actions: Examining benefits of using a Single Pane Orchestration model Monitoring
talk-data.com
Topic
AWS Glue
etl
data_catalog
aws
1
tagged
Activity Trend
10
peak/qtr
2020-Q1
2026-Q1
Top Events
AWS re:Invent 2024
15
O'Reilly Data Engineering Books
6
Data + AI Summit 2025
3
Data Engineering Podcast
2
O'Reilly Data Science Books
2
Databricks DATA + AI Summit 2023
2
Leaders of Analytics
1
Data Expo NL 2025
1
dbt Coalesce 2025
1
PyData Amsterdam 2025
1
Airflow Summit 2023
1
Experiencing Data w/ Brian T. O’Neill (AI & data product management leadership—powered by UX design)
1
Filtering by:
Airflow Summit 2023
×