talk-data.com talk-data.com

P

Speaker

Philippe Gagnon

1

talks

Senior Solutions Architect at Astronomer
Filtering by: Airflow Summit 2024 ×

Filter by Event / Source

Talks & appearances

Showing 1 of 5 activities

Search activities →

The scheduler is unarguably the most important component of an Airflow cluster. It is also the most complex and misunderstood by practitioners and administrators alike. In this talk, we will follow the path that a task instance takes to progress from creation to execution, and discuss the various configuration settings allowing users to tune the scheduler and executor to suit their workload patterns. Finally, we will dive deep into critical sections of the Airflow codebase and explore opportunities for optimization.