talk-data.com
Airflow Summit
session
2024-07-01
What If...? Running Airflow Tasks without the workers
Event:
Airflow Summit 2024
Topics
Description
Airflow executes all tasks on the workers, including deferrable operators that must run on the workers before deferring to the triggerer. However, running some tasks directly from the triggerer can be beneficial in certain situations. This presentation will explain how deferrable operators function and examine ways to modify the Airflow implementation to enable tasks to run directly from the triggerer.