Airflow 2.2 introduced Deferrable Tasks (sometimes called “async operators”), a new mechanism to efficiently run tasks that depend on external activity. But when should you use them, how do they work, what do you need to do to make Operators support them, and what else could we do in Airflow with this model?

Airflow has a lot of moving parts, and it can be a little overwhelming as a new user - as I was not too long ago. Join me as we go though Airflow’s architecture at a high level, explore how DAGs work and run, and look at some of the good, the bad, and the unexpected things lurking inside.