talk-data.com talk-data.com

B

Speaker

Bas Harenslak

1

talks

Staff Architect Astronomer

Staff Architect at Astronomer.

Bio from: Amsterdam Apache Airflow® Meetup at Booking.com!

Filtering by: Amsterdam Apache Airflow® Meetup at Booking.com! ×

Filter by Event / Source

Talks & appearances

Showing 1 of 4 activities

Search activities →

Historically Airflow was only capable of time-based scheduling, where a DAG would run at certain times. For data updates at varying times, such as an external party delivering data to an S3 bucket, that meant having to run a DAG and continuously poll for updates. Airflow 3 introduces event-driven scheduling that enables you to trigger DAGs based on such updates. In this talk I'll demonstrate how this changes your DAG's code and how this works internally in Airflow. Lastly, I'll demonstrate a practical use case that leverages Airflow 3's event-driven scheduling.