talk-data.com
Airflow Summit
session
2020-07-01
Testing Airflow workflows - ensuring your DAGs work before going into production
Event:
Airflow Summit 2020
Topics
Description
How do you ensure your workflows work before deploying to production? In this talk I’ll go over various ways to assure your code works as intended - both on a task and a DAG level. In this talk I cover: How to test and debug tasks locally How to test with and without task instance context How to test against external systems, e.g. how to test a PostgresOperator? How to test the integration of multiple tasks to ensure they work nicely together