Apache Airflow has over 650 Python dependencies. In case you did not know already, dependencies in Python are difficult subject. And Airflow has its own, custom ways of managing the dependencies. Airflow has a rather complex system to manage dependencies in their CI system, but this talk is not about it. This talk is directed to the users of Airflow who want to keep their dependencies updated, describing ways they can do it. This presentation will explain how to effectively manage and handle custom dependencies in Airflow. Jarek will guide you through practical solutions and best practices to make your Airflow experience with dependencies - yes you guessed it - a breeze.
talk-data.com
J
Speaker
Jarek Potiuk
2
talks
Independent Open-Source Contributor and Advisor
Apache Software Foundation
PMC member and Security Committee member for Apache Airflow.
Bio from: NYC Airflow Rooftop Happy Hour ft. PMC Member Jarek Potiuk!
Frequent Collaborators
Filtering by:
Airflow Summit 2023
×
Filter by Event / Source
Talks & appearances
Showing 2 of 15 activities
This sesion is about the current state of implementation for multi-tenancy feature of Airflow. This is a long-term feature that involves multiple changes, separate AIPs to implement, with the long-term vision of having single Airflow instance supporting multiple, independed teams using it - either from the same company or as part of Airflow-As-A-Service implementation.