Airflow 3.1 is here, bringing new features and enhancements that make orchestration faster, more flexible, and easier to manage at scale. In this session, we’ll walk through the most impactful changes in 3.1, including improvements to the user experience, DAG management, and core functionality. Whether you’re running Airflow in production or just getting started, you’ll leave with a clear picture of what’s new, why it matters, and how to take advantage of it.
talk-data.com
Speaker
Brent Bovenzi
8
talks
Senior Frontend Engineer at Astronomer; Airflow Committer & PMC Member.
Bio from: NYC Airflow Meetup at Astronomer HQ!
Frequent Collaborators
Filter by Event / Source
Talks & appearances
8 activities · Newest first
In Airflow 2 there was a plugin mechanism to extend the UI for new functions as well as be able to add hooks and other features. As Airflow 3 rewrote the UI old Plugins were not working for all cases anymore. Airflow 3.1 now provides a re-vamped option to extend the UI with a new plugin schema in native React components and embedded iframes following AIP-68 definitions. In this session we will provide an overview about capabilities and give some intro how you can roll-your-own.
Apache Airflow® 3 is here, bringing major improvements to data orchestration. In this keynote, core Airflow contributors will walk through key enhancements that boost flexibility, efficiency, and user experience. Vikram Koka will kick things off with an overview of Airflow 3, followed by deep dives into DAG versioning (Jed Cunningham), enhanced backfilling (Daniel Standish), and a modernized UI (Brent Bovenzi & Pierre Jeambrun). Next, Ash Berlin-Taylor, Kaxil Naik, and Amogh Desai will introduce the Task Execution Interface and Task SDK, enabling tasks in any environment and language. Jens Scheffler will showcase the Edge Executor, while Constance Martineau, Tzu-ping Chung and Vincent Beck will demo event-driven scheduling and data assets. Finally, Buğra Öztürk will unveil CLI enhancements for automation and debugging. This keynote sets the stage for Airflow 3—don’t miss the chance to learn from the experts shaping the future of workflow orchestration!
Join this live demo of the Airflow UI while we answer your questions and discuss your ideas on how to improve the experience.
Soon we will finally switch to a 100% React UI with a full separation between the API and UI as well. While we are doing such a big change, let’s also take the opportunity to imagine whole new interfaces vs just simply modernizing the existing views. How can we use design to help you better understand what is going on with your DAG? Come listen to some of our proposed ideas and bring your own big ideas as the second half will be an open discussion.
We are continuing to modernize the Airflow UI to make it easier to manage all aspects of your DAGs. See a demo of the latest updates and improve your workflows with new tips and tricks. Then get a preview of what else will be coming soon. Followed up by Q&A for people to field their own use-cases and explore new ideas on how to improve the user experience.
Sneak peek at the future of the Airflow UI. In Airflow 2.3 with the Tree -> Grid view changes, we began to swap out parts of the Flask app with React. This was one step towards AIP-38, to build a fully modern UI for Airflow. Come check out what is in store after Grid view in the current UI. Discuss the possibilities to rethink Airflow with a brand new UI down the line. Such as: Integrating all DAG visualizations into each other and remove constant page reloads More live data Greater cross-DAG visualizations (ie: DAG Dependencies view from 2.1) Improved user settings: (dark mode, color blind support, language, date format) And more!