talk-data.com talk-data.com

S

Speaker

Syed Hussain

2

talks

Software Developer - AWS

Filter by Event / Source

Talks & appearances

2 activities · Newest first

Search activities →

Since version 2.7 and the advent of AIP-51, Airflow has started to fully support the creation of custom executors. Before we dive into the components of an executor and how they work, we will briefly discuss the Executor Decoupling initiative which allowed this new feature. Once we understand the parts required, we will explore the process of crafting our own executors, using real-world examples, and demonstrations of executors developed within the Amazon Provider Package as a guide. By demystifying the process of executor creation and emphasizing the opportunities for contribution, we aim to empower Airflow users and providers to harness the full potential of custom executors, enriching the Airflow ecosystem as a whole!

Deep dive into how AWS is developing Deferrable Operators for the Amazon Provider Package to help users realize the potential cost-savings provided by Deferrable Operators, and promote their usage.