With recent works in the direction of Executor Decoupling and interest in Hybrid Execution, we find it’s still quite common for Airflow users to use the old-time rule of thumbs like “Don’t Use Airflow with LocalExecutor in production”, “If your scheduler lags, split your DAGs over two separate Airflow Clusters”, and so on. In our talk, we will show a deep dive comparison between various Execution models Airflow support and hopefully update understanding of their efficiency and limitations.
talk-data.com
Speaker
Deepan Ignaatious
1
talks
Sr. Product Manager at DoubleCloud
Deepan Ignaatious, working as a Senior Product Manager at DoubleCloud, is deeply immersed in shaping the product’s features and roadmap. With extensive experience in the realm of data products, he focuses on delivering tangible customer value within the data analytics sphere. His role revolves around the product team to define strategic features and steer the product’s direction. With a wealth of expertise, Deepan navigates the complex landscape of data analytics, striving to translate insights into meaningful customer solutions.
Bio from: Big Data LDN 2024
Filtering by:
Airflow Summit 2024
×
Filter by Event / Source
Talks & appearances
Showing 1 of 2 activities