talk-data.com
Airflow Summit
session
2025-07-01
Operation Airlift: Uber's ongoing journey of migrating 200K pipelines to a single Airflow3 instance
Event:
Airflow Summit 2025
Topics
Description
Yes, you read that right — 200,000 pipelines, nearly 1 million task executions per day, all powered by a single Airflow instance. In this session, we’ll take you behind the scenes of one of the boldest orchestration projects ever attempted: how Uber’s data platform team is executing what might be the largest Apache Airflow migration in history — and doing it straight to Airflow 3. From scaling challenges and architectural choices to lessons learned in high-throughput orchestration, this is a deep dive into the tech, the chaos, and the strategy behind making data fly at unprecedented scale.