This is an overview of migrating from Apache Airflow to Lakeflow Jobs for modern data orchestration. It covers key differences, best practices and practical examples of transitioning from traditional Airflow DAGs orchestrating legacy systems to declarative, incremental ETL pipelines with Lakeflow. Attendees will gain actionable tips on how to improve efficiency, scalability and maintainability in their workflows.

Discover how Databricks serverless compute revolutionizes data workflows by eliminating infrastructure management, enabling rapid scaling and optimizing costs for Notebooks, Jobs and Lakeflow Declarative Pipelines. This session will delve into the serverless architecture, highlighting its ability to dynamically allocate resources, reduce idle costs and simplify development cycles. Learn about recent advancements, including cost savings and practical strategies for migration and optimization. Tailored for Data Engineers and Architects, this talk will also explore use cases, features, limitations and future roadmap, empowering you to make informed infrastructure decisions while unlocking the full potential of Databricks’ serverless capabilities.