In today's data-driven world, the ability to efficiently manage and transform data is crucial for any organization. This presentation will explore the process of converting a complex and messy workflow into a clean and simple Lakeflow Declarative Pipelines at a large integrated health system, Intermountain Health.Alteryx is a powerful tool for data preparation and blending, but as workflows grow in complexity, they can become difficult to manage and maintain. Lakeflow Declarative Pipelines, on the other hand, offers a more democratized, streamlined and scalable approach to data engineering, leveraging the power of Apache Spark and Delta Lake.We will begin by examining a typical legacy workflow, identifying common pain points such as tangled logic, performance bottlenecks and maintenance challenges. Next, we will demonstrate how to translate this workflow into a Lakeflow Declarative Pipelines, highlighting key steps such as data transformation, validation and delivery.
talk-data.com
Speaker
Peter Jones
1
talks
Analytics Engineer
Intermountain Healthcare
Peter works as an Analytics Engineer at Intermountain Health and serves as the Databricks account administrator.
Bio from: Data + AI Summit 2025
Filtering by:
Data + AI Summit 2025
×
Filter by Event / Source
Talks & appearances
Showing 1 of 4 activities