talk-data.com talk-data.com

PyData talk 2024-09-25 at 11:50

Building Large Scale ETL Pipelines with Dask

Description

Building scalable ETL pipelines and deploying them in the cloud can seem daunting. It shouldn't be. Leveraging proper technologies can make this process easy. We will discuss the whole process of developing a composable and scalable ETL pipeline centred around Dask that is fully built with Open Source tools and how we can deploy to the cloud.