Modern data engineering leverages Python to build robust, scalable, end-to-end workflows. In this talk, we will cover how Snowflake offers you a flexible development environment for developing Python data pipelines, performing transformation at scale, orchestrating and deploying your pipelines at scale. Topics we’ll cover include:

•Ingest: Data source APIs, Snowflake file-to-read and ingest data of any format when files arrive, with sources outside Snowflake •Develop: Packaging (artifact repo), Python runtimes, IDE (Notebook, vscode) •Transform: Snowpark pandas, UDFs, UDAFs •Deploy: Tasks, Notebook scheduling