talk-data.com talk-data.com

Databricks Summit talk 2025-06-10 at 19:40

Spark Connect: Flexible, Local Access to Apache Spark at Scale

Description

What if you could run Spark jobs without worrying about clusters, versions and upgrades? Did you know Spark has this functionality built-in today? Join us to take a look at this functionality โ€” Spark Connect. Join us to dig into how Spark Connect works โ€” abstracting away Spark clusters away in favor of the DataFrame API and unresolved logical plans. You will learn some of the cool things Spark Connect unlocks, including: Moving you from thinking about clusters to just thinking about jobs Making Spark code more portable and platform agnostic Enabling support for languages such as Go