DuckDB is the best way to execute SQL on a single node. But with its embedding-friendly nature, it makes an excellent foundation for building distributed systems. George Fraser, CEO of Fivetran, will tell us how Fivetran used DuckDB to power its Iceberg data lake writer—coordinating thousands of small, parallel tasks across a fleet of workers, each running DuckDB queries on bounded datasets. The result is a high-throughput, dual-format (Iceberg + Delta) data lake architecture where every write scales linearly, snapshots stay perfectly in sync, and performance rivals a commercial database while remaining open and portable.
talk-data.com
Speaker
George Fraser
2
talks
George Fraser is the co-founder and CEO of Fivetran, a global leader in modern data movement. After a career as a neuroscientist, he leveraged his analytical background to build Fivetran with Taylor Brown in 2012, following the Y Combinator accelerator. Under his leadership, the company has grown from a startup into a global enterprise valued at $5.6 billion, serving thousands of companies worldwide to automate data workflows. He holds a PhD in Neurobiology from the University of Pittsburgh and a BS in Cognitive Science and Biology from Carnegie Mellon University.
Bio from: Databricks DATA + AI Summit 2023
Frequent Collaborators
Filter by Event / Source
Talks & appearances
Showing 2 of 14 activities