talk-data.com talk-data.com

Event

Small Data SF 2025

2025-11-04 – 2025-11-06 Small Data SF Visit website ↗

Activities tracked

2

Filtering by: George Fraser ×

Top Topics

AI/ML 7 Analytics 5 Motherduck 4 Data Engineering 3 Data Modelling 3 DuckDB 3 SQL 2 Python 2 Big Data 2 API 2 Iceberg 2 LLM 2 Data Management 1 Data Lakehouse 1 Data Lake 1

Top Speakers

Benn Stancil 2 George Fraser 2 Joe Reis 2 Jordan Tigani 2 Shelby Heinecke, PhD 2 Adi Polak 1 Barr Moses 1 Barry McCardel 1 Becca Bruggman 1 Brian Douglas 1 Colin Zima 1 Dennis Hume 1

Sessions & talks

Showing 1–2 of 2 · Newest first

Search within this event →

Building Distributed DuckDB Processing for Lakes

2025-11-05
talk
George Fraser (Fivetran)
Data Lake Delta DuckDB Fivetran Iceberg SQL

DuckDB is the best way to execute SQL on a single node. But with its embedding-friendly nature, it makes an excellent foundation for building distributed systems. George Fraser, CEO of Fivetran, will tell us how Fivetran used DuckDB to power its Iceberg data lake writer—coordinating thousands of small, parallel tasks across a fleet of workers, each running DuckDB queries on bounded datasets. The result is a high-throughput, dual-format (Iceberg + Delta) data lake architecture where every write scales linearly, snapshots stay perfectly in sync, and performance rivals a commercial database while remaining open and portable.