talk-data.com
Databricks Summit
lightning_talk
2025-06-11 at 19:20
Reducing Transaction Conflicts in Databricks—Fundamentals and Applications at Asana
Event:
Data + AI Summit 2025
Topics
Description
When using ACID-guaranteed transactions on Databricks concurrently, we can run into transaction conflicts. This talk discusses the basics of concurrent transaction functionality in Databricks—what happens when various combinations of INSERT, UPDATE and MERGE INTO happen concurrently. We discuss how table isolation level, partitioning and deletion vectors affect this. We also mention how Asana used an intermediate blind append stage to support several hundred concurrent transaction updates into the same table.