Modern data platforms face increasing pressure to make their data platforms both high-performing and cost-efficient. This session provides a deep dive into practical strategies for optimizing both compute and storage resources, ensuring scalability, efficiency, and reliability across diverse workloads. Attendees will learn how to choose the right storage formats, right-size compute resources, and design workload-aware cluster configurations. The session also covers techniques for addressing critical performance bottlenecks such as small files, skewed partitions, inefficient query plans, and underutilized compute clusters. Whether you’re designing a new platform or tuning an existing one, you’ll gain actionable insights to enhance performance, minimize costs, and maximize the value of your data ecosystem.

Key Takeaways - Understand how to select optimal storage formats and data access patterns for batch and real-time workloads. - Learn best practices for right-sizing compute resources and configuring workload-aware clusters. - Identify and troubleshoot performance bottlenecks such as small files, data skew, and inefficient queries. - Apply systematic approaches to detect, monitor, and eliminate underutilized compute clusters. - Gain actionable strategies to improve performance, reduce costs, and scale modern data platforms effectively.