talk-data.com talk-data.com

Topic

Databricks

big_data analytics spark

509

tagged

Activity Trend

515 peak/qtr
2020-Q1 2026-Q1

Activities

Showing filtered results

Filtering by: Data + AI Summit 2025 ×
What’s New in Databricks SQL: Latest Features and Live Demos

Databricks SQL has added significant features in the last year at a fast pace. This session will share the most impactful features and the customer use cases that inspired them. We will highlight the new SQL editor, SQL coding features, streaming tables and materialized views, BI integrations, cost management features, system tables and observability features, and more. We will also share AI-powered performance optimizations.

Kill Bill-ing? Revenge is a Dish Best Served Optimized with GenAI

In an era where cloud costs can spiral out of control, Sportsbet achieved a remarkable 49% reduction in Total Cost of Ownership (TCO) through an innovative AI-powered solution called 'Kill Bill.' This presentation reveals how we transformed Databricks' consumption-based pricing model from a challenge into a strategic advantage through an intelligent automation and optimization. Understand how to use GenAI to reduce Databricks TCO Leverage generative AI within Databricks solutions enables automated analysis of cluster logs, resource consumption, configurations, and codebases to provide Spark optimization suggestions Create AI agentic workflows by integrating Databricks' AI tools and Databricks Data Engineering tools Review a case study demonstrating how Total Cost of Ownership was reduced in practice. Attendees will leave with a clear understanding of how to implement AI within Databricks solutions to address similar cost challenges in their environments.

Solving Exclusive Data Access With Role-Based Access Control

Do you have users that wear multiple hats over a day? Like working with data from various customers and hoping they don’t inadvertently aggregate data? Or are they working on sensitive datasets such as clinical trials that should not be combined, or are data sets that are subject to regulations? We have a solution! In this session, we will present a new capability that allows users wearing multiple hats to switch roles in the Databricks workspace to work exclusively on a dedicated project, data of a particular client or clinical trial. When switching to a particular role, the workspace adapts in such a way that only workspace objects and UC data of that particular role are accessible. We will also showcase the administrative experience of setting up exclusive access using groups and UC permissions.

Solving Health AI’s Data Problem

AI in healthcare has a data problem. Fragmented data remains one of the biggest challenges, and bottlenecks the development and deployment of AI solutions across life sciences, payers, and providers. Legacy paper-driven workflows and fragmented technology perpetuate silos, making it difficult to create a comprehensive, real-time picture of patient health. Datavant is leveraging Databricks and AWS technology to solve this problem at scale. Through our partnership with Databricks, we are centralizing storage of clinical data from what is arguably the largest health data network so that we can transform it into structured, AI-ready data – and shave off 80 percent of the work of deploying a new AI use case. Learn how we are handling the complexity of this effort while preserving the integrity of source data. We’ll also share early use cases now available to our healthcare customers.

Sponsored by: DataHub | Beyond the Lakehouse: Supercharging Databricks with Contextual Intelligence

While Databricks powers your data lakehouse, DataHub delivers the critical context layer connecting your entire ecosystem. We'll demonstrate how DataHub extends Unity Catalog to provide comprehensive metadata intelligence across platforms. DataHub's real-time platform:Cut AI model time-to-market with our unified REST and GraphQL APIs that ensure models train on reliable and compliant data from across platforms, with complete lineage trackingDecrease data incidents by 60% using our event-driven architecture that instantly propagates changes across systems*Transform data discovery from days to minutes with AI-powered search and natural language interfaces.Leaders use DataHub to transform Databricks data into integrated insights that drive business value. See our demo of syncback technology—detecting sensitive data and enforcing Databricks access controls automatically—plus our AI assistant that enhances' LLMs with cross-platform metadata.

Sponsored by: definity | How You Could Be Saving 50% of Your Spark Costs

Enterprise lakehouse platforms are rapidly scaling – and so are complexity and cost. After monitoring over 1B vCore-hours across Databricks and other Apache Spark™ environments, we consistently saw resource waste, preventable data incidents, and painful troubleshooting. Join this session to discover how definity’s unique full-stack observability provides job-level visibility in-motion, unifying infrastructure performance, pipeline execution, and data behavior, and see how enterprise teams use definity to easily optimize jobs and save millions – while proactively ensuring SLAs, preventing issues, and simplifying RCA.

Sponsored by: Retool | Retooling Intelligence: Build Scalable, Secure AI Agents for the Enterprise with Databricks + Retool

Enterprises need AI agents that are both powerful and production-ready while being scalable and secure. In this lightning session, you’ll learn how to leverage Retool’s platform and Databricks to design, deploy, and manage intelligent agents that automate complex workflows. We’ll cover best practices for integrating real-time Databricks data, enforcing governance, and ensuring scalability all while avoiding common pitfalls. Whether you’re automating internal ops or customer-facing tasks, walk away with a blueprint for shipping AI agents that actually work in the real world.

Summit Live: AI/BI Genie & Dashboards - Talk With Your Data With GenAI Powered Business Intelligence

AI/BI Genie lets anyone simply talk with their own data, using natural language, fully secured through UC to provide accurate answers within the context for your organization. AI/BI Dashboards goes beyond traditional BI tools, democratizing everyone to self-serve immediate interactive visuals on your own secured data. Hear from a customer and Databricks experts on the latest developments.

Accelerating Growth in Capital Markets: Data-Driven Strategies for Success

Growth in capital markets thrives on innovation, agility and real-time insights. This session highlights how leading firms use Databricks’ Data Intelligence Platform to uncover opportunities, optimize trading strategies and deliver personalized client experiences. Learn how advanced analytics and AI help organizations expand their reach, improve decision-making and unlock new revenue streams. Industry leaders share how unified data platforms break down silos, deepen insights and drive success in a fast-changing market. Key takeaways: Predictive analytics and machine learning strategies for growth Real-world examples of optimized trading and enhanced client engagement Tools to innovate while ensuring operational efficiency Discover how data intelligence empowers capital markets firms to thrive in today’s competitive landscape!

AI-Powered Data Discovery and Curation With Unity Catalog

This session is repeated. In today’s data landscape, the challenge isn’t just storing or processing data — it’s enabling every user, from data stewards to analysts, to find and trust the right data, fast. This session explores how Databricks is reimagining data discovery with the new Discover Page Experience — an intuitive, curated interface showcasing key data and workspace assets. We’ll dive into AI-assisted governance and AI-powered discovery features like AI-generated metadata, AI-assisted lineage and natural language data exploration in Unity Catalog. Plus, see how new certifications and deprecations bring clarity to complex data environments. Whether you’re a data steward highlighting trusted assets or an analyst navigating data without deep schema knowledge, this session will show how Databricks is making data discovery seamless for everyone.

Cutting Costs, Not Performance: Optimizing Databricks at Scale

As Databricks transforms data processing, analytics and machine learning, managing platform costs has become crucial for organizations aiming to maximize value while staying within budget. While Databricks offers unmatched scalability and performance, inefficient usage can lead to unexpected cost overruns. This presentation will explore common challenges organizations face in controlling Databricks costs and provide actionable best practices for optimizing resource allocation, preventing over-provisioning and eliminating underutilization. Drawing from NTT DATA’s experience, I'll share how we reduced Databricks costs by up to 50% through strategies like choosing the right compute resource, leveraging manage tables and using Unity Catalog features, such as system tables, to monitor consumption. Join this session to gain practical insights and tools that will empower your team to optimize Databricks without overspending.

Databricks AI Factory Transforming Seven West Media

The implementation of the Databricks AI Factory enabled Seven West Media to transform its business by accelerating the launch of AI-driven use cases, fostering innovation and reducing time to market. By leveraging a unified data and AI platform, the company achieved better ROI through optimized workflows, improved operational efficiency and scalable machine learning models. The AI Factory empowered data teams to experiment faster, unlocking deeper audience insights that enhanced engagement and content personalization. This transformation positioned Seven West Media as a leader in AI-driven media, driving measurable business impact and future-proofing its data strategy.

Databricks Lakeflow: the Foundation of Data + AI Innovation for Your Industry

Every analytics, BI and AI project relies on high-quality data. This is why data engineering, the practice of building reliable data pipelines that ingest and transform data, is consequential to the success of these projects. In this session, we'll show how you can use Lakeflow to accelerate innovation in multiple parts of the organization. We'll review real-world examples of Databricks customers using Lakeflow in different industries such as automotive, healthcare and retail. We'll touch on how the foundational data engineering capabilities Lakeflow provides help power initiatives that improve customer experiences, make real-time decisions and drive business results.

From Days to Minutes - AI Transforms Audit at KPMG

Imagine performing complex regulatory checks in minutes instead of days. We made this a reality using GenAI on the Databricks Data Intelligence Platform. Join us for a deep dive into our journey from POC to a production-ready AI audit tool. Discover how we automated thousands of legal requirement checks in annual reports with remarkable speed and accuracy. Learn our blueprint for: High-Performance AI: Building a scalable, >90% accurate AI system with an optimized RAG pipeline that auditors praise. Robust Productionization: Achieving secure, governed deployment using Unity Catalog, MLflow, LLM-based evaluation, and MLOps best practices. This session provides actionable insights for deploying impactful, compliant GenAI in the enterprise.

This session is repeated. This introductory workshop caters to data engineers seeking hands-on experience and data architects looking to deepen their knowledge. The workshop is structured to provide a solid understanding of the following data engineering and streaming concepts: Introduction to Lakeflow and the Data Intelligence Platform Getting started with Lakeflow Declarative Pipelines for declarative data pipelines in SQL using Streaming Tables and Materialized Views Mastering Databricks Workflows with advanced control flow and triggers Understanding serverless compute Data governance and lineage with Unity Catalog Generative AI for Data Engineers: Genie and Databricks Assistant We believe you can only become an expert if you work on real problems and gain hands-on experience. Therefore, we will equip you with your own lab environment in this workshop and guide you through practical exercises like using GitHub, ingesting data from various sources, creating batch and streaming data pipelines, and more.

Want to learn how to build your own custom data intelligence applications directly in Databricks? In this workshop, we’ll guide you through a hands-on tutorial for building a Streamlit web app that leverages many of the key products at Databricks as building blocks. You’ll integrate a live DB SQL warehouse, use Genie to ask questions in natural language, and embed AI/BI dashboards for interactive visualizations. In addition, we’ll discuss key concepts and best practices for building production-ready apps, including logging and observability, scalability, different authorization models, and deployment. By the end, you'll have a working AI app—and the skills to build more.

Health Data, Delivered: How Lakeflow Declarative Pipelines Powers the HealthVerity Marketplace

Building scalable, reliable ETL pipelines is a challenge for organizations managing large, diverse data sources. Theseus, our custom ETL framework, streamlines data ingestion and transformation by fully leveraging Databricks-native capabilities, including Lakeflow Declarative Pipelines, auto loader and event-driven orchestration. By decoupling supplier logic and implementing structured bronze, silver, and gold layers, Theseus ensures high-performance, fault-tolerant data processing with minimal operational overhead. The result? Faster time-to-value, simplified governance and improved data quality — all within a declarative framework that reduces engineering effort. In this session, we’ll explore how Theseus automates complex data workflows, optimizes cost efficiency and enhances scalability, showcasing how Databricks-native tools drive real business outcomes.

How We Transformed Two Businesses With Databricks as the Cornerstone

In this talk, we will discuss the lessons learned and future vision of transforming two business units to a modern financial data platform at Nasdaq. We'll highlight the transition from disjointed systems to a unified platform using Databricks. Our target audience includes financial engineers, data architects and technical leaders. The agenda covers challenges of legacy systems, reasons for choosing Databricks and key architectural decisions.

Igniting Innovation at Gilead: Convergence of Cloud, Data, AI and Agents

The convergence of cloud, data and AI is revolutionizing the pharmaceutical industry, creating a powerful ecosystem that drives innovation at scale across the entire value chain. At Gilead, teams harness these technologies on a unified cloud, data, & AI platform, accelerating business processes in pre-clinical and clinical stage, enabling smarter manufacturing and commercial processes, and deliver AI initiatives by reusing data products. Gilead will discuss how they have leveraged AWS, Databricks, and Data Mesh to manage vast amounts of heterogeneous data. Also, showcase use cases of traditional AI/ML, and Generative AI, and a Marketplace approach to drive adoption of AI Agents, demonstrating how this cloud-based, AI-powered platform is transforming the entire value chain. Gilead will also discuss how they are exploring the future of pharmaceutical innovation through Agentic AI, where the synergy of cloud, data and AI is unlocking new possibilities for a healthier world. In the second part, Muddu Sudhakar, Founder and Investor, will discuss how organizations can build and buy solutions for AI, Agents with Data Platforms. AWS and Databricks provide industry-leading platforms to build Agentic AI solutions. We will also cover Agentic AI Platform, Agent orchestration, Agent Interoperability, Agent Guardrails and Agentic workflows. This discussion also covers challenges in deploying and managing Agentic AI platforms. Enterprises need impactful AI initiatives & Agents to realize the promise and vision of AI and drive significant ROI.

Introduction to Unity Catalog Metrics: Define Your Business Metrics Once, Trust Everywhere

Today’s organizations need faster, more reliable insights — but metric sprawl and inconsistent KPIs make that difficult. In this session, you’ll learn how Unity Catalog Metrics helps unify business semantics across your organization. Define your KPIs once, apply enterprise-grade governance with fine-grained access controls, auditing and lineage, and use them across any Databricks tool — from AI/BI Dashboards and Genie to notebooks and Lakeflow. You’ll learn how to eliminate metric chaos by centrally defining and governing metrics with Unity Catalog. You’ll walk away with strategies to boost trust through built-in governance and empower every team — regardless of technical skill — to work from the same certified metrics.