talk-data.com talk-data.com

Topic

Databricks

big_data analytics spark

1286

tagged

Activity Trend

515 peak/qtr
2020-Q1 2026-Q1

Activities

1286 activities · Newest first

Sponsored by: Infosys | Beyond Hype: Scale & Democratize Agentic AI across enterprise to realize business outcomes.

Agentic AI and multimodal data are the next frontiers for realizing intelligent and autonomous business systems. Learn how Infosys innovates with Databricks for accelerating data to AI agent journey at scale across an enterprise. Hear our pragmatic capability driven approach instead of use case-based approach to bring the data universe, AI foundations, agent management, data and AI governance and collaboration under unified management.

The explosion of AI has helped make the enterprise data landscape more important, and complex, than ever before. Join us to learn how Databricks’ and Tableau’s platforms come together to empower users of all kinds to see, understand, and act on your data in a secure, governed, and performant way.

Summit Live: Databricks Apps -  Empowering data and AI teams to build and deploy applications with ease

Databricks Apps empowers data and AI teams to easily build and deploy applications with ease - in just minutes not months! It’s the fastest and most secure way to deliver impactful solutions, with built-in governance based on Unity Catalog and the Databricks Data Intelligence Platform. See demos on the latest and greatest, and how YOU can get started right away.

Crypto at Scale: Building a High-Performance Platform for Real-Time Blockchain Data

In today’s fast-evolving crypto landscape, organizations require fast, reliable intelligence to manage risk, investigate financial crime, and stay ahead of evolving threats. In this session we will discover how Elliptic built a scalable, high-performance Data Intelligence Platform that delivers real-time, actionable Blockchain insights to their customers. We’ll walk you through some of the key components of the Elliptic Platform, including the Elliptic Entity Graph and our User-Facing Analytics. Our focus will be put on the evolution of our User-Facing Analytics capabilities, and specifically how components from the Databricks ecosystem such as Structured Streaming, Delta Lake, and SQL Warehouse have played a vital role. We’ll also share some of the optimizations we’ve made to our streaming jobs to maximize performance and ensure Data Completeness. Whether you’re looking to enhance your streaming capabilities, expand your knowledge of how crypto analytics works or simply discover novel approaches to data processing at scale, this session will provide concrete strategies and valuable lessons learned.

Databricks Observability: Using System Tables to Monitor and Manage Your Databricks Instance

The session will cover how to use Unity Catalog governed system tables to understand what is happening in Databricks. We will touch on key scenarios for FinOps, DevOps and SecOps to ensure you have a well-observed Data Intelligence Platform. Learn about new developments in system tables and other features that will help you observe your Databricks instance.

Data Intelligence for Marketing Breakout: Agentic Systems for Bayesian MMM and Consumer Testing

This talk dives into leveraging GenAI to scale sophisticated decision intelligence. Learn how an AI copilot interface simplifies running complex Bayesian probabilistic models, accelerating insight generation, and accurate decision making at the enterprise level. We talk through techniques for deploying AI agents at scale to simulate market dynamics or product feature impacts, providing robust, data-driven foresight for high-stakes innovation and strategy directly within your Databricks environment. For marketing teams, this approach will help you leverage autonomous AI agents to dynamically manage media channel allocation while simulating real-world consumer behavior through synthetic testing environments.

Delivering Sub-Second Latency for Operational Workloads on Databricks

As enterprise streaming adoption accelerates, more teams are turning to real-time processing to support operational workloads that require sub-second response times. To address this need, Databricks introduced Project Lightspeed in 2022, which recently delivered Real-Time Mode in Apache Spark™ Structured Streaming. This new mode achieves consistent p99 latencies under 300ms for a wide range of stateless and stateful streaming queries. In this session, we’ll define what constitutes an operational use case, outline typical latency requirements and walk through how to meet those SLAs using Real-Time Mode in Structured Streaming.

Developing the Dreamers of Data + AI’s Future: How 84.51˚ builds upskilling to accelerate adoption

“Once an idea has taken hold of the brain it's almost impossible to eradicate. An idea that is fully formed — fully understood — that sticks, right in there somewhere.” The Data Scientists and Engineers at 84.51˚ utilize the Databricks Lakehouse for a wide array of tasks, including data exploration, analysis, machine learning operations, orchestration, automated deployments and collaboration. In this talk, 84.51˚’s Data Science Learning Lead, Michael Carrico, will share their approach to upskilling a diverse workforce to support the company’s strategic initiatives. This approach includes creating tailored learning experiences for a variety of personas using content curated in partnership with Databricks’ educational offerings. Then he will demonstrate how he puts his 11 years of data science and engineering experience to work by using the Databricks Lakehouse not just as a subject, but also as a tool to create impactful training experiences and a learning culture at 84.51˚.

Empowering the Warfighter With AI

The new Budget Execution Validation process has transformed how the Navy reviews unspent funds. Powered by Databricks Workflows, MLflow, Delta Lake and Apache Spark™, this data-driven model predicts which financial transactions are most likely to have errors, streamlining reviews and increasing accuracy. In FY24, it helped review $40 billion, freeing $1.1 billion for other priorities, including $260 million from active projects. By reducing reviews by 80%, cutting job runtime by over 50% and lowering costs by 60%, it saved 218,000 work hours and $6.7 million in labor costs. With automated workflows and robust data management, this system exemplifies how advanced tools can improve financial decision-making, save resources and ensure efficient use of taxpayer dollars.

Healthcare and Life Sciences: Getting Started with AI Agents

Healthcare and life sciences organizations are exploring AI Agents, driving transformation through intelligent supply chains to helping up-level the patient experience via virtual assistants. This session explores how you can get started with AI Agents, powered by Databricks and robust data governance, and tapping into the full potential of all your data. You’ll learn practical steps for getting started: unifying data with Databricks, ensuring compliance with Unity Catalog, and rapidly deploying AI Agents to drive operational efficiency, improve care, and foster innovation across healthcare and life sciences.

How to Migrate From Oracle to Databricks SQL

Migrating your legacy Oracle data warehouse to the Databricks Data Intelligence Platform can accelerate your data modernization journey. In this session, learn the top strategies for completing this data migration. We will cover data type conversion, basic to complex code conversions, validation and reconciliation best practices. Discover the pros and cons of using CSV files to PySpark or using pipelines to Databricks tables. See before-and-after architectures of customers who have migrated, and learn about the benefits they realized.

Iceberg Table Format Adoption and Unified Metadata Catalog Implementation in Lakehouse Platform

DoorDash Data organization actively adopts LakeHouse paradigm. This presentation describes the methodology which allows to migrate the classic Data Warehouse and Data Lake platforms to unified LakeHouse solution.The objective of this effort include Elimination of excessive data movement.Seamless integration and consolidation of the query engine layers, including Snowflake, Databricks, EMR and Trino.Query performance optimization.Abstracting away complexity of underlying storage layers and table formatsStrategic and justified decision on the Unified Metadata catalog used across varios compute platforms

Optimizing Analytics Infrastructure: Lessons from Migrating Snowflake to Databricks

This session explores the strategic migration from Snowflake to Databricks, focusing on the journey of transforming a data lake to leverage Databricks’ advanced capabilities. It outlines the assessment of key architectural differences, performance benchmarks, and cost implications driving the decision. Attendees will gain insights into planning and execution, including data ingestion pipelines, schema conversion and metadata migration. Challenges such as maintaining data quality, optimizing compute resources and minimizing downtime are discussed, alongside solutions implemented to ensure a seamless transition. The session highlights the benefits of unified analytics and enhanced scalability achieved through Databricks, delivering actionable takeaways for similar migrations.

Scaling Identity Graph Ingestion to 1M Events/Sec with Spark Streaming & Delta Lake

Adobe’s Real-Time Customer Data Platform relies on the identity graph to connect over 70 billion identities and deliver personalized experiences. This session will showcase how the platform leverages Databricks, Spark Streaming and Delta Lake, along with 25+ Databricks deployments across multiple regions and clouds — Azure & AWS — to process terabytes of data daily and handle over a million records per second. The talk will highlight the platform’s ability to scale, demonstrating a 10x increase in ingestion pipeline capacity to accommodate peak traffic during events like the Super Bowl. Attendees will learn about the technical strategies employed, including migrating from Flink to Spark Streaming, optimizing data deduplication, and implementing robust monitoring and anomaly detection. Discover how these optimizations enable Adobe to deliver real-time identity resolution at scale while ensuring compliance and privacy.

Semiconductor AI Success: Marvell’s Data + AI Governance

Marvell’s AI-driven solutions, powered by Databricks’ Data Intelligence Platform, provide a robust framework for secure, compliant and transparent Data and AI workflows leveraging Data & AI Governance through Unity Catalog. Marvell ensures centralized management of data and AI assets with quality, security, lineage and governance guardrails. With Databricks Unity Catalog, Marvell achieves comprehensive oversight of structured and unstructured data, AI models and notebooks. Automated governance policies, fine-grained access controls and lineage tracking help enforce regulatory compliance while streamlining AI development. This governance framework enhances trust and reliability in AI-powered decision-making, enabling Marvell to scale AI innovation efficiently while minimizing risks. By integrating data security, auditability and compliance standards, Marvell is driving the future of responsible AI adoption with Databricks.

Supercharge Your Enterprise BI: A Practitioner’s Guide for Migrating to AI/BI

Are you striving to build a data-driven culture while managing costs and reducing reporting latency? Are your BI operations bogged down by complex data movements rather than delivering insights? Databricks IT faced these challenges in 2024 and embarked on an ambitious journey to make Databricks AI/BI our enterprise-wide reporting platform. In just two quarters, we migrated 2,000 dashboards from a traditional BI tool — without disrupting business operations. We’ll share how we executed this large-scale transition cost-effectively, ensuring seamless change management and empowering non-technical users to leverage AI/BI. You’ll gain insights into: Key migration strategies that minimized disruption and optimized efficiency Best practices for user adoption and training to drive self-service analytics Measuring success with clear adoption metrics and business impact Join us to learn how your organization can achieve the same transformation with AI-powered enterprise reporting.

Transforming Customer Processes and Gaining Productivity With Lakeflow Declarative Pipelines

Bradesco Bank is one of the largest private banks in Latin America, with over 75 million customers and over 80 years of presence in FSI. In the digital business, velocity to react to customer interactions is crucial to succeed. In the legacy landscape, acquiring data points on interactions over digital and marketing channels was complex, costly and lacking integrity due to typical fragmentation of tools. With the new in-house Customer Data Platform powered by Databricks Intelligent Platform, it was possible to completely transform the data strategy around customer data. Using some key components such Uniform and Lakeflow Declarative Pipelines, it was possible to increase data integrity, reduce latency and processing time and, most importantly, boost personal productivity and business agility. Months of reprocessing, weeks of human labor and cumbersome and complex data integrations were dramatically simplified achieving significant operational efficiency.

Unifying GTM Analytics: The Strategic Shift to Native Analytics and AI/BI Dashboards at Databricks

The GTM team at Databricks recently launched the GTM Analytics Hub—a native AI/BI platform designed to centralize reporting, streamline insights, and deliver personalized dashboards based on user roles and business needs. Databricks Apps also played a crucial role in this integration by embedding AI/BI Dashboards directly into internal tools and applications, streamlining access to insights without disrupting workflows. This seamless embedding capability allows users to interact with dashboards within their existing platforms, enhancing productivity and collaboration. Furthermore, AI/BI Dashboards leverage Databricks' unified data and governance framework. Join us to learn how we’re using Databricks to build for Databricks—transforming GTM analytics with AI/BI Dashboards, and what it takes to drive scalable, user-centric analytics adoption across the business.