talk-data.com talk-data.com

Event

Big Data LDN 2025

2025-09-24 – 2025-09-25 Big Data LDN/Paris

Activities tracked

14

Filtering by: Data Engineering ×

Sessions & talks

Showing 1–14 of 14 · Newest first

Search within this event →

Accelerating Data Science with NVIDIA RAPIDS

2025-09-25
Face To Face

This presentation provides an overview of how NVIDIA RAPIDS accelerates data science and data engineering workflows end-to-end. Key topics include leveraging RAPIDS for machine learning, large-scale graph analytics, real-time inference, hyperparameter optimization, and ETL processes. Case studies demonstrate significant performance improvements and cost savings across various industries using RAPIDS for Apache Spark, XGBoost, cuML, and other GPU-accelerated tools. The talk emphasizes the impact of accelerated computing on modern enterprise applications, including LLMs, recommenders, and complex data processing pipelines.

Dash-bored? The future of data engineering

2025-09-25
Face To Face
Sian Rodway (Manuka AI) , Sam Cremins (Kingsley Napley) , Leanne Lynch (ISS UK&I)

Data remains one of the most valuable assets a company has to guide its decision making. How that data is processed, used and presented is changing rapidly and with it the role and skills of data engineers. 

In this fireside chat, Manuka will explore the future of data engineering and the ongoing challenges of overcoming legacy constrains and governance with the latest breakthroughs in AI.

Expect a grounded discussion on:

• What “AI-ready” really means for data engineers

• Engineering through legacy constraints in a highly regulated environment

• Designing ingestion, orchestration, and observability that scale

• Embedding governance and quality without slowing delivery

• What’s next for data engineering in the age of generative AI

Whether you’re building pipelines, managing platforms, or designing modern data infrastructure, this is a rare behind-the-scenes look at how data engineering is evolving to meet the AI moment.

Let's talk about Context - Why AI Agents are not that smart!

2025-09-25
Face To Face
Guy Fighel (Hetz Ventures) , Omri Lifshitz (Upriver)

Are AI code generators delivering SQL that "looks right but works wrong" for your data engineering challenges? Is your AI generating brilliant-sounding but functionally flawed results? 

The critical bottleneck isn't the AI's intelligence; it's the missing context.

In this talk, we will put thing in context and reveal how providing AI with structured, deep understanding—from data semantics and lineage to user intent and external knowledge—is the true paradigm shift. 

We'll explore how this context engineering powers the rise of dependable AI agents and leverages techniques like Retrieval-Augmented Generation (RAG) to move beyond mere text generation towards trustworthy, intelligent automation across all domains. 

This limitation highlights a broader challenge across AI applications: the need for systems to possess a deep understanding of all relevant signals, ranging from environmental cues and user history to explicit intent, to achieve reliable and meaningful operation.

Join us for real-world, practical case studies directly from data engineers that demonstrate precisely how to unlock this transformative power and achieve truly reliable AI.

AI Agents for Customer Data Architectures

2025-09-25
Face To Face
Marcus Owens (Amperity)

Join Amperity’s Marcus Owens, Lead Solution Consultant, to learn more about the rapid innovations in data architecture brought by the new wave of AI agents. This session will start with a quick overview of what makes a good AI Agent – and then focus on how Agentic strategies can accelerate two key needs in customer data: 

Make Customer Data Usable – How AI Agents accelerate customer data engineering with Amperity’s Stitch and Chuck Data – saving data engineering teams hundreds of hours of effort. 

Make Use of Customer Data – How AmpAI allows Marketers to build outcome-driven customer journeys, going from intent to results faster than ever before.

Making your Data AI ready with DataOps

2025-09-25
Face To Face
Guy Adams (DataOps.live)

AI is only as good as the data it runs on. Yet Gartner predicts in 2026, over 60% of AI projects will fail to deliver value - because the underlying data isn’t truly AI-ready. “Good enough” data isn’t enough.

In this exclusive BDL launch session, DataOps.live reveal Momentum, the next generation of its DataOps automation platform designed to operationalize trusted AI at enterprise scale.

Based on experiences from building over 9000 Data Products to date, Momentum introduces breakthrough capabilities including AI-Ready Data Scoring to ensure data is fit for AI use cases, Data Product Lineage for end-to-end visibility, and a Data Engineering Agent that accelerates building reusable data products. Combined with automated CI/CD, continuous observability, and governance enforcement, Momentum closes the AI-readiness gap by embedding collaboration, metadata, and automation across the entire data lifecycle.

Backed by Snowflake Ventures and trusted by leading enterprises, including AstraZeneca, Disney and AT&T, DataOps.live is the proven catalyst for scaling AI-ready data. In this session, you’ll unpack what AI-ready data really means, learn essential practices, discover a faster, easier, and more impactful way to make your AI initiatives succeed.

Be the first to see Momentum in action - the future of AI-ready data.

Future of Data Engineering in an Agentic World

2025-09-25
Face To Face
Cyril Sonnefraud (Matillion) , Joe Herbert (Matillion)

This session will provide a Maia demo with roadmap teasers. The demo will showcase Maia's core capabilities: authoring pipelines in business language, multiplying productivity by accelerating tasks, and enabling self-service. It demonstrates how Maia takes natural language prompts and translates them into YAML-based, human-readable Data Pipeline Language (DPL), generating graphical pipelines. Expect to see Maia interacting with Snowflake metadata to sample data and suggest transformations, as well as its ability to troubleshoot and debug pipelines in real-time. The session will also cover how Maia can create custom connectors from REST API documentation in seconds, a task that traditionally takes days . Roadmap teasers will likely include the upcoming Semantic Layer, a Pipeline Reviewing Agent, and enhanced file type support for various legacy ETL tools and code conversions.

Declarative LLM Engineering with DSPy and Dagster

2025-09-25
Face To Face
Pedram Navid (Dagster Labs)

Data teams know the pain of moving from proof-of-concepts to production. We’ve all seen brittle scripts, one-off notebooks, and manual fixes turn into hidden risks. With large language models, the same story is playing out, unless we borrow the lessons of modern data engineering.

This talk introduces a declarative approach to LLM engineering using DSPy and Dagster. DSPy treats prompts, retrieval strategies, and evaluation metrics as first-class, composable building blocks. Instead of tweaking text by hand, you declare the behavior you want, and DSPy optimizes and tunes the pipeline for you. Dagster is built on a similar premise; with Dagster Components, you can build modular and declarative pipelines.

This approach means:

- Trust & auditability: Every LLM output can be traced back through a reproducible graph.

- Safety in production: Automated evaluation loops catch drift and regressions before they matter.

- Scalable experimentation: The same declarative spec can power quick tests or robust, HIPAA/GxP-grade pipelines.

By treating LLM workflows like data pipelines: declarative, observable, and orchestrate, we can avoid the prompt spaghetti trap and build AI systems that meet the same reliability bar as the rest of the stack.

The Great Data Engineering Reset: From Pipelines to Agents

2025-09-25
Face To Face
Joe Reis (Reis Megacorp)

For years, data engineering was a story of predictable pipelines: move data from point A to point B. But AI just hit the reset button on our entire field. Now, we're all staring into the void, wondering what's next. While the fundamentals haven't changed, data remains challenging in the traditional areas of data governance, data management, and data modeling, which still present challenges. Everything else is up for grabs.

This talk will cut through the noise and explore the future of data engineering in an AI-driven world. We'll examine how team structures will evolve, why agentic workflows and real-time systems are becoming non-negotiable, and how our focus must shift from building dashboards and analytics to architecting for automated action. The reset button has been pushed. It's time for us to invent the future of our industry.

Data Engineer Things (DET) London Meetup

2025-09-24
Face To Face

The Data Engineer Things (DET) London Meetup focuses on hosting in-person professional networking events for the data engineering community. Join our meetup for talks from industry experts and connect with like-minded data professionals! All experience levels are welcome.

Driving Impact Through Data: The Evolution of Data Quality at OutSystems

2025-09-24
Face To Face
Pedro Sá Martins (Outsystems)

As the pioneers of the low-code market since 2001, enterprise software delivery solution OutSystems has evolved rapidly alongside the changing landscape of data. With a global presence and a vast community of over 750,000 members, OutSystems continues to leverage innovative tools, including data observability and generative AI, to help their customers succeed.

In this session, Pedro Sá Martins, Head of Data Engineering, will share the evolution of OutSystems’ data landscape, including how OutSystems has partnered with Snowflake, Fivetran and Monte Carlo to address their modern data challenges. He’ll share best practices for implementing scalable data quality programs to drive innovative technologies, as well as what’s on the data horizon for the OutSystems team.

Machine Scale vs. Human Scale: The Looming Crisis in Data Engineering

2025-09-24
Face To Face
Julian Wiffen (Matillion) , Frank Weigel (Matillion)

A paradigm shift is underway; the primary consumer of data is evolving from human analysts to AI agents. This presents a strategic challenge to every data leader: how do we architect an ecosystem that satisfies relentless, machine-scale demand for governed data without overwhelming our most valuable human experts? A chaotic free-for-all, with AI agents querying sources directly, is a regression that would erase a decade of progress in data warehousing and governance.

To solve this machine-scale problem, we must deploy a machine-scale solution. This session casts a vision for the future, exploring why current models are ill-equipped for the AI era. We will introduce the concept of the virtual data engineer—an AI-powered partner designed to augment and accelerate human capabilities on a collaborative platform. Discover how to evolve your team and architecture to turn this challenge into a strategic advantage, ensuring you lead the way through this transformation.

The Great Data Engineering Reset: From Pipelines to Agents

2025-09-24
Face To Face
Joe Reis (Reis Megacorp)

For years, data engineering was a story of predictable pipelines: move data from point A to point B. But AI just hit the reset button on our entire field. Now, we're all staring into the void, wondering what's next. While the fundamentals haven't changed, data remains challenging in the traditional areas of data governance, data management, and data modeling, which still present challenges. Everything else is up for grabs.

This talk will cut through the noise and explore the future of data engineering in an AI-driven world. We'll examine how team structures will evolve, why agentic workflows and real-time systems are becoming non-negotiable, and how our focus must shift from building dashboards and analytics to architecting for automated action. The reset button has been pushed. It's time for us to invent the future of our industry.

The State of Data Engineering

2025-09-24
Face To Face
Jesse Anderson (Big Data Institute)

The data landscape is fickle, and once-coveted roles like 'DBA' and 'Data Scientist' have faced challenges. Now, the spotlight shines on Data Engineers, but will they suffer the same fate? This talk dives into historical trends.

In the early 2010’s, DBA/data warehouse was the sexiest job. Data Warehouse became the “No Team.”

In the mid-2010’s, data scientist was the sexiest job. Data Science became the “mistaken for” team.

Now, data engineering is the sexiest job. Data Engineering became the “confused team”. The confusion run rampant with questions about the industry: What is a data engineer? What do they do? Should we have all kinds of nuanced titles for variations? Just how technical should they be?

Together, let’s go back to history and look for ways on how data engineering can avoid the same fate as data warehousing and data science. This talk provides a thought-provoking discussion on navigating the exciting yet challenging world of data engineering. Let's avoid the pitfalls of the past and shape a future where data engineers thrive as essential drivers of innovation and success.

A Day in the Life of Tomorrow’s Data Engineer

2025-09-24
Face To Face
Cyril Sonnefraud (Matillion) , Julian Wiffen (Matillion)

This session will explore the evolving role of data engineers. Data engineering is currently a bottleneck due to overwhelming requests and complex knowledge work. Maia acts as a "digital data engineer" or a "virtual data team" that amplifies productivity by 100x. It enables users, from skilled engineers to citizen data analysts, to author pipelines in natural business language. The session will demonstrate Maia's ability to accelerate mundane and advanced tasks,troubleshoot and debug pipelines in real-time, and generate high-quality, auditable pipelines using Matillion's proprietary, human-readable Data Pipeline Language (DPL), which avoids "spaghetti code" common with generic LLMs.