Data teams know the pain of moving from proof-of-concepts to production. We’ve all seen brittle scripts, one-off notebooks, and manual fixes turn into hidden risks. With large language models, the same story is playing out, unless we borrow the lessons of modern data engineering.

This talk introduces a declarative approach to LLM engineering using DSPy and Dagster. DSPy treats prompts, retrieval strategies, and evaluation metrics as first-class, composable building blocks. Instead of tweaking text by hand, you declare the behavior you want, and DSPy optimizes and tunes the pipeline for you. Dagster is built on a similar premise; with Dagster Components, you can build modular and declarative pipelines.

This approach means:

- Trust & auditability: Every LLM output can be traced back through a reproducible graph.

- Safety in production: Automated evaluation loops catch drift and regressions before they matter.

- Scalable experimentation: The same declarative spec can power quick tests or robust, HIPAA/GxP-grade pipelines.

By treating LLM workflows like data pipelines: declarative, observable, and orchestrate, we can avoid the prompt spaghetti trap and build AI systems that meet the same reliability bar as the rest of the stack.

The Generative AI revolution is here, but so is the operational headache. For years, teams have matured their MLOps practices for traditional models, but the rapid adoption of LLMs has introduced a parallel, often chaotic, world of LLMOps. This results in fragmented toolchains, duplicated effort, and a state of "Ops Overload" that slows down innovation.

This session directly confronts this challenge. We will demonstrate how a unified platform like Google Cloud's Vertex AI can tame this complexity by providing a single control plane for the entire AI lifecycle.

Data is one of the most valuable assets in any organisation, but accessing and analysing it has been limited to technical experts. Business users often rely on predefined dashboards and data teams to extract insights, creating bottlenecks and slowing decision-making.

This is changing with the rise of Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG). These technologies are redefining how organisations interact with data, allowing users to ask complex questions in natural language and receive accurate, real-time insights without needing deep technical expertise.

In this session, I’ll explore how LLMs and RAG are driving true data democratisation by making analytics accessible to everyone, enabling real-time insights with AI-powered search and retrieval and overcoming traditional barriers like SQL, BI tool complexity, and rigid reporting structures.

Large Language Models (LLMs) are transformative, but static knowledge and hallucinations limit their direct enterprise use. Retrieval-Augmented Generation (RAG) is the standard solution, yet moving from prototype to production is fraught with challenges in data quality, scalability, and evaluation.

This talk argues the future of intelligent retrieval lies not in better models, but in a unified, data-first platform. We'll demonstrate how the Databricks Data Intelligence Platform, built on a Lakehouse architecture with integrated tools like Mosaic AI Vector Search, provides the foundation for production-grade RAG.

Looking ahead, we'll explore the evolution beyond standard RAG to advanced architectures like GraphRAG, which enable deeper reasoning within Compound AI Systems. Finally, we'll show how the end-to-end Mosaic AI Agent Framework provides the tools to build, govern, and evaluate the intelligent agents of the future, capable of reasoning across the entire enterprise.

Espresso AI uses two main techniques to run Snowflake workloads faster and cheaper: ML-based job scheduling and LLM-based query optimization. This talk will dive into the details behind both approaches.

It sounds simple: “Hey AI, refresh my Salesforce data.” But what really happens when that request travels through your stack?

Using Airbyte’s architecture as a model, this talk explores the complexity behind natural language data triggers - from spinning up connectors and handling credentials, to enforcing access controls and orchestrating safe, purpose-driven movement. We’ll introduce a unified framework for thinking about all types of data movement, from bulk ingestion to fine-grained activation - a model we’ve developed to bring clarity to a space crowded with overlapping terms and toolchains.

We’ll also explore how this foundation—and any modern data movement platform—must evolve for an AI-native world, where speed, locality, and security are non-negotiable. That includes new risks: leaking credentials into LLMs, or triggering unintended downstream effects from a single prompt.

We’ll close with a live demo: spinning up a local data plane and moving data via Airbyte—simply by chatting with a bot.

In this talk, we'll share how we're using computer vision to automate stock counting, right on the conveyor belt. We'll discuss the challenges we've faced with the hardware, software, and GenAI components, and we'll also review our own benchmark results for the various state-of-the-art models. Finally, we'll cover the practical aspects of GenAI deployment, including prompt optimization, preventing LLM "yapping," and creating a robust feedback loop for continuous improvement.

I’ll walk through my research in code analysis for web security, showing how graph-based static analysis can help surface privacy violations and security vulnerabilities in the Node.js ecosystem. I’ll introduce Cogna and our experience using LLMs to automatically generate tests, focusing on our approach, key lessons, and how it helps us detect bugs early in the development process.

This talk presents a technical case study of applying agentic AI systems to automate community operations at PyCon DE & PyData, treated as an open-source testbed. The key lesson is simple: AI only works when put on a leash. Reliable results required good architecture, a clear plan, and structured data models — from YAML and Pydantic schemas to reproducible pipelines with GitHub Actions. With that foundation, LLM agents supported logistics, FAQs, video processing, and scheduling; without it, they failed. By contrasting successes and failure modes across different coding agents, the talk demonstrates that robust design, validation, and controlled context are prerequisites for making agentic AI usable in real-world workflows.

In this session, we’ll take a closer look at the security risks that come with integrating LLMs into applications. LLMs can be powerful allies in cybersecurity — helping with detection, testing, and protection — but they can just as easily be exploited for attacks. We’ll explore key threats such as prompt injection, jailbreaking, and agent-specific vulnerabilities, and discuss why they are currently seen as the most pressing risks. Finally, we’ll look at defense strategies, from prompt-level safeguards to system-wide controls, and show how best practices can make a real difference in securing AI systems.

In the world of investment, inflation indicators play a pivotal role in planning for the future. Hedge funds in particular must grapple with text-based signals that provide deep insight into the future of stocks and industries. This talk will showcase how a combination of natural language processing, semantic embeddings, and cutting-edge large language models, can help transform those signals into bankable success.

Brought to You By: •⁠ Statsig ⁠ — ⁠ The unified platform for flags, analytics, experiments, and more. Statsig built a complete set of data tools that allow engineering teams to measure the impact of their work. This toolkit is SO valuable to so many teams, that OpenAI - who was a huge user of Statsig - decided to acquire the company, the news announced last week. Talk about validation! Check out Statsig. •⁠ Linear – The system for modern product development. Here’s an interesting story: OpenAI switched to Linear as a way to establish a shared vocabulary between teams. Every project now follows the same lifecycle, uses the same labels, and moves through the same states. Try Linear for yourself. — What does it take to do well at a hyper-growth company? In this episode of The Pragmatic Engineer, I sit down with Charles-Axel Dein, one of the first engineers at Uber, who later hired me there. Since then, he’s gone on to work at CloudKitchens. He’s also been maintaining the popular Professional programming reading list GitHub repo for 15 years, where he collects articles that made him a better programmer.  In our conversation, we dig into what it’s really like to work inside companies that grow rapidly in scale and headcount. Charles shares what he’s learned about personal productivity, project management, incidents, interviewing, plus how to build flexible skills that hold up in fast-moving environments.  Jump to interesting parts: • 10:41 – the reality of working inside a hyperscale company • 41:10 – the traits of high-performing engineers • 1:03:31 – Charles’ advice for getting hired in today’s job market We also discuss: • How to spot the signs of hypergrowth (and when it’s slowing down) • What sets high-performing engineers apart beyond shipping • Charles’s personal productivity tips, favorite reads, and how he uses reading to uplevel his skills • Strategic tips for building your resume and interviewing  • How imposter syndrome is normal, and how leaning into it helps you grow • And much more! If you’re at a fast-growing company, considering joining one, or looking to land your next role, you won’t want to miss this practical advice on hiring, interviewing, productivity, leadership, and career growth. — Timestamps (00:00) Intro (04:04) Early days at Uber as engineer #20 (08:12) CloudKitchens’ similarities with Uber (10:41) The reality of working at a hyperscale company (19:05) Tenancies and how Uber deployed new features (22:14) How CloudKitchens handles incidents (26:57) Hiring during fast-growth (34:09) Avoiding burnout (38:55) The popular Professional programming reading list repo (41:10) The traits of high-performing engineers  (53:22) Project management tactics (1:03:31) How to get hired as a software engineer (1:12:26) How AI is changing hiring (1:19:26) Unexpected ways to thrive in fast-paced environments (1:20:45) Dealing with imposter syndrome  (1:22:48) Book recommendations  (1:27:26) The problem with survival bias  (1:32:44) AI’s impact on software development  (1:42:28) Rapid fire round — The Pragmatic Engineer deepdives relevant for this episode: •⁠ Software engineers leading projects •⁠ The Platform and Program split at Uber •⁠ Inside Uber’s move to the Cloud •⁠ How Uber built its observability platform •⁠ From Software Engineer to AI Engineer – with Janvi Kalra — Production and marketing by ⁠⁠⁠⁠⁠⁠⁠⁠https://penname.co/⁠⁠⁠⁠⁠⁠⁠⁠. For inquiries about sponsoring the podcast, email [email protected].

Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe

The rapid evolution of AI, fueled by powerful Large Language Models (LLMs) and autonomous agents, is reshaping how we build, deploy, and manage AI systems. This presentation explores the critical intersection of MLOps and AI architecture, highlighting the paradigm shifts required to integrate LLMs and agents into production. We will address key architectural challenges, including scalability, observability, and security, while examining emerging MLOps practices such as robust data pipelines, model monitoring, and continuous optimization. Attendees will gain practical insights and actionable strategies to navigate the complexities of modern AI deployments, unlocking the full potential of LLMs and agents while ensuring operational excellence.

As AI evolves with powerful Large Language Models (LLMs) and autonomous agents, deploying and managing these systems requires new approaches. This presentation explores the crucial intersection of MLOps and AI architecture, highlighting the shift toward scalable, observable, and secure AI deployments. We’ll examine key architectural considerations for integrating LLMs and agents into production, alongside evolving MLOps practices such as robust data pipelines, model monitoring, and continuous optimization.

As AI agents become embedded in everyday workflows — from healthcare diagnostics to financial services chatbots — the line between human and machine continues to blur. This panel brings together industry leaders to tackle the tough questions:

• How do we trust AI agents in high-risk environments?

• What are the new rules of ownership and accountability when autonomous systems act on data?

• Is AI replacing or enhancing the human workforce — and how do we keep the balance right?

We'll unpack how AI agents are evolving across sectors, debate whether the current LLM paradigm is enough, and explore the new guardrails needed to futureproof agentic AI — without losing control.

The data engineer’s role is shifting in the AI era. With LLMs and agents as new consumers, the challenge moves from SQL and schemas to semantics, context engineering, and making databases LLM-friendly. This session explores how data engineers can design semantic layers, document relationships, and expose data through MCPs and AI interfaces. We’ll highlight new skills required, illustrate pipelines that combine offline and online LLM processing, and show how data can serve business users, developers, and AI agents alike.

While there has been much excitement about the potential of large language models (LLMs) to automate tasks that previously required human intelligence or creativity, many early projects have failed because of LLMs’ innate willingness to lie. This presentation explores these “hallucination” issues and proposes a solution.

By combining generative AI with more traditional symbolic computation, reliability can be maintained, explainability improved, and private knowledge and data injected. This talk will show simple examples of combining language-based thinking with computational thinking to generate solutions that neither could achieve on its own.

An example application of an AI scientific research assistant will be shown that brings together the ideas presented in a most demanding real-world task, where false information is not acceptable. This is a fast-evolving space with enormous potential—and we’re just getting started.

This session will explore the evolving role of data engineers. Data engineering is currently a bottleneck due to overwhelming requests and complex knowledge work. Maia acts as a "digital data engineer" or a "virtual data team" that amplifies productivity by 100x. It enables users, from skilled engineers to citizen data analysts, to author pipelines in natural business language. The session will demonstrate Maia's ability to accelerate mundane and advanced tasks,troubleshoot and debug pipelines in real-time, and generate high-quality, auditable pipelines using Matillion's proprietary, human-readable Data Pipeline Language (DPL), which avoids "spaghetti code" common with generic LLMs.

Are you ready to build the next generation of data-driven applications? This session demystifies the world of Autonomous Agents, explaining what they are and why they are the future of AI. We’ll dive into Google Cloud's comprehensive platform for creating and deploying these agents, from our multimodal data handling to the seamless integration of Gemini models. You will learn the principles behind building your own custom data agents and understand why Google Cloud provides the definitive platform for this innovation. Join us to gain the knowledge and tools needed to architect and deploy intelligent, self-sufficient data solutions.

See how LLMs inside Alteryx can instantly convert complex documents into clean, tabular data that’s ready for analysis.