Amazon Aurora DSQL is a serverless, distributed SQL database that uses decades of Amazon innovation and operational excellence. Join this session to dive into the design and key innovative technologies that make Aurora DSQL the ideal choice for new applications and applications which require the highest level of resilience. Learn how active-active architecture in Aurora DSQL delivers single- and multi-Region resiliency and virtually unlimited scale.

Learn more: More AWS events: https://go.aws/3kss9CP

Subscribe: More AWS videos: http://bit.ly/2O3zS75 More AWS events videos: http://bit.ly/316g9t4

ABOUT AWS: Amazon Web Services (AWS) hosts events, both online and in-person, bringing the cloud computing community together to connect, collaborate, and learn from AWS experts. AWS is the world's most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—are using AWS to lower costs, become more agile, and innovate faster.

AWSreInvent #AWSreInvent2025 #AWS

Participate in a live AMA with SQL Leadership following all the excitement from Microsoft Ignite 2025! This session invites attendees to submit questions ahead of time (highly recommended). Submit questions at https://aka.ms/SQL-AMA. Attendees can upvote questions to help prioritize topics most important to the community.

Learn how AWS is enhancing its SQL analytics offerings with new capabilities in Amazon Redshift and Amazon Athena. Discover how Redshift's AI-powered data warehousing capabilities are enabling customers to modernize their analytics workloads with enhanced performance and cost optimization. Explore Athena's latest features for interactively querying data directly in their Amazon S3 data lakes. This session showcases new features and real-world examples of how organizations are using these services to accelerate business insights while optimizing costs.

Learn more: More AWS events: https://go.aws/3kss9CP

Subscribe: More AWS videos: http://bit.ly/2O3zS75 More AWS events videos: http://bit.ly/316g9t4

ABOUT AWS: Amazon Web Services (AWS) hosts events, both online and in-person, bringing the cloud computing community together to connect, collaborate, and learn from AWS experts. AWS is the world's most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—are using AWS to lower costs, become more agile, and innovate faster.

AWSreInvent #AWSreInvent2025 #AWS

As organizations mature their generative AI strategy, more complex use cases emerge. Originally focused on unstructured data, organizations now require structured data as a source of knowledge for AI applications. This session will showcase a method for storing and accessing structured and unstructured data in tandem through an AI agent. Learn how to put text data into a vector store and create a text-to-SQL system with advanced knowledge bases, then efficiently direct user questions using AI agents for a smooth user experience. This approach demonstrates how to leverage both structured and unstructured data sources to enhance the capabilities of generative AI systems in enterprise environments.

Learn more: More AWS events: https://go.aws/3kss9CP

Subscribe: More AWS videos: http://bit.ly/2O3zS75 More AWS events videos: http://bit.ly/316g9t4

ABOUT AWS: Amazon Web Services (AWS) hosts events, both online and in-person, bringing the cloud computing community together to connect, collaborate, and learn from AWS experts. AWS is the world's most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—are using AWS to lower costs, become more agile, and innovate faster.

AWSreInvent #AWSreInvent2025 #AWS

Learn how AWS can help you with data integration and preparing data for analytics, machine learning (ML) and generative AI workloads. Explore new capabilities that enable your users to have controlled access to all relevant data, easily build and maintain scalable and resilient data pipelines, and enhance decision-making quality,all with exceptional price performance. See how zero-ETL and query federation can complement ETL and ELT data pipelines.

Learn more: More AWS events: https://go.aws/3kss9CP

Subscribe: More AWS videos: http://bit.ly/2O3zS75 More AWS events videos: http://bit.ly/316g9t4

ABOUT AWS: Amazon Web Services (AWS) hosts events, both online and in-person, bringing the cloud computing community together to connect, collaborate, and learn from AWS experts. AWS is the world's most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—are using AWS to lower costs, become more agile, and innovate faster.

AWSreInvent #AWSreInvent2025 #AWS

PostgreSQL performance isn’t magic—it’s engineering. In this code talk, we dive deep into practical tuning techniques to avoid common pitfalls that silently degrade performance by improving an underperforming PostgreSQL database workload. Learn how excessive indexes hurt write throughput, why HOT updates fail, and how vacuum behavior can stall your system. We’ll demonstrate how to use Query Plan Management (QPM) and pg_hint_plan for plan stability and decode wait events to uncover hidden bottlenecks. With real SQL examples and system insights, this session equips you to tune PostgreSQL for predictable, high-performance workloads.

Learn more: More AWS events: https://go.aws/3kss9CP

Subscribe: More AWS videos: http://bit.ly/2O3zS75 More AWS events videos: http://bit.ly/316g9t4

ABOUT AWS: Amazon Web Services (AWS) hosts events, both online and in-person, bringing the cloud computing community together to connect, collaborate, and learn from AWS experts. AWS is the world's most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—are using AWS to lower costs, become more agile, and innovate faster.

AWSreInvent #AWSreInvent2025 #AWS

At Qdrant Conference, builders, researchers, and industry practitioners shared how vector search, retrieval infrastructure, and LLM-driven workflows are evolving across developer tooling, AI platforms, analytics teams, and modern search research.

Andrey Vasnetsov (Qdrant) explained how Qdrant was born from the need to combine database-style querying with vector similarity search—something he first built during the COVID lockdowns. He highlighted how vector search has shifted from an ML specialty to a standard developer tool and why hosting an in-person conference matters for gathering honest, real-time feedback from the growing community.

Slava Dubrov (HubSpot) described how his team uses Qdrant to power AI Signals, a platform for embeddings, similarity search, and contextual recommendations that support HubSpot’s AI agents. He shared practical use cases like look-alike company search, reflected on evaluating agentic frameworks, and offered career advice for engineers moving toward technical leadership.

Marina Ariamnova (SumUp) presented her internally built LLM analytics assistant that turns natural-language questions into SQL, executes queries, and returns clean summaries—cutting request times from days to minutes. She discussed balancing analytics and engineering work, learning through real projects, and how LLM tools help analysts scale routine workflows without replacing human expertise.

Evgeniya (Jenny) Sukhodolskaya (Qdrant) discussed the multi-disciplinary nature of DevRel and her focus on retrieval research. She shared her work on sparse neural retrieval, relevance feedback, and hybrid search models that blend lexical precision with semantic understanding—contributing methods like Mini-COIL and shaping Qdrant’s search quality roadmap through end-to-end experimentation and community education.

Speakers

Andrey Vasnetsov Co-founder & CTO of Qdrant, leading the engineering and platform vision behind a developer-focused vector database and vector-native infrastructure. Connect: https://www.linkedin.com/in/andrey-vasnetsov-75268897/

Slava Dubrov Technical Lead at HubSpot working on AI Signals—embedding models, similarity search, and context systems for AI agents. Connect: https://www.linkedin.com/in/slavadubrov/

Marina Ariamnova Data Lead at SumUp, managing analytics and financial data workflows while prototyping LLM tools that automate routine analysis. Connect: https://www.linkedin.com/in/marina-ariamnova/

Evgeniya (Jenny) Sukhodolskaya Developer Relations Engineer at Qdrant specializing in retrieval research, sparse neural methods, and educational ML content. Connect: https://www.linkedin.com/in/evgeniya-sukhodolskaya/

Summary In this crossover episode, Max Beauchemin explores how multiplayer, multi‑agent engineering is transforming the way individuals and teams build data and AI systems. He digs into the shifting boundary between data and AI engineering, the rise of “context as code,” and how just‑in‑time retrieval via MCP and CLIs lets agents gather what they need without bloating context windows. Max shares hard‑won practices from going “AI‑first” for most tasks, where humans focus on orchestration and taste, and the new bottlenecks that appear — code review, QA, async coordination — when execution accelerates 2–10x. He also dives deep into Agor, his open‑source agent orchestration platform: a spatial, multiplayer workspace that manages Git worktrees and live dev environments, templatizes prompts by workflow zones, supports session forking and sub‑sessions, and exposes an internal MCP so agents can schedule, monitor, and even coordinate other agents.

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data managementData teams everywhere face the same problem: they're forcing ML models, streaming data, and real-time processing through orchestration tools built for simple ETL. The result? Inflexible infrastructure that can't adapt to different workloads. That's why Cash App and Cisco rely on Prefect. Cash App's fraud detection team got what they needed - flexible compute options, isolated environments for custom packages, and seamless data exchange between workflows. Each model runs on the right infrastructure, whether that's high-memory machines or distributed compute. Orchestration is the foundation that determines whether your data team ships or struggles. ETL, ML model training, AI Engineering, Streaming - Prefect runs it all from ingestion to activation in one platform. Whoop and 1Password also trust Prefect for their data operations. If these industry leaders use Prefect for critical workflows, see what it can do for you at dataengineeringpodcast.com/prefect.Data migrations are brutal. They drag on for months—sometimes years—burning through resources and crushing team morale. Datafold's AI-powered Migration Agent changes all that. Their unique combination of AI code translation and automated data validation has helped companies complete migrations up to 10 times faster than manual approaches. And they're so confident in their solution, they'll actually guarantee your timeline in writing. Ready to turn your year-long migration into weeks? Visit dataengineeringpodcast.com/datafold today for the details.Composable data infrastructure is great, until you spend all of your time gluing it together. Bruin is an open source framework, driven from the command line, that makes integration a breeze. Write Python and SQL to handle the business logic, and let Bruin handle the heavy lifting of data movement, lineage tracking, data quality monitoring, and governance enforcement. Bruin allows you to build end-to-end data workflows using AI, has connectors for hundreds of platforms, and helps data teams deliver faster. Teams that use Bruin need less engineering effort to process data and benefit from a fully integrated data platform. Go to dataengineeringpodcast.com/bruin today to get started. And for dbt Cloud customers, they'll give you $1,000 credit to migrate to Bruin Cloud.Your host is Tobias Macey and today I'm interviewing Maxime Beauchemin about the impact of multi-player multi-agent engineering on individual and team velocity for building better data systemsInterview IntroductionHow did you get involved in the area of data management?Can you start by giving an overview of the types of work that you are relying on AI development agents for?As you bring agents into the mix for software engineering, what are the bottlenecks that start to show up?In my own experience there are a finite number of agents that I can manage in parallel. How does Agor help to increase that limit?How does making multi-agent management a multi-player experience change the dynamics of how you apply agentic engineering workflows?Contact Info LinkedInLinks AgorApache AirflowApache SupersetPresetClaude CodeCodexPlaywright MCPTmuxGit WorktreesOpencode.aiGitHub CodespacesOnaThe intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA

In this lab you'll help a coffee shop unify their operational and analytical workloads with Cosmos DB in Microsoft Fabric. You'll blend operational data with curated sources using cross-database SQL, stream and visualize real-time POS events, and create a gold layer for personalization. Finally, you'll implement reverse ETL to Cosmos for lightning-fast serving and train a lightweight Spark notebook model to deliver the right offer at the right time before your customer’s order is ready.

Please RSVP and arrive at least 5 minutes before the start time, at which point remaining spaces are open to standby attendees.

Dive into the world of data analytics with 'SQL for Data Analytics'. This book takes you beyond simple query writing to teach you how to use SQL to analyze, interpret, and derive actionable insights from real-world data. By the end, you'll build technical skills that allow you to solve complex problems and demonstrate results using data. What this Book will help me do Understand how to create, manage, and utilize structured databases for analytics. Use advanced SQL techniques such as window functions and subqueries effectively. Analyze various types of data like geospatial, JSON, and time-series data in SQL. Apply statistical principles within the context of SQL for enhanced insights. Automate data workflows and presentations using SQL and Python integration. Author(s) The authors Jun Shan, Haibin Li, Matt Goldwasser, Upom Malik, and Benjamin Johnston bring together a wealth of knowledge in data analytics, database management, and applied statistics. Together, they aim to empower readers through clear explanations, practical examples, and a focus on real-world applicability. Who is it for? This book is aimed at data professionals and learners such as aspiring data analysts, backend developers, and anyone involved in data-driven decision-making processes. The ideal reader has a basic understanding of SQL and mathematics and is eager to extend their skills to tackle real-world data challenges effectively.

Master Microsoft Fabric from basics to advanced architectures with expert guidance to unify, secure, and scale analytics on real-world data platforms Key Features Build a complete data analytics platform with Microsoft Fabric Apply proven architectures, governance, and security strategies Gain real-world insights from five seasoned data experts Purchase of the print or Kindle book includes a free PDF eBook Book Description Microsoft Fabric is reshaping how organizations manage, analyze, and act on data by unifying ingestion, storage, transformation, analytics, AI, and visualization in a single platform. The Definitive Guide to Microsoft Fabric takes you from your very first workspace to building a secure, scalable, and future-proof analytics environment. You’ll learn how to unify data in OneLake, design data meshes, transform and model data, implement real-time analytics, and integrate AI capabilities. The book also covers advanced topics, such as governance, security, cost optimization, and team collaboration using DevOps and DataOps principles. Drawing on the real-world expertise of five seasoned professionals who have built and advised on platforms for startups, SMEs, and Europe’s largest enterprises, this book blends strategic insight with practical guidance. By the end of this book, you’ll have gained the knowledge and skills to design, deploy, and operate a Microsoft Fabric platform that delivers sustainable business value. What you will learn Understand Microsoft Fabric architecture and concepts Unify data storage and data governance with OneLake Ingest and transform data using multiple Fabric tools Implement real-time analytics and event processing Design effective semantic models and reports Integrate AI and machine learning into data workflows Apply governance, security, and compliance controls Optimize performance and costs at scale Who this book is for This book is for data engineers, analytics engineers, architects, and data analysts moving into platform design roles. It’s also valuable for technical leaders seeking to unify analytics in their organizations. You’ll need only a basic grasp of databases, SQL, and Python.

Unlock the future of data productivity with a hands-on exploration of AI-powered Copilots! We’ll dive into GitHub Copilot in SQL Server Management Studio (SSMS), Microsoft Copilot in Azure and Copilot for SQL Databases in Microsoft Fabric. With real-world demos and best practice tips along the way, you’ll leave ready to transform how you interact with SQL, anywhere.

A single database that performs at scale for both transactional and vector data is the best solution to meet the AI application needs of today’s fast paced enterprise. CockroachDB is that database. Learn how to build Real Time Agentic systems that leverage existing customer data that is optimized for performance, cost, and scale to serve the growing demand for enterprise scale AI applications.

In this session, we will explore how to integrate your SQL Server/Azure SQL into Microsoft Fabric, addressing one of the most common challenges for data professionals: bridging the gap between operational and analytical workloads. You will learn best practices for setting up seamless integration, understand the potential impacts on your current architecture, and discover how to achieve near real-time data synchronization without the need to build complex interfaces

Join Microsoft’s product team for a hands-on lab where you'll design and deploy an AI-powered application using SQL Database in Microsoft Fabric. This session dives into HTAP capabilities, enabling seamless transactional and analytical processing. You'll provision a SaaS-native SQL Database, use Copilot to generate schema and queries, and implement advanced patterns like RAG with vector search. Walk away with practical skills and a working solution you can apply immediately.

Please RSVP and arrive at least 5 minutes before the start time, at which point remaining spaces are open to standby attendees.

Learn more about migration and modernization of Windows Server, SQL Server, and .NET apps. Learn how to assess and migrate Windows Server and SQL Server workloads using Azure Migrate and Azure Database Migration Service (DMS). This lab covers discovery, dependency mapping, migration execution, and post-migration optimization as well as leveraging Microsoft Defender for cloud for securing workloads post migration.

Please RSVP and arrive at least 5 minutes before the start time, at which point remaining spaces are open to standby attendees.

SQL Server 2025 and Azure SQL DB now support vector search and embeddings, enabling semantic queries that go beyond keywords. This session shows how to store, index, and query embeddings using familiar T-SQL. You’ll learn how to power smarter search, recommendations, and natural language interfaces—all within your database. Real-world examples will highlight practical use cases. If you're ready to level up your queries, this is where it starts.

In this lab you'll help a coffee shop unify their operational and analytical workloads with Cosmos DB in Microsoft Fabric. You'll blend operational data with curated sources using cross-database SQL, stream and visualize real-time POS events, and create a gold layer for personalization. Finally, you'll implement reverse ETL to Cosmos for lightning-fast serving and train a lightweight Spark notebook model to deliver the right offer at the right time before your customer’s order is ready.

Please RSVP and arrive at least 5 minutes before the start time, at which point remaining spaces are open to standby attendees.

Discover how Azure Arc simplifies SQL Server migration with a unified, end-to-end experience. From automated assessments and real-time replication to seamless provisioning and cutover, Azure Arc accelerates modernization while delivering near-zero downtime. Learn how IT and business leaders can confidently migrate, optimize performance, and unlock the full potential of their data estate.

Are you using Copilot, or GitHub Copilot in SSMS?  Or maybe you want to try it, but aren't sure how it will help with admin tasks or writing T-SQL? Join us in the community hub for a conversation where you do the talking.  We'll have questions, we want to hear your answers.

Connection Pods accommodate up to 15 people. Please RSVP and arrive at least 5 minutes before the start time, at which point remaining spaces are open to standby attendees.