Summary In this episode of the Data Engineering Podcast Professor Paul Groth, from the University of Amsterdam, talks about his research on knowledge graphs and data engineering. Paul shares his background in AI and data management, discussing the evolution of data provenance and lineage, as well as the challenges of data integration. He explores the impact of large language models (LLMs) on data engineering, highlighting their potential to simplify knowledge graph construction and enhance data integration. The conversation covers the evolving landscape of data architectures, managing semantics and access control, and the interplay between industry and academia in advancing data engineering practices, with Paul also sharing insights into his work with the intelligent data engineering lab and the importance of human-AI collaboration in data engineering pipelines.

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data managementData migrations are brutal. They drag on for months—sometimes years—burning through resources and crushing team morale. Datafold's AI-powered Migration Agent changes all that. Their unique combination of AI code translation and automated data validation has helped companies complete migrations up to 10 times faster than manual approaches. And they're so confident in their solution, they'll actually guarantee your timeline in writing. Ready to turn your year-long migration into weeks? Visit dataengineeringpodcast.com/datafold today for the details.Your host is Tobias Macey and today I'm interviewing Paul Groth about his research on knowledge graphs and data engineeringInterview IntroductionHow did you get involved in the area of data management?Can you start by describing the focus and scope of your academic efforts?Given your focus on data management for machine learning as part of the INDELab, what are some of the developing trends that practitioners should be aware of?ML architectures / systems changing (matteo interlandi) GPUs for data mangementYou have spent a large portion of your career working with knowledge graphs, which have largely been a niche area until recently. What are some of the notable changes in the knowledge graph ecosystem that have resulted from the introduction of LLMs?What are some of the other ways that you are seeing LLMs change the methods of data engineering?There are numerous vague and anecdotal references to the power of LLMs to unlock value from unstructured data. What are some of the realitites that you are seeing in your research?A majority of the conversations in this podcast are focused on data engineering in the context of a business organization. What are some of the ways that management of research data is disjoint from the methods and constraints that are present in business contexts?What are the most interesting, innovative, or unexpected ways that you have seen LLM used in data management?What are the most interesting, unexpected, or challenging lessons that you have learned while working on data engineering research?What do you have planned for the future of your research in the context of data engineering, knowledge graphs, and AI?Contact Info WebsiteemailParting Question From your perspective, what is the biggest gap in the tooling or technology for data management today?Closing Announcements Thank you for listening! Don't forget to check out our other shows. Podcast.init covers the Python language, its community, and the innovative ways it is being used. The AI Engineering Podcast is your guide to the fast-moving world of building AI systems.Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.Links INDELabData ProvenanceElsevierSIGMOD 2025Digital TwinKnowledge GraphWikiDataKuzuDBPodcast Episodedata.worldPodcast EpisodeGraphRAGSPARQLSemantic WebGQL == Graph Query LanguageCypherAmazon NeptuneRDF == Resource Description FrameworkSwellDBFlockMTLDuckDBPodcast EpisodeMatteo InterlandiPaolo PapottiNeuromorphic ComputingPoint CloudsLongform.aiBASIL DBThe intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA

Abstract: Detecting problems as they happen is essential in today’s fast-moving, data-driven world. In this talk, you’ll learn how to build a flexible, real-time anomaly detection pipeline using Apache Kafka and Apache Flink, backed by statistical and machine learning models. We’ll start by demystifying what anomaly really means - exploring the different types (point, contextual, and collective anomalies) and the difference between unintentional issues and intentional outliers like fraud or abuse. Then, we’ll look at how anomaly detection is solved in practice: from classical statistical models like ARIMA to deep learning models like LSTM. You’ll learn how ARIMA breaks time series into AutoRegressive, Integrated, and Moving Average components, no math degree required (just a Python library). We’ll also uncover why forgetting is a feature, not a bug, when it comes to LSTMs, and how these models learn to detect complex patterns over time. Throughout, we’ll show how Kafka handles high-throughput streaming data and how Flink enables low-latency, stateful processing to catch issues as they emerge. You’ll leave knowing not just how these systems work, but when to use each type of model depending on your data and goals. Whether you're monitoring system health, tracking IoT devices, or looking for fraud in transactions, this talk will give you the foundations and tools to detect the unexpected - before it becomes a problem.

Explore AI and Machine Learning fundamentals, tools, and applications in this beginner-friendly guide. Learn to build models in Python and understand AI ethics. Key Features Covers AI fundamentals, Machine Learning, and Python model-building Provides a clear, step-by-step guide to learning AI techniques Explains ethical considerations and the future role of AI in society Book Description This book is an ideal starting point for anyone interested in Artificial Intelligence and Machine Learning. It begins with the foundational principles of AI, offering a deep dive into its history, building blocks, and the stages of development. Readers will explore key AI concepts and gradually transition to practical applications, starting with machine learning algorithms such as linear regression and k-nearest neighbors. Through step-by-step Python tutorials, the book helps readers build and implement models with hands-on experience. As the book progresses, readers will dive into advanced AI topics like deep learning, natural language processing (NLP), and generative AI. Topics such as recommender systems and computer vision demonstrate the real-world applications of AI technologies. Ethical considerations and privacy concerns are also addressed, providing insight into the societal impact of these technologies. By the end of the book, readers will have a solid understanding of both the theory and practice of AI and Machine Learning. The final chapters provide resources for continued learning, ensuring that readers can continue to grow their AI expertise beyond the book. What you will learn Understand key AI and ML concepts and how they work together Build and apply machine learning models from scratch Use Python to implement AI techniques and improve model performance Explore essential AI tools and frameworks used in the industry Learn the importance of data and data preparation in AI development Grasp the ethical considerations and the future of AI in work Who this book is for This book is ideal for beginners with no prior knowledge of AI or Machine Learning. It is tailored to those who wish to dive into these topics but are not yet familiar with the terminology or techniques. There are no prerequisites, though basic programming knowledge can be helpful. The book caters to a wide audience, from students and hobbyists to professionals seeking to transition into AI roles. Readers should be enthusiastic about learning and exploring AI applications for the future.

Hands-on Python workshop for ages 12-18 featuring the Emoji Master Challenge. Students progress through levels, including Level 3 with a rose emoji and time-saving shortcuts, and Level 5 with revealing superheroes; the session begins with a Python introduction and ends with a reveal of the students' emoji-created superheroes.

Formation immersive et orientée pratique pour créer et déployer un modèle ML capable de prédire le prix d'une voiture, avec manipulation de données et déploiement en production (Python, TensorFlow, PyTorch, Flask, Ngrok).

Formation immersive et orientée pratique sur la création et le déploiement d'une IA capable de prédire le prix d’une voiture. Approche hands-on avec Python, TensorFlow, PyTorch, Flask et Ngrok.

How to create a customized running plan using Python, Elasticsearch, and Agno\nWhen working out, deciding what workout to do next to achieve a specific goal can be challenging. This talk will guide you through creating a four-week workout plan as a Notion page with to-do list items tailored to your workout history, helping you run a faster 5K

When working out, deciding what workout to do next to achieve a specific goal can be challenging. This talk will guide you through creating a four-week workout plan as a Notion page with to-do list items tailored to your workout history, helping you run a faster 5K

Async Python for Data Science: Speeding Up IO - Bound Workflows\nMost Python scripts in data science are synchronous — fetching one record at a time, waiting for APIs, or slowly scraping websites. In this talk, we’ll introduce Python’s asyncio ecosystem and show how it transforms IO - heavy data workflows. You'll see how httpx , aiofiles , and async constructs speed up tasks like web scraping and batch API calls. We’ll compare async vs threading, walk through a real - world case study, and wrap with performance benchmarks that demonstrate async's value.\nKeywords: p ython 3.x , AsyncIO, Web Scraping, API, Concurrency, Performance, Optimization

Most Python scripts in data science are synchronous — fetching one record at a time, waiting for APIs, or slowly scraping websites. In this talk, we’ll introduce Python’s asyncio ecosystem and show how it transforms IO - heavy data workflows. You'll see how httpx , aiofiles , and async constructs speed up tasks like web scraping and batch API calls. We’ll compare async vs threading, walk through a real - world case study, and wrap with performance benchmarks that demonstrate async's value.

When working with Large Language Models (LLMs), how do we ensure a probabilistic blob of text is something our code can actually use? In this talk, we explore how Pydantic emerged at a perfect moment exactly for this task; bridging Python's flexibility with the structured data needs of modern AI applications. We will introduce Pydantic and then demonstrate practical applications of it; from prompt engineering and parsing responses, to example of robust function calling and tool chaining via APIs.

Hands-on example: Bayesian linear regression; prior predictive checks; posterior sampling with NUTS; basic model diagnostics; posterior predictive checks.

Recommended installation procedure; PyMC's computational backends; troubleshooting common installation issues; setting up development environments.

Overview of PyMC and its role in the Python data science ecosystem; understanding probabilistic vs Bayesian approaches; a survey of the probabilistic programming landscape; real-world applications and case studies.

Model contexts and random variables; prior and likelihood specification; working with observed data; PyMC's relationship with ArviZ.

Summary In this episode of the Data Engineering Podcast Lucas Thelosen and Drew Gilson from Gravity talk about their development of Orion, an autonomous data analyst that bridges the gap between data availability and business decision-making. Lucas and Drew share their backgrounds in data analytics and how their experiences have shaped their approach to leveraging AI for data analysis, emphasizing the potential of AI to democratize data insights and make sophisticated analysis accessible to companies of all sizes. They discuss the technical aspects of Orion, a multi-agent system designed to automate data analysis and provide actionable insights, highlighting the importance of integrating AI into existing workflows with accuracy and trustworthiness in mind. The conversation also explores how AI can free data analysts from routine tasks, enabling them to focus on strategic decision-making and stakeholder management, as they discuss the future of AI in data analytics and its transformative impact on businesses.

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data managementData migrations are brutal. They drag on for months—sometimes years—burning through resources and crushing team morale. Datafold's AI-powered Migration Agent changes all that. Their unique combination of AI code translation and automated data validation has helped companies complete migrations up to 10 times faster than manual approaches. And they're so confident in their solution, they'll actually guarantee your timeline in writing. Ready to turn your year-long migration into weeks? Visit dataengineeringpodcast.com/datafold today for the details.Your host is Tobias Macey and today I'm interviewing Lucas Thelosen and Drew Gilson about the engineering and impact of building an autonomous data analystInterview IntroductionHow did you get involved in the area of data management?Can you describe what Orion is and the story behind it?How do you envision the role of an agentic analyst in an organizational context?There have been several attempts at building LLM-powered data analysis, many of which are essentially a text-to-SQL interface. How have the capabilities and architectural patterns grown in the past ~2 years to enable a more capable system?One of the key success factors for a data analyst is their ability to translate business questions into technical representations. How can an autonomous AI-powered system understand the complex nuance of the business to build effective analyses?Many agentic approaches to analytics require a substantial investment in data architecture, documentation, and semantic models to be effective. What are the gradations of effectiveness for autonomous analytics for companies who are at different points on their journey to technical maturity?Beyond raw capability, there is also a significant need to invest in user experience design for an agentic analyst to be useful. What are the key interaction patterns that you have found to be helpful as you have developed your system?How does the introduction of a system like Orion shift the workload for data teams?Can you describe the overall system design and technical architecture of Orion?How has that changed as you gained further experience and understanding of the problem space?What are the most interesting, innovative, or unexpected ways that you have seen Orion used?What are the most interesting, unexpected, or challenging lessons that you have learned while working on Orion?When is Orion/agentic analytics the wrong choice?What do you have planned for the future of Orion?Contact Info LucasLinkedInDrewLinkedInParting Question From your perspective, what is the biggest gap in the tooling or technology for data management today?Closing Announcements Thank you for listening! Don't forget to check out our other shows. Podcast.init covers the Python language, its community, and the innovative ways it is being used. The AI Engineering Podcast is your guide to the fast-moving world of building AI systems.Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.Links OrionLookerGravityVBA == Visual Basic for ApplicationsText-To-SQLOne-shotLookMLData GrainLLM As A JudgeGoogle Large Time Series ModelThe intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA

Hands-on Emoji Master Challenge session designed for aspiring programmers aged 12-18. Begin with a Python introduction and progress through level-based tasks (e.g., Level 3 displays a rose emoji 10 times; Level 5 conceals a superhero using emojis). The workshop ends with a reveal of the superheroes created by the students and introduces practical Python concepts, interactive games, and graphics with PyGame and Turtle.

Summary In this episode of the Data Engineering Podcast Andy Warfield talks about the innovative functionalities of S3 Tables and Vectors and their integration into modern data stacks. Andy shares his journey through the tech industry and his role at Amazon, where he collaborates to enhance storage capabilities, discussing the evolution of S3 from a simple storage solution to a sophisticated system supporting advanced data types like tables and vectors crucial for analytics and AI-driven applications. He explains the motivations behind introducing S3 Tables and Vectors, highlighting their role in simplifying data management and enhancing performance for complex workloads, and shares insights into the technical challenges and design considerations involved in developing these features. The conversation explores potential applications of S3 Tables and Vectors in fields like AI, genomics, and media, and discusses future directions for S3's development to further support data-driven innovation.

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data managementTired of data migrations that drag on for months or even years? What if I told you there's a way to cut that timeline by up to 6x while guaranteeing accuracy? Datafold's Migration Agent is the only AI-powered solution that doesn't just translate your code; it validates every single data point to ensure perfect parity between your old and new systems. Whether you're moving from Oracle to Snowflake, migrating stored procedures to dbt, or handling complex multi-system migrations, they deliver production-ready code with a guaranteed timeline and fixed price. Stop burning budget on endless consulting hours. Visit dataengineeringpodcast.com/datafold to book a demo and see how they're turning months-long migration nightmares into week-long success stories.Your host is Tobias Macey and today I'm interviewing Andy Warfield about S3 Tables and VectorsInterview IntroductionHow did you get involved in the area of data management?Can you describe what your goals are with the Tables and Vector features of S3?How did the experience of building S3 Tables inform your work on S3 Vectors?There are numerous implementations of vector storage and search. How do you view the role of S3 in the context of that ecosystem?The most directly analogous implementation that I'm aware of is the Lance table format. How would you compare the implementation and capabilities of Lance with what you are building with S3 Vectors?What opportunity do you see for being able to offer a protocol compatible implementation similar to the Iceberg compatibility that you provide with S3 Tables?Can you describe the technical implementation of the Vectors functionality in S3?What are the sources of inspiration that you looked to in designing the service?Can you describe some of the ways that S3 Vectors might be integrated into a typical AI application?What are the most interesting, innovative, or unexpected ways that you have seen S3 Tables/Vectors used?What are the most interesting, unexpected, or challenging lessons that you have learned while working on S3 Tables/Vectors?When is S3 the wrong choice for Iceberg or Vector implementations?What do you have planned for the future of S3 Tables and Vectors?Contact Info LinkedInParting Question From your perspective, what is the biggest gap in the tooling or technology for data management today?Closing Announcements Thank you for listening! Don't forget to check out our other shows. Podcast.init covers the Python language, its community, and the innovative ways it is being used. The AI Engineering Podcast is your guide to the fast-moving world of building AI systems.Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.Links S3 TablesS3 VectorsS3 ExpressParquetIcebergVector IndexVector DatabasepgvectorEmbedding ModelRetrieval Augmented GenerationTwelveLabsAmazon BedrockIceberg REST CatalogLog-Structured Merge TreeS3 MetadataSentence TransformerSparkTrinoDaftThe intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA

Formation immersive et orientée pratique sur la création et le déploiement d'une IA capable de prédire le prix d'une voiture. Manipulation de données, création d'un modèle de régression, et mise en production avec Python, TensorFlow, PyTorch, Flask et Ngrok. Animation en direct par un formateur expert, approche interactive et pratique.

Formation pratique guidée par un formateur expert. Manipuler des données, créer un modèle de régression et le mettre en production avec Python, TensorFlow, PyTorch, Flask et Ngrok. Approche progressive et interactive pour transformer vos compétences en programmation en solutions d’IA.