talk-data.com talk-data.com

Event

PyData Amsterdam 2025

2025-09-24 – 2025-09-26 PyData

Activities tracked

119

Sessions & talks

Showing 26–50 of 119 · Newest first

Search within this event →

Composable Pipelines for ML: Automating Feature Engineering with Hopsworks’ Brewer

2025-09-26
talk

Operationalizing ML isn’t just about models — it’s about moving and engineering data. At Hopsworks, we built a composable AI pipeline builder (Brewer) based on two principles: Tasks and Data Sources. This lets users define workflows that automatically analyse, clean, create and update feature groups, without glue code or brittle scheduling logic.

In this talk, we’ll show how Brewer drives the automation of feature engineering, enabling reproducible, declarative pipelines that respond to changes in upstream data. We’ll explore how this fits into broader ML workflows, from ingestion to feature materialization, and how it integrates with warehouses, streams, and file-based systems.

How to Keep Your LLM Chatbots Real: A Metrics Survival Guide

How to Keep Your LLM Chatbots Real: A Metrics Survival Guide

2025-09-26 Watch
talk

In this brave new world of vibe coding and YOLO-to-prod mentality, let’s take a step back and keep things grounded (pun intended). None of us would ever deploy a classical ML model to production without clearly defined metrics and proper evaluation, so let's talk about methodologies for measuring performance of LLM-powered chatbots. Think of retriever recall, answer relevancy, correctness, faithfulness and hallucination rates. With the wild west of metric standards still in full swing, I’ll guide you through the challenges of curating a synthetic test set, and selecting suitable metrics and open-source packages that help evaluating your use case. Everything is possible, from simple LLM-as-a-judge approaches like those inherent to many packages like MLFLow now up to complex multi-step quantification approaches with Ragas. If you work in the GenAI space or with LLM-powered chatbots, this session is for you! Prior or background knowledge is of advantage, but not required.

Kafka Internals I Wish I Knew Sooner: The Non-Boring Truths

Kafka Internals I Wish I Knew Sooner: The Non-Boring Truths

2025-09-26 Watch
talk

Most of us start with Kafka by building a simple producer/consumer demo. It just works — until it doesn’t. Suddenly, disk space isn’t freed up after data “expires,” rebalances loop endlessly during deploys, and strange errors about missing leaders clog your logs. In the panic, we dive into Kafka’s ocean of config options — hoping something will stick. Sound familiar?

This talk is a collection of hard-won lessons — not flashy tricks, but the kind of insights you only gain after operating Kafka in production for years. You’ll walk away with mental models that make Kafka’s internal behavior more predictable and less surprising.

We’ll cover: - Storage internals: Why expired data doesn’t always free space — and how Kafka actually reclaims disk - Transactions & delivery semantics: What “exactly-once” really means, and when it silently downgrades - Consumer group rebalancing: Why rebalances loop, and how the controller’s hidden behavior affects them

If you’ve used Kafka — or plan to — these insights will save you hours of frustration and debugging. A basic understanding of partitions, replication, and Kafka’s general architecture will help get the most out of this session.

Optimal Observability: Partitioning Data into Time-Series for Enhanced Anomaly Detection and Improved Monitoring Coverage

2025-09-26
talk

This talk presents a principled methodology for partitioning item-level data into homogeneous time-series, with the objective of maximizing monitoring coverage and improving the detection of anomalies and drifts. We discuss the theoretical underpinnings of clustering algorithms for this task and describe practical algorithms enabling efficient search for optimal partitioning. We exemplify our approach with a real-world application in large-scale monitoring environments from the online payment domain.

Data that Keeps Our Energy in Balance - From churn prediction with deep learning to real-time trading systems

2025-09-26
talk

This talk explores how data science helps balance energy systems in the face of demand volatility, generation volatility, and the push for sustainability. We’ll dive into two technical case studies: churn prediction using survival models, and the design of a high-availability real-time trading system on Databricks. These examples illustrate how data can support operational resilience and sustainability efforts in the energy sector.

Declarative Feature Engineering: Bridging Spark and Flink with a Unified DSL

2025-09-26
talk

Building ML features at scale shouldn’t require every ML Scientist to become an expert in Spark or Flink. At Adyen, the Feature Platform team built a Python-based DSL that lets data scientists define features declaratively — while automatically generating the necessary batch or real-time pipelines behind the scenes.

Detection of Unattended Objects in Public Spaces using AI

2025-09-26
talk

This talk presents an end-to-end solution for detecting unattended objects in public transport hubs to enhance social security. The project, developed in a three-week challenge, focuses on proactively identifying unattended items using existing camera infrastructure. We will cover the entire pipeline, from data anonymization and preprocessing to building a data labeling platform, object detection with YOLO, and tracking objects over time. The presentation will also discuss the evaluation of the system.

Scaling Trust: A practical guide on evaluating LLMs and Agents

2025-09-26
talk

Recently, the integration of Generative AI (GenAI) technologies into both our personal and professional lives has surged. In most organizations, the deployment of GenAI applications is on the rise, and this trend is expected to continue in the foreseeable future. Evaluating GenAI systems presents unique challenges not present in traditional ML. The main peculiarity is the absence of ground truth for textual metrics such as: text clarity, location extraction accuracy, factual accuracy and so on. Nevertheless the non-negligible model serving cost demands an even more thorough evaluation of the system to be deployed in production.

Defining the metric ground truth is a costly and time consuming process requiring human annotation. To address this, we are going to present how to evaluate LLM-based applications by leveraging LLMs themselves as evaluators. Moreover we are going to outline the complexities and evaluation methods for LLM-based Agents which operate with autonomy and present further evaluation challenges. Lastly, we will explore the critical role of evaluation in the GenAI lifecycle and outline the steps taken to integrate these processes seamlessly.

Whether you are an AI practitioner, user or enthusiast, join us to gain insights into the future of GenAI evaluation and its impact on enhancing application performance.

Model Context Protocol: Principles and Practice

Model Context Protocol: Principles and Practice

2025-09-26 Watch
talk

Large‑language‑model agents are only as useful as the context and tools they can reach.

Anthropic’s Model Context Protocol (MCP) proposes a universal, bidirectional interface that turns every external system—SQL databases, Slack, Git, web browsers, even your local file‑system—into first‑class “context providers.”

In just 30 minutes we’ll step from high‑level buzzwords to hands‑on engineering details:

  • How MCP’s JSON‑RPC message format, streaming channels, and version‑negotiation work under the hood.
  • Why per‑tool sandboxing via isolated client processes hardens security (and what happens when an LLM tries rm ‑rf /).
  • Techniques for hierarchical context retrieval that stretch a model’s effective window beyond token limits.
  • Real‑world patterns for accessing multiple tools—Postgres, Slack, GitHub—and plugging MCP into GenAI applications.

Expect code snippets and lessons from early adoption.

You’ll leave ready to wire your own services into any MCP‑aware model and level‑up your GenAI applications—without the N×M integration nightmare.

Optimize the Right Thing: Cost-Sensitive Classification in Practice

Optimize the Right Thing: Cost-Sensitive Classification in Practice

2025-09-26 Watch
talk

Not all mistakes in machine learning are equal—a false negative in fraud detection or medical diagnosis can be far costlier than a false positive. Cost-sensitive learning helps navigate these trade-offs by incorporating error costs into the training process, leading to smarter decision-making. This talk introduces Empulse, an open-source Python package that brings cost-sensitive learning into scikit-learn. Attendees will learn why standard models fall short in cost-sensitive scenarios and how to build better classifiers with Scikit-Learn and Empulse.

Untitled13.ipynb

Untitled13.ipynb

2025-09-26 Watch
talk

For well over a decade, Python notebooks revolutionized our field. They gave us so much creative freedom and dramatically lowered the entry barrier for newcomers. Yet despite all this ... it has been a decade! And the notebook is still in roughly the same form factor.

So what if we allow ourselves to rethink notebooks ... really rethink it! What features might we come up with? Can we make the notebook understand datasources? What about LLMs? Can we generate widgets on the fly? What if we make changes to Python itself?

This presentation will be a stream of demos that help paint a picture of what the future might hold. I will share my latest work in the anywidget/marimo ecosystem as well as some new hardware integrations.

The main theme that I will work towards: if you want better notebooks, reactive Python might very well be the future.

Coffee break

2025-09-26
talk

Coffee break

2025-09-26
talk

Coffee break

2025-09-26
talk

Coffee break

2025-09-26
talk

Coffee Break

2025-09-26
talk

Coffee Break

2025-09-26
talk
Image processing, artificial intelligence, and autonomous systems

Image processing, artificial intelligence, and autonomous systems

2025-09-26 Watch
talk

In this talk, an overview of the field of image processing and the impact of artificial intelligence on this field are shown. Starting from the different tasks that can be performed with image processing, solutions using different AI technologies are shown, including the use of generative AI. Finally, the effect of AI for autonomous systems, and the challenges that are faced are discussed.

Registration and breakfast

2025-09-26
talk

Registration and breakfast

2025-09-26
talk

Registration and breakfast

2025-09-26
talk

Registration and breakfast

2025-09-26
talk

Registration and breakfast

2025-09-26
talk

Registration and breakfast

2025-09-26
talk

Social Event

2025-09-25
talk