Abstract: Detecting problems as they happen is essential in today’s fast-moving, data-driven world. In this talk, you’ll learn how to build a flexible, real-time anomaly detection pipeline using Apache Kafka and Apache Flink, backed by statistical and machine learning models. We’ll start by demystifying what anomaly really means - exploring the different types (point, contextual, and collective anomalies) and the difference between unintentional issues and intentional outliers like fraud or abuse. Then, we’ll look at how anomaly detection is solved in practice: from classical statistical models like ARIMA to deep learning models like LSTM. You’ll learn how ARIMA breaks time series into AutoRegressive, Integrated, and Moving Average components, no math degree required (just a Python library). We’ll also uncover why forgetting is a feature, not a bug, when it comes to LSTMs, and how these models learn to detect complex patterns over time. Throughout, we’ll show how Kafka handles high-throughput streaming data and how Flink enables low-latency, stateful processing to catch issues as they emerge. You’ll leave knowing not just how these systems work, but when to use each type of model depending on your data and goals. Whether you're monitoring system health, tracking IoT devices, or looking for fraud in transactions, this talk will give you the foundations and tools to detect the unexpected - before it becomes a problem.
talk-data.com
Speaker
Olena Kutsenko
1
talks
Olena Kutsenko is a Staff Developer Advocate at Confluent and an expert in data streaming and analytics. With two decades in software engineering, she has built mission-critical applications, led high-performing teams, and driven large-scale technology adoption at Nokia, HERE Technologies, AWS, and Aiven. She advocates for real-time data processing and AI-driven applications, empowers developers, and regularly speaks at global conferences. She is an AWS Community Builder, a mentor, and a volunteer instructor at a nonprofit tech school.
Bio from: Crypto Streams to AI Predictions: Apache Kafka®, Apache Flink® & Apache Iceberg®
Filter by Event / Source
Talks & appearances
Showing 1 of 5 activities