talk-data.com

In this session, we’ll walk through how Apache Flink was used to enable near real-time operational insights using manufacturing IIoT Data sets. The goal: deliver actionable KPIs to production teams with sub-30-second latency, using streaming data pipelines built Kafka, Flink and Grafana. We’ll cover the key architectural patterns that made this possible, including handling structured data joins, managing out-of-order events, and integrating with downstream systems like PostgreSQL and Grafana. We’ll also share real-world performance benchmarks, lessons learned from scaling tests, and practical considerations for deploying Flink in a production-grade, low-latency analytics pipeline. The session will also include a live demo

If you're building Flink-based solutions for time-sensitive operations—whether in manufacturing, IoT, or other domains—this talk will provide proven insights from the field.

DISCLAIMER We don't cater to attendees under the age of 18. If you want to host or speak at a meetup, please email [email protected]

In this session, we’ll walk through how Apache Flink was used to enable near real-time operational insights using manufacturing IIoT Data sets. The goal: deliver actionable KPIs to production teams with sub-30-second latency, using streaming data pipelines built Kafka, Flink and Grafana. We’ll cover the key architectural patterns that made this possible, including handling structured data joins, managing out-of-order events, and integrating with downstream systems like PostgreSQL and Grafana. We’ll also share real-world performance benchmarks, lessons learned from scaling tests, and practical considerations for deploying Flink in a production-grade, low-latency analytics pipeline. The session will also include a live demo

If you're building Flink-based solutions for time-sensitive operations—whether in manufacturing, IoT, or other domains—this talk will provide proven insights from the field.

According to Wikipedia, Infrastructure as Code is the process of managing and provisioning computer data center resources through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools. This also applies to resources and reference data, connector plugins, connector configurations, and stream processes to clean up the data.

In this talk, we are going to discuss the use cases based on the Network Rail Data Feeds, the scripts used to spin up the environment and cluster in the Confluent Cloud as well as the different components required for the ingress and processing of the data.

This particular environment is used as a teaching tool for Event Stream Processing for Kafka Streams, ksqlDB, and Flink. Some examples of further processing and visualisation will also be provided.

According to Wikipedia, Infrastructure as Code is the process of managing and provisioning computer data center resources through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools. This also applies to resources and reference data, connector plugins, connector configurations, and stream processes to clean up the data.

In this talk, we are going to discuss the use cases based on the Network Rail Data Feeds, the scripts used to spin up the environment and cluster in the Confluent Cloud as well as the different components required for the ingress and processing of the data.

This particular environment is used as a teaching tool for Event Stream Processing for Kafka Streams, ksqlDB, and Flink. Some examples of further processing and visualisation will also be provided.

Join us for this special event with the opportunity to learn all about open source Apache Kafka from the largest payments provider in Europe. Please note you must register before entry.

Date and Time: 🗓️ Tuesday 8th October, ⏰ 18:00 - 20:00 PM 🕘 Venue: Worldpay, The Walbrook Building, 25 Walbrook, London, EC4N 8AF Attending Brands: Worldpay, OSO & Confluent Networking: Pizza + Beer at the start

Schedule: 18:00: Welcome/doors Open, Food, drinks, networking 18:30 - 19:15: “Kafka: The challenges of technology adoption in a hybrid world” Paul Medcalf, Worldpay 19:15 - 20:00pm: “IaC in data streaming projects”, Sven Erik Knop, Confluent 20:00- 20:30pm: Additional Q&A & Networking

🎙️ \~Talk 1\~ Talk Title: Kafka: The challenges of technology adoption in a hybrid world

Summary: Most enterprises need to integrate different systems at some level. Kafka provides an effective message backbone to provide that enterprise integration transport.

When the enterprise implements hybrid systems across cloud and on-premise many factors can influence the problem of how best to build a Kafka platform. Combine the location diversity with extreme ranges of Kafka development team experience and building a fully operational platform to support diverse teams in multiple geographic areas is a significant challenge. The addition of a very agressive rollout timeline adds another dimension to the problem. In this presentation, we walk through how Worldpay addressed the problem and ultimately delivered an operational enterprise wide Kafka platform in just a few short months from the ground up.

🗣️ Speaker 1: Paul Medcalf - Hybrid Cloud Systems Architect - Worldpay Paul Medcalf is a senior Systems Architect at Worldpay, designing and engineering a range of hybrid cloud systems. His background in development, systems and middleware design underpins an engineering team delivering Kafka, Java, DotNet and PHP based application platforms using infrastructure as code.

🎙️ \~Talk 2\~ Talk Title: Infrastructure as Code is a well-known buzzword, but what does it mean for event streaming projects?

Summary: This talk tells the story of setting up an event streaming platform for a bootcamp in Confluent Cloud. This involves setting up the cluster, the topics, users and permissions, but it also includes uploading and configuring custom connectors to stream live events, providing reference data and use KSQL statements to prepare the events ready for consumption as data products - all driven by a single Terraform script as the single source of truth.

🗣️ Speaker 1: Sven Erik is a Staff Technical Instructor at Confluent, the enterprise event-streaming platform pioneer. Previously, he worked as Lead Consultant, Trainer and Evangelist at Perforce, deploying and maintaining large configuration management installations at various customer from leading gaming developers, financial institutions to chip manufacturers. He is a regular presenter at meetups and contributes to various open-source projects around Apache Kafka. In his spare time find him on a mountain hiking with his family and his dog, or sitting down playing board games with friends and family.

DISCLAIMER

* WE DO NOT CATER TO THOSE UNDER THE AGE OF 18.

*THIS IS AN OFFICE SPACE, PLEASE ENSURE TO CLEAN UP AFTER YOURSELVES AND TAKE CARE OF THIS SPACE. RESPECT THE SPACE AND THOSE IN IT. THIS INCLUDES: NO PHOTOS OF THE OFFICE, NO VANDALISM, GOING OUTSIDE OF THE MEETUP AREA ETC.