talk-data.com
Activities & events
| Title & Speakers | Event |
|---|---|
|
Kroxylicious, the snappy open-source proxy for Apache Kafka
2025-05-15 · 19:50
Anatoly Zelenin
– Founder and Chief Trainer
@ DataFlow Academy
Proxies are a powerful architectural pattern commonly used in other application-layer protocols like HTTP, but they remain underutilized in the Kafka ecosystem. Kroxylicious is a groundbreaking, early-stage project that aims to change this by making it easier to develop Kafka proxies with minimal effort. While some organizations have built their own Kafka-aware proxies to address specific challenges, these solutions often remain closed-source and highly specialized. Kroxylicious tackles this problem by offering a set of pre-built capabilities and common features out-of-the-box, enabling developers to concentrate on the custom logic their applications need. In this session, I will introduce Kroxylicious, highlight its potential use cases, and demonstrate how it can simplify Kafka proxy development, reduce complexity, and unlock new possibilities for real-time data processing. |
|
|
How Do You Architect a Kafka Streams Topology?
2025-05-15 · 19:00
Hartmut Armbruster
– Software Architect | Developer
Struggled with the complexity of designing Kafka Streams applications? Without sufficient up-front architecture work, it’s all too easy to stumble into misunderstandings, rework, or outright failure. Although standards like UML and C4 model have guided software designs for years, stream processing has lacked its own visual framework—until now. KSTD (Kafka Streams Topology Design) introduces an open standard and component library for describing and visualising Kafka Stream Topologies with Excalidraw. Simple design principles ensure teams can keep diagrams simple yet include important details, build trust in their designs, and streamline the development lifecycle. You will learn how standardised diagrams support team alignment, and how KSTD fosters consistent and clear communication for Kafka Streams. Design up-front, avoid mistakes, save time, and build trust. |
|
|
In-PERSON: Apache Kafka® x Kong Meetup in London
2025-03-27 · 18:00
Join us for an Apache Kafka® x Kong meetup on Thursday, March 27th from 6:00pm in London hosted by TurinTech AI! 📍Venue: TurinTech AI 1 Finsbury Ave, City of London, London EC2M 2PF · London IMPORTANT: If you RSVP on the this page, you don't need to RSVP on Kong London meetup page. 🗓 Agenda:
💡 Speaker One: Johannes Brännström, Staff Solutions Engineer, Kong Title of Talk: Extending the worlds most used API platform to Kafka -- Why it matters, what's here today, what's coming and how to learn more. Abstract: In this talk, we’ll explore how Kong facilitates the way Kafka users interact with both synchronous and asynchronous APIs. Through the lens of example companies, we'll showcase how Kong helps manage and optimize their API ecosystems, balancing the needs for real-time event streaming with traditional API patterns. Attendees will gain an understanding of Kong’s current capabilities for Kafka integration, highlighting key features that enable smooth API management, enhanced scalability, and reliability. We'll also offer ways for you to get first-hand experience. Looking ahead, we’ll share exciting developments on the horizon, including upcoming features, enhancements that will further empower Kafka users as well as learn more. Bio: Joining Kong from MongoDB & Red Hat Johannes is passionate about agile, modern application architecture and continuous delivery in all areas required to build great software. 💡 Speaker Two: Harry Kimpel, Principal Developer Relations Engineer, New Relic Title of Talk: Unlock full end-to-end tracing through Kong and Kafka with OpenTelemetry Abstract: Ideally, you should be using distributed tracing to trace requests through your system, but Kafka decouples producers and consumers, which means there are no direct transactions to trace between them. Kafka also uses asynchronous processes, which have implicit, not explicit, dependencies. Another key component of such an architecture is securing and exposing endpoints with a powerful platform such as Kong. All of this makes it challenging to understand how your microservices are working together. However, it is possible to observe your Kong Gateway interactions and Kafka-based applications with distributed tracing and OpenTelemetry. You can then analyze and visualize your traces in an open source distributed tracing tool like Jaeger or a full observability platform like New Relic. In this talk I will leverage a simple application to show how you can achieve this. Bio: Passionate software craftsman with 25+ years experience in a broad spectrum of development technologies and platforms. Mostly focusing on .NET/C#, the Microsoft development stack and cloud-native software architectures. Key focus also on Microsoft Azure as well as Amazon Web Services (AWS) and Google Cloud Platform (GCP). Observability, monitoring and securing these environments are key aspects. Currently extremely interested in the Distributed Application Runtime (Dapr). *** DISCLAIMER NOTE: We are unable to cater for any attendees under the age of 18. If you would like to speak or host our next event please let us know! [email protected] |
In-PERSON: Apache Kafka® x Kong Meetup in London
|
|
In-PERSON: Apache Kafka® x Kong Meetup in London
2025-03-27 · 18:00
We are thrilled to welcome you to our in-person Meetup in London in collaboration with Confluent! The event will be hosted by TurinTech AI. When March 27, 18:00 - 21:00 Where 1 Finsbury Ave, London EC2M 2PF Agenda 18:00 - 18:30 Doors open 18:00 - 18:30 Food & Drinks and Networking 18:30 - 19:00 TurinTech AI / Welcome 19:00 - 19:45 [Talk 1] Extending the worlds most used API platform to Kafka -- Why it matters, what's here today, what's coming and how to learn more by Johannes Brännström (Staff Solutions Engineer, Kong) 19:45 - 20:30 [Talk 2] Unlock full end-to-end tracing through Kong and Kafka with OpenTelemetry from Harry Kimpel (Principal Developer Relations Engineer, New Relic) 20:30 - 21:00 Additional Q&A and Networking Talks 1) Extending the worlds most used API platform to Kafka -- Why it matters, what's here today, what's coming and how to learn more In this talk, we’ll explore how Kong facilitates the way Kafka users interact with both synchronous and asynchronous APIs. Through the lens of example companies, we'll showcase how Kong helps manage and optimize their API ecosystems, balancing the needs for real-time event streaming with traditional API patterns. Attendees will gain an understanding of Kong’s current capabilities for Kafka integration, highlighting key features that enable smooth API management, enhanced scalability, and reliability. We'll also offer ways for you to get first-hand experience. Looking ahead, we’ll share exciting developments on the horizon, including upcoming features, enhancements that will further empower Kafka users as well as learn more. 2) Unlock full end-to-end tracing through Kong and Kafka with OpenTelemetry Ideally, you should be using distributed tracing to trace requests through your system, but Kafka decouples producers and consumers, which means there are no direct transactions to trace between them. Kafka also uses asynchronous processes, which have implicit, not explicit, dependencies. Another key component of such an architecture is securing and exposing endpoints with a powerful platform such as Kong. All of this makes it challenging to understand how your microservices are working together. However, it is possible to observe your Kong Gateway interactions and Kafka-based applications with distributed tracing and OpenTelemetry. You can then analyze and visualize your traces in an open source distributed tracing tool like Jaeger or a full observability platform like New Relic. In this talk I will leverage a simple application to show how you can achieve this. Speakers Johannes Brännström (Staff Solutions Engineer, Kong) Joining Kong from MongoDB & Red Hat Johannes is passionate about agile, modern application architecture and continuous delivery in all areas required to build great software.highly available. Harry Kimpel (Principal Developer Relations Engineer, New Relic) Passionate software craftsman with 25+ years experience in a broad spectrum of development technologies and platforms. Mostly focusing on .NET/C#, the Microsoft development stack and cloud-native software architectures. Key focus also on Microsoft Azure as well as Amazon Web Services (AWS) and Google Cloud Platform (GCP). Observability, monitoring and securing these environments are key aspects. Currently extremely interested in the Distributed Application Runtime (Dapr). See you there! Kong Community Team By registering for this event, I understand that my personal information will be processed in accordance with Kong's Privacy Policy. |
In-PERSON: Apache Kafka® x Kong Meetup in London
|
|
Making Distributed State Accessible: An Introduction to Kafka Interactive Queries v2
2024-10-22 · 19:50
Ramin Gharib
– Software and Data Engineer
@ bakdata
In a world increasingly reliant on real-time data processing, we require efficient access to live, distributed states. This talk delves into Interactive Queries v2 in Kafka Streams — a powerful feature that allows direct, real-time access to the state of distributed Kafka Streams applications. We will build on the context of stream processing and why accessing the state of distributed applications is crucial for modern data-driven architectures. Through this lens, you will see how Interactive Queries bridge the gap between real-time analytics state and external consumption, enabling you to unlock new possibilities for monitoring, data exploration, and responsive application behavior. Building on this foundation, we will discuss a running example illustrating how Interactive Queries can be leveraged to create a dynamic and responsive application. We will guide you through the different types of state stores and how to query them. Attendees will leave with a solid understanding of how Interactive Queries work, the types of problems they solve, and how they can be applied effectively to enhance the value of Kafka-based streaming applications. |
|
|
Intro to Apache Kafka
2024-10-22 · 19:00
David Anderson
– Software Practice Lead
@ Confluent
Dive into the world of real-time data streaming with this introduction to Apache Kafka. This talk is tailored for developers, data engineers, and IT professionals who want to gain a foundational understanding of Kafka, a powerful open-source platform used for building scalable, event-driven applications.You will learn about: Kafka fundamentals: the core concepts of Kafka, including topics, partitions, producers, and consumers The Kafka ecosystem: brokers, clients, Schema Registry, and Kafka Connect Stream processing: Kafka Streams and Apache Flink Use cases: discover how data streaming with Kafka has transformed various industries |
|
|
LDPaC In-Person Meetup 12th Sep: Transforming SQL Auth & Flink-Kafka Fusion
2024-09-12 · 16:45
Price - Free Thank you to Accenture for sponsoring and hosting this event. Join us for a In-Person User Group Meeting (LDPaC), where you can network, learn, ask a question, and meet other likeminded folks. These events are a really great opportunity to socialise in an informal learning setting. Remember to tell your friends and the people you work with; make sure you register as soon as you can. We will need to provide a list of names to Accenture before the event, so to ensure there are no issues with access on the day please make sure you have registered. 17.45 - 18:00 Network 🤝 18.00 - 18:30 Drinks & Pizza 🍕 18:30 - 18:40 Intro🎙️ 18:40 - 19:30 Transforming SQL Authentication: Real-World Scenarios with Azure Managed Identity. - Dieter Gobeyn 🔊 Discover how to enhance the security of your SQL databases by transitioning from traditional password-based authentication to a modern, passwordless approach using Azure Managed Identity. In this session, I will explore real-world scenarios that demonstrate the practical implementation of transitioning from connectionstrings to Role-Based Access Control (RBAC) with Managed Identity for Azure applications. Highlighting the benefits of eliminating passwords and simplifying access management. 19:30 - 19:40 10-min Break 🥤 19:40 - 20:30 Flink-Kafka Fusion: Patterns for Building Scalable Stream Processing Applications. - Dunith Dhanushka 🔊 When properly configured, Apache Kafka and Apache Flink provide a solid foundation for building scalable and reliable stream processing applications. However, it is difficult for most data professionals to get this combination right first, leading to project delays and costly mistakes. Consequently, having a set of patterns for stream processing applications is essential, just as it is for other software architectures. This talk will walk you through more than ten stream processing patterns using Kafka and Flink. First, we discuss various computational patterns covering stateless and stateful operations. Then, we look at several state management patterns for state recovery and maintaining accuracy. Finally, we discuss several non-functional patterns, especially related to the deployment architecture, which helps us achieve high availability, security, and cost reductions of streaming applications. For each pattern, the business motivation, a sample code in Flink, and other related patterns will be provided, making it easy for the audience to grasp the essence. This talk can benefit data practitioners of all levels, including application developers, data engineers, and architects, by providing them with blueprints for stream processing. Knowing these patterns will save time and avoid making the same mistakes that others have made in the past. Come and join the Leeds Data Community and start learning and networking! All are welcome! |
LDPaC In-Person Meetup 12th Sep: Transforming SQL Auth & Flink-Kafka Fusion
|
|
IN-PERSON: Kafka Summit London Meetup
2024-03-18 · 17:00
Join us for a special event the night before Kafka Summit London 2024 as we delve into the fascinating world of Kafka Streams. This meetup will focus on the critical role Kafka plays in enabling large enterprises connect siloed systems in large organisations. Date and Time: 🗓️ Monday 18th, ⏰ 5:00 PM - 7:00 PM 🕘 Venue: Keyboards & Dreams Royal Docks, Unit 10, 4 Hoy Street, E16 1XD Event Theme: Connecting enterprise data with Kafka Streams Attending Brands: OSO, Axual, Norsk Helsenett Networking: Pizza + Beer at the end Schedule: 17:00 - 17:15 Welcome intros 17:15 - 17:45 Talk 1: Disconnecting our systems with Kafka (including Q&A - hosted by OSO) 17:45 - 18:15 Break (Pizza & Drinks) 18:15 - 18:40 Talk 2: Kafka Streams use cases using KSML 18:40 - close Networking (Drinks) 🎙️ \~Talk 1\~ Talk Title: Disconnecting our systems with Kafka Summary: In NHN, we have many teams that have spawned from a patchwork of different organisations. Thusly, we have many applications, and teams, that operate as silos. Often aided by the requirements of the health care sectors governance rules about data that needs to be compartmentalised. Here we'll talk about how we are giving teams a way to collaborate and share data across silos, but without creating hard couplings. Key talk takeaways:
🗣️ Speakers: Simon Randby & Fredrik Bekkevold - Principal Developers at Norsk helsenett Simon works with connecting the healthcare sector in Norway at NHN. He has previously touched upon many disciplines, from refining Programvareverkstedet's operations, working on Samfundet's websites and marketing, connecting children with coding and robotics at Kodeløypa, and making awesome demos with the Ninjadev group. Loves cooking and music. Fredrik has over 20 years of experience creating solutions, I have seen a lot. The last 6 years I have worked primarily in healthcare and Fin-Tech. Both domains have similarities and are both seen as complex. By using the right mindset, technology and just being polite, I try to reduce complexity and make things happen. And sometimes great things do happen. 🎙️ \~Talk 2\~ Talk Title: Kafka Streams use cases using KSML Summary: KSML stands for Kafka Streams Markup Language. It allows developers to write Kafka Streams apps using a combination of YAML and Python. In this talk we will demonstrate a simple use case where several Municipal Home Care Service Providers are responsible for collecting heartbeat, sleep and oxygen saturation readings from patients, and for sharing them centrally. These readings are then picked up and enriched (joined) with the patient's personal information to be exposed to researchers downstream. Key talk takeaways:
🗣️ Speaker: Jeroen van Disseldorp - CEO & Founder of Axual Jeroen is an entrepreneur, solution architect and all-round techie. In 2015 he founded Axual, a Dutch company that created an enterprise event streaming platform based on Apache Kafka. Before that he was Technology Officer Open Source for Capgemini where he gained extensive experience in sectors like energy, banking and central government. |
IN-PERSON: Kafka Summit London Meetup
|
|
Kong NYC In-Person Meetup (June)
2023-06-29 · 21:00
We are thrilled to invite you to our second in-person meetup in NYC for 2023 in collaboration with EPAM! Get ready for an evening full of insightful talks, networking and delicious food and drinks. Kong makes connecting APIs and microservices across today’s hybrid, multi-cloud environments easier and faster than ever. We power trillions of API transactions for leading organizations globally through our end-to-end API platform. When: June 29, 17:00 - 21:00 Where: 275 7th Ave Suite 2002, New York, NY 10001, United States Agenda: 17:00 - 18:00 Everyone arrives & Networking 18:00 - 18:45 Talk 1 Gateway to the Future: A Brief History of Kubernetes Ingress by Viktor Gamov (Principal Developer Advocate at Kong) 18:45 - 19:30 Talk 2 Kong API Gateway - Custom Plugin Development by Andrew Smirnov (Delivery Manager at EPAM) 19:30 - 21:00 Networking, Food & Drinks Talks Talk 1 Gateway to the Future: A Brief History of Kubernetes Ingress Join Viktor Gamov, a Principal Developer Advocate with Kong, as he navigates the evolution of Kubernetes Ingress. Starting with the basics, he'll explain how Ingress is pivotal in managing external access to services within a Kubernetes cluster through HTTP and HTTPS routes. He'll also address its limitations - the challenge of managing multiple Ingress resources, inconsistencies across different controllers, and inherent limitations in handling non-L7 protocols like TCP and UDP. The talk then shifts gear toward the future - the Gateway API. This next evolution of Kubernetes networking expands beyond HTTP/HTTPS, promising improved traffic routing and the capability to handle diverse protocols, thus addressing Ingress's limitations. He'll explain how the Gateway API introduces resources like GatewayClass and Gateway to provide a flexible, structured way of defining traffic routing paths, simplifying traffic management in complex environments. Adding to the mix, Viktor will hint at how the Kong Gateway and Ingress Controller can leverage the Gateway API to extend their capabilities, offering a robust and more flexible networking experience. Whether you're a Kubernetes beginner or a seasoned pro, this talk promises a wealth of insights. Join us to explore Kubernetes Ingress and step into its promising future with the Gateway API. Talk 2 Kong API Gateway - Custom Plugin Development During the session, we will explore the structure of the Kong plugin and witness a live demonstration on developing a custom plugin using a plugin template. Additionally, this talk will provide insights on incorporating user-adjustable settings into the plugin and conducting tests using the Pongo tool. These tests can be seamlessly integrated into the CI/CD pipeline. Speakers: Viktor Gamov is a Principal Developer Advocate at Kong. He has a comprehensive expertise in building distributed systems and cloud-native architectures using open source technologies. He enjoys helping architects and developers design and develop low latency, scalable, and highly available systems. He is a professional conference speaker on distributed systems, streaming data, JVM, and DevOps topics. He's regular on events including JavaOne, Devoxx, OSCON, QCon, and others. Viktor has co-authored books O'Reilly's «Enterprise Web Development» and Manning's «Apache Kafka® in Action.» He blogs at http://gamov.io and co-hosts KongCast https://pod.link/kongcast. Follow Viktor on Twitter @gamussa, where he posts there about gym life, food, open-source, and Kong and Cloud. Andrey Smirnov has over 12 years of experience in Software Development and 10+ years in delivery management, I bring expertise in various domains. My focus lies on providing solution architecture decisions and successful project delivery. Throughout my career, I have gained valuable experience in diverse industries such as Business Intelligence, Oil & Gas, IT, Finance (Investment, Insurance), and Media. |
Kong NYC In-Person Meetup (June)
|