Salomon will showcase our open-source approach helping data engineers to define complex Kafka pipelines consisting of multiple interconnected Kafka Streams applications, Kafka Producers/Consumers, and Kafka Connectors in a straightforward manner. Our ready-to-use approach provides automated lifecycle management of Kafka pipelines on Kubernetes, i.e., handling deployment, update, destroy, reset, and clean-up transparently. In addition, we demonstrate how this approach provides a comprehensible view of all Kafka pipelines and their metrics.

https://github.com/bakdata/kpops