Replay du meetup OVHcloud & Aiven du 3 avril 2025. Stéphane Ligneul, Olivier Huber & Stéphane Heckel démarrent la première session de discussion autour de trois principes clés d'une plateforme data: composabilité, portabilité et programmabilité. Exploration de l'architecture multi-engines intégrant des moteurs de traitement modulaires et des OTFs (open table format). Combinaison des infrastructures cloud, on-prem et hybrides pour une stratégie de données portable et souveraine. Automatisation des plateformes et maîtrise des coûts. Autant de questions que nous allons couvrir lors de ce premier échange.

Experience the power of AlloyDB Omni, a cutting-edge PostgreSQL-compatible database designed for multicloud and hybrid cloud environments. This session explores how AlloyDB Omni accelerates the development of modern applications, enabling generative AI experiences, efficient vector search, real-time operational analytics, and scalable transactional performance. We’ll also showcase how to run your applications on multiple clouds using Aiven’s seamless managed service, and how to supercharge hybrid cloud deployments with cloud-ready partners.

Que ce soit pour vos besoins retail, e-commerce ou même gaming, il n’a jamais été aussi facile de pousser vos données “temps réels” dans votre Data Warehouse. Nous allons, en live coding et à l’aide de la plateforme Aiven, monter une stack de data streaming Open Source en moins de 30 minutes avec la gestion des logs et du monitoring ainsi que ses connecteurs vers les services Google.

Apache Kafka is a distributed system. At the heart of Apache Kafka is a set of brokers that contain topics. Topics are split into partitions. Dividing topics into smaller pieces allows us to work with data in parallel and achieve higher data throughput.\n\nSuch parallelization is the key to a performant cluster, however it comes with a price. First, reading from multiple partitions will eventually mess up the order of records, meaning that the resulting order will be different from when the data was pushed into the cluster. Another big challenge is uneven distribution of data across partitions.\n\nOverloaded partitions present a dangerous issue for performance of all involved parties, but especially for brokers and consumers. Therefore, when building our product architecture we should carefully weigh up how many partitions we need, how to ensure proper message ordering, how to balance records across partitions, not forgetting about data load distribution over time. And do all of this while still maintaining good performance of the cluster.\n\nIf you're fresh to Apache Kafka, or looking for good practices to design your partitions and avoid common pitfalls, you'll find this session useful!