talk-data.com

📍8 rue Cambacérès, 75008 Paris 📆 October 9th, 7:00 p.m. ⚠️ Limited spots available. Be sure to reserve your place in advance!

👥 Alexandre Défossez - Chief Exploration Officer @ Kyutai ➡️ Moshi: a speech-text foundation model for real-time dialogue. Summary: We will discuss Moshi, our recently released model. Moshi is capable of full-duplex dialogue, e.g. it can both speak and listen at any time, offering the most natural speech interaction to date. Besides, Moshi is also multimodal, in particular it is able to leverage its inner text monologue to improve the quality of its generation. We will cover the design choices behind Moshi in particular the efficient joint sequence modeling permitted by RQ-Transformer, and the use of large scale synthetic instruct data.

👥 Louis Lacombe, Valentin Laurent, Thibault Cordier - Data Scientist @ Quantmetry - Part of Capgemini Invent ➡️ Enhancing NLP Model Reliability with MAPIE: Conformal Prediction for Uncertainty Quantification Summary: This talk introduces MAPIE, an open-source Python library designed to quantify uncertainties and control risks in machine learning models, with a focus on NLP applications. We will begin by discussing the importance of uncertainty quantification based on conformal prediction framework that ensures guarantees with few assumptions. Then, we will present MAPIE, showcasing how to compute conformal prediction sets for NLP tasks like text classification. Finally, we will explore practical use cases, highlighting the capabilities of MAPIE and providing attendees with a comprehensive overview of its potential applications.

This event is in-person only and will be followed by a networking apéro. We are looking forward to seeing you all in person!

***

Florent Gbelidji - Hugging Face Title: Customizing RAG System Components to Build Domain-Specific Assistant Summary : Retrieval Augmented Generation (RAG) has become a prevalent approach in developing Large Language Models (LLM) applications, incorporating industry-specific data and the most recent information. In this session, we'll delve into the mechanisms of RAG applications, focusing on key components like the retriever and the LLM. Our exploration will include leveraging tools from the open-source ecosystem to fine-tune these components, enhancing their performance in providing assistance, especially when confronted with domain-specific questions.

***

Guillaume Richard and Marie Lopez - InstaDeep Title: The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human Genomics Summary : Closing the gap between measurable genetic information and observable traits is a longstanding challenge in genomics. Yet, the prediction of molecular phenotypes from DNA sequences alone remains limited and inaccurate, often driven by the scarcity of annotated data and the inability to transfer learnings between prediction tasks. Here, we present an extensive study of foundation models pre-trained on DNA sequences, named the Nucleotide Transformer, ranging from 50M up to 2.5B parameters and integrating information from 3,202 diverse human genomes, as well as 850 genomes selected across diverse phyla, including both model and non-model organisms. These transformer models yield transferable, context-specific representations of nucleotide sequences, which allow for accurate molecular phenotype prediction even in low-data settings. We show that the developed models can be fine-tuned at low cost and despite low available data regime to solve a variety of genomics applications. Despite no supervision, the transformer models learned to focus attention on key genomic elements, including those that regulate gene expression, such as enhancers. Lastly, we demonstrate that utilizing model representations can improve the prioritization of functional genetic variants. The training and application of foundational models in genomics explored in this study provide a widely applicable stepping stone to bridge the gap of accurate molecular phenotype prediction from DNA sequence.