This event is in-person only and will be followed by a networking apéro. We are looking forward to seeing you all in person!

***

Florent Gbelidji - Hugging Face Title: Customizing RAG System Components to Build Domain-Specific Assistant Summary : Retrieval Augmented Generation (RAG) has become a prevalent approach in developing Large Language Models (LLM) applications, incorporating industry-specific data and the most recent information. In this session, we'll delve into the mechanisms of RAG applications, focusing on key components like the retriever and the LLM. Our exploration will include leveraging tools from the open-source ecosystem to fine-tune these components, enhancing their performance in providing assistance, especially when confronted with domain-specific questions.

***

Guillaume Richard and Marie Lopez - InstaDeep Title: The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human Genomics Summary : Closing the gap between measurable genetic information and observable traits is a longstanding challenge in genomics. Yet, the prediction of molecular phenotypes from DNA sequences alone remains limited and inaccurate, often driven by the scarcity of annotated data and the inability to transfer learnings between prediction tasks. Here, we present an extensive study of foundation models pre-trained on DNA sequences, named the Nucleotide Transformer, ranging from 50M up to 2.5B parameters and integrating information from 3,202 diverse human genomes, as well as 850 genomes selected across diverse phyla, including both model and non-model organisms. These transformer models yield transferable, context-specific representations of nucleotide sequences, which allow for accurate molecular phenotype prediction even in low-data settings. We show that the developed models can be fine-tuned at low cost and despite low available data regime to solve a variety of genomics applications. Despite no supervision, the transformer models learned to focus attention on key genomic elements, including those that regulate gene expression, such as enhancers. Lastly, we demonstrate that utilizing model representations can improve the prioritization of functional genetic variants. The training and application of foundational models in genomics explored in this study provide a widely applicable stepping stone to bridge the gap of accurate molecular phenotype prediction from DNA sequence.