talk-data.com talk-data.com

Meetup Presentation 2025-03-06 at 17:00

Introducing GneissWeb - a state-of-the-art LLM pre-training dataset

Event: [AI Alliance] Introducing Gneissweb: A State-Of-The-Art LLM Pre-training Dataset

Speakers

Shahrokh Daijavad

Research Scientist · IBM Almaden Research Center

Topics

LLM pre-training dataset ai huggingface

Description

In this session we will go over how we created GneissWeb and discuss tools and techniques used. We will provide code examples that you can try at your leisure.

👉 > 2% avg improvement in benchmark performance over FineWeb 👉 Huggingface page 👉 Data prep kit detailed recipe 👉 Data prep kit bloom filter for quick reproduction 👉 Recipe models for reproduction 👉 announcement 👉 Paper