LLMs have opened up new avenues in NLP with their possible applications, but evaluating their output introduces a new set of challenges. In this talk, we discuss how the evaluation of LLMs differs from the evaluation of classic ML-based solutions and how we tackle the challenges.

LLMs have opened up new avenues in NLP with their possible applications, but evaluating their output introduces a new set of challenges. In this talk, we discuss these challenges and our approaches to measuring the model output quality. We will talk about the existing evaluation methods and their pros and cons and then take a closer look at their application in a practical case study.

This talk will help you understand the main responsibilities of analytical and computational linguists at the company, the types of tasks and projects they work on, and how they collaborate with the project teams. You will learn what kind of linguistic expertise is required for building AI-powered solutions at Grammarly.