Fireside chat on evaluating quality in LLM models pre-production.

LLMs have unlocked new opportunities in NLP with their possible applications. Features that used to take months to be planned and developed now require a day to be prototyped. But how can we make sure that a successful prototype will turn into a high-quality feature useful for millions of customers? In this talk, we will explore real examples of the challenges that arise when ensuring the quality of LLM outputs and how we address them at Grammarly.