talk-data.com talk-data.com

C

Speaker

Christoph Stuber

1

talks

Software Engineer Grammarly

Grammarly software engineer focused on ML infrastructure and serving LLMs; uses a combination of third-party LLM APIs and in-house LLMs.

Bio from: Mastering LLM Serving and Management at Grammarly

Filter by Event / Source

Mastering LLM Serving and Management at Grammarly 1

Talks & appearances

1 activities · Newest first

Search activities →

ML infrastructure and serving LLMs at scale

2024-05-15 · Mastering LLM Serving and Management at Grammarly
talk
llms ml infrastructure third-party llm apis in-house llms load testing

This talk covers Grammarly's approach to using a combination of third-party LLM APIs and in-house LLMs, the role of LLMs in Grammarly's product offerings, an overview of the tools and processes used in our ML infrastructure, and how we address challenges such as access, cost control, and load testing of LLMs, sharing our experience in optimizing and serving LLMs.