talk-data.com

Showing 1 result

Activities & events

Title & Speakers	Event
ML infrastructure and serving LLMs at scale 2024-05-15 · 19:00 Christoph Stuber – Software Engineer @ Grammarly This talk covers Grammarly's approach to using a combination of third-party LLM APIs and in-house LLMs, the role of LLMs in Grammarly's product offerings, an overview of the tools and processes used in our ML infrastructure, and how we address challenges such as access, cost control, and load testing of LLMs, sharing our experience in optimizing and serving LLMs. llms ml infrastructure third-party llm apis in-house llms load testing	Mastering LLM Serving and Management at Grammarly

Title & Speakers

Event

ML infrastructure and serving LLMs at scale 2024-05-15 · 19:00

Christoph Stuber – Software Engineer @ Grammarly

This talk covers Grammarly's approach to using a combination of third-party LLM APIs and in-house LLMs, the role of LLMs in Grammarly's product offerings, an overview of the tools and processes used in our ML infrastructure, and how we address challenges such as access, cost control, and load testing of LLMs, sharing our experience in optimizing and serving LLMs.

llms ml infrastructure third-party llm apis in-house llms load testing

Mastering LLM Serving and Management at Grammarly

Showing 1 result