Join Kostia Omelianchuk and Lukas Beisteiner as they unpack the full scope of Grammatical Error Correction (GEC) from task framing, evaluation, and training to inference optimization and serving high-performance production systems at Grammarly. They will discuss: The modern GEC recipe (shift from heavily human-annotated corpora to semi-synthetic data generation), LLM-as-a-judge techniques for scalable evaluation, and techniques to make deployment fast and affordable, including Speculative Decoding.

Ever wondered what actually happens when you call an LLM API? This talk breaks down the inference pipeline from tokenization to text generation, explaining what's really going on under the hood. He will walk through the key sampling strategies and their parameters - temperature, top-p, top-k, beam search. We'll also cover performance tricks like quantization, KV caching, and prompt caching that can speed things up significantly. If time allows, we will also touch on some use-case-specific techniques like pass@k and majority voting.

Finetuning and Inference

Finetuning and inference workflows for geospatial foundation models using TerraTorch.

We will dive deeper into inference from Apple Silicon to huge production clusters, and cover tricks how to make it faster.