Live-demo showing automated testing of large language models. Addresses non-determinism in ML systems and demonstrates how a second LLM can act as a judge. Also explores Retrieval Augmented Generation (RAG) for querying documents and guiding tests.

What does the term release day bring to mind? A bundle of stress? Testing work piling up like a balloon and exploding? Engineers trying to sneak features in after you've finished testing? Does your job feel like a crescendo of pain that culminates in a release, only to start all over again? Should your job location say release hell and your job description firefighter? You are not alone! This is an entrenched problem in quality processes everywhere, beyond just software engineering. And yes, there are solutions. One of them goes by the cryptic name Shift-Left. Shift-Left is often miscommunicated. In this talk, Anupam rescues this term from management jargon and SEO buzzwords to unpack what it means for you as a quality professional. And here's the spoiler — Shift-Left is the key to upgrade your career and get paid better.