Training an AI-powered Slackbot sounds straightforward - until your model starts ignoring half of the data you feed it. At AWS User Group Vienna, we built OTTO, a Slack-integrated AI assistant, fine-tuned using the open source tool InstructLab and deployed on Amazon Bedrock. But as we scaled up, we ran into real-world bottlenecks: training on MacBooks was slow, retrieval was inconsistent, and debugging was way harder than expected. This talk goes beyond the ‘perfect AI stack’ and into the messy reality of model tuning, infrastructure choices, and the unexpected lessons we learned. If you’re working on AI-powered assistants (or just curious how fast things can go sideways), this talk will provide practical insights into our approach, with focus on cost efficiency, and what we learned on the way.

What we learned building BeepGPT: An AI bot that reads your Slack so you don’t have to. Are you tired of seeing Gen AI demos of yet another chat bot? Are you looking for the interesting applications of this technology that’s supposed to change everything? Generative AI can do more than answer questions - it can understand what’s happening in the world in real-time, and use that understanding to creatively make our lives better. But to do so, Generative AI apps need access to real-time information, which has traditionally been difficult to deal with. Connecting AI to real-time data is the problem that the OSS Kakada project (which, in full disclosure, I co-created) aims to solve. To demonstrate how the right tools make this type of application easy, we built BeepGPT - an AI bot that reads your Slack so you don’t have to. In this presentation we’ll walk you through our process and share some of the lessons we learned along the way.