Training an AI-powered Slackbot sounds straightforward - until your model starts ignoring half of the data you feed it. At AWS User Group Vienna, we built OTTO, a Slack-integrated AI assistant, fine-tuned using the open source tool InstructLab and deployed on Amazon Bedrock. But as we scaled up, we ran into real-world bottlenecks: training on MacBooks was slow, retrieval was inconsistent, and debugging was way harder than expected. This talk goes beyond the ‘perfect AI stack’ and into the messy reality of model tuning, infrastructure choices, and the unexpected lessons we learned. If you’re working on AI-powered assistants (or just curious how fast things can go sideways), this talk will provide practical insights into our approach, with focus on cost efficiency, and what we learned on the way.