Enterprise customer IT environment can have specificities that makes it challenging to deploy Large Language Models as we know them. In this talk, I will detail his journey in implementing such models and the best practices he learned.