Tasked with a mixed audience of Gophers and Rustaceans, Bryan went for the layer that is common to both - the CPU executing the code. All large servers adopt a design known as Non-Uniform Memory Access, where some memory is faster to access than the rest. Bryan will explain how this comes about, what it means for the performance of your programs, and what control you have over NUMA.

Running models locally on the CPU and possibly a GPU means we can experiment with the latest quantised models on real client data without anything leaving the machine. We can explore text question answering, image analysis and calling these tools via a Python API for rapid PoC experimentation. This quickly exposes the ways that LLMs go weird and maybe that helps us avoid some of the examples of early LLM deployments making embarrassing mistakes!