At CERN- the European Organization for Nuclear Research, machine learning is applied across a wide range of scenarios, from simulations and event reconstruction to classifying interesting experimental events, all while handling data rates in the order of terabytes per second. As a result, beyond developing complex models, CERN also requires highly optimized mechanisms for model inference.

From the ML4EP team at CERN, we have developed SOFIE (System for Optimized Fast Inference code Emit), an open-source tool designed for fast inference on ML models with minimal dependencies and low latency. SOFIE is under active development, driven by feedback not only from high-energy physics researchers but also from the broader scientific community.

With upcoming upgrades to CERN’s experiments expected to increase data generation, we have been investigating optimization methods to make SOFIE even more efficient in terms of time and memory usage, while improving its accessibility and ease of integration with other software stacks.

In this talk, we will introduce SOFIE and present novel optimization strategies developed to accelerate ML inference and reduce resource overhead.

At CERN (European Organization for Nuclear Research), machine learning models are developed and deployed for various applications, including data analysis, event reconstruction, and classification. These models must not only be highly sophisticated but also optimized for efficient inference. A critical application is in Triggers- systems designed to identify and select interesting events from an immense stream of experimental data. Experiments like ATLAS and CMS generate data at rates of approximately 100 TB/s, requiring Triggers to rapidly filter out irrelevant events. This talk will explore the challenges of deploying machine learning in such high-throughput environments and discuss solutions to enhance their performance and reliability.