Session Abstract

AI workloads are increasingly moving to the edge for performance, data locality and even privacy reasons. Standard ML frameworks (e.g., PyTorch or TensorFlow) and general-purpose OSes and distributions (e.g., Linux/Ubuntu) provide great functionality, but consume a significant amount of the limited resources of IoT/edge devices. In this presentation we will introduce a system able to take standard ML framework scripts and automatically generate a purpose-built stack and operating system to run them extremely efficiently on ARM64 IoT devices; a MobilNet deployment, for instance, can run with as little as 25MB including the trained network and underlying operating system. The presentation will further show a live demo of the system.

Session Speakers

Felipe Huici

Chief Researcher, NEC Laboratories Europe GmbH

I’m a chief researcher in the systems group at NEC Laboratories Europe in Heidelberg, Germany. My main research and work interests lie in the areas of high-performance software systems, and in particular specialization, virtualization, and the application of machine learning techniques to tackle open problems in the systems area. Previously, I received an undergraduate degree with honours from the University of Virginia, a Masters in Data Communications, Networks and Distributed Systems from University College London (top of the class), and a Ph.D. also from UCL. I have published on several top-tier conferences and journals such as SOSP, SIGCOMM, NSDI, CoNEXT, and SIGCOMM CCR and regularly act as TPC member of conferences and journals such as IMC , INFOCOM, CoNEXT and SIGCOMM CCR.