Abstract
In the past decade, Deep Neural Networks (DNNs) have achieved unprecedented improvement in the inference accuracy for several hard-to-tackle applications such as image classification, object detection and identification, natural language processing, etc. The state-of-the-art DNNs that achieve close to 100% inference accuracy are typically large in size requiring gigabytes of memory to load the models. On the other end of the spectrum, the tinyML community is pushing the limits of compressing DNNs in order to embed them on memory-limited IoT devices. Performing local inference for data samples on IoT devices reduces delay, saves network bandwidth, and improves energy efficiency of the system, but it suffers in terms of low QoE as the tinyML DNNs have lower inference accuracy. In order to reap the benefits of doing local inference while not compromising on the inference accuracy, we explore the idea of Hierarchical Inference (HI), wherein the local inference is accepted only when it is correct, otherwise the data sample is offloaded. However, in general, it is impossible to know if the local inference is correct or not a priori. Nevertheless, in this talk, I will discuss how HI can be implemented for two important applications, namely, Machine Fault Detection and Image Classification. For Machine Fault Detection, we show that HI can be implemented using a simple threshold-based algorithm. For image classification, we present an online learning framework for identifying incorrect local inference. The resulting problem turns out to be Prediction with Experts' Advice with continuous action space. We will present HI learning algorithms with sub-linear regrets and evaluate their efficacy using datasets from ImageNet and MNIST dataset. Jaya Prakash Champati, IMDEA Networks Institiute
Jaya Prakash Champati is an Assistant Professor at IMDEA Networks Institute, where he leads the Edge Networks group. His general research interest is in the scheduling of communication and computation for emerging applications in edge computing systems, Internet of Things (IoT), and Cyber-Physical Systems (CPS). Prior to joining IMDEA, he was a post-doctoral researcher in the division of Information Science and Engineering, EECS, KTH Royal Institute of Technology, Sweden. He obtained his PhD in Electrical and Computer Engineering from the University of Toronto, Canada in 2017, and his master of technology degree from the Indian Institute of Technology (IIT) Bombay, India in 2010. Prior to joining PhD, he worked at Broadcom Communications, where he contributed to the LTE MAC layer development. He was awarded the prestigious Marie Skłodowska-Curie Actions (MSCA) postdoctoral fellowship, 2021, and he was a recipient of the best paper award at IEEE National Conference on Communications, India, 2011.