Improving the robustness of analog deep neural networks through a Bayes-optimized noise injection approach

Nanyang Ye¹, Linfeng Cao¹, Liujia Yang¹, Ziqing Zhang², Zhicheng Fang¹, Qinying Gu³ & Guang-Zhong Yang¹

¹Shanghai Jiao Tong University, Shanghai, P.R. China. ²University of Cambridge, Cambridge, United Kingdom. ³Shanghai Artificial Intelligence Laboratory,
Shanghai, P.R. China.

Abstract

Analog deep neural networks (DNNs) provide a promising solution, especially for deployment on resource-limited platforms, for example in mobile settings. However, the practicability of analog DNNs has been limited by their instability due to multi-factor reasons from manufacturing, thermal noise, etc. Here, we present a theoretically guaranteed noise injection approach to improve the robustness of analog DNNs without any hardware modifications or sacrifice of accuracy, which proves that within a certain range of parameter perturbations, the prediction results would not change. Experimental results demonstrate that our algorithmic framework can outperform state-of-the-art methods on tasks including image classification, object detection, and large-scale point cloud object detection in autonomous driving by a factor of 10 to 100. Together, our results may serve as a way to ensure the robustness of analog deep neural network systems, especially for safety-critical applications.