Understanding information leakage of distributed inference with deep neural networks: overview of information theoretic approach and initial results

Tiffany Tuor; Shiqiang Wang; Kin K. Leung; Bong Jun Ko

doi:10.1117/12.2306000

4 May 2018 Understanding information leakage of distributed inference with deep neural networks: overview of information theoretic approach and initial results

Tiffany Tuor, Shiqiang Wang, Kin K. Leung, Bong Jun Ko

Author Affiliations +

Proceedings Volume 10635, Ground/Air Multisensor Interoperability, Integration, and Networking for Persistent ISR IX; 106350K (2018) https://doi.org/10.1117/12.2306000
Event: SPIE Defense + Security, 2018, Orlando, FL, United States

Abstract

With the emergence of Internet of Things (IoT) and edge computing applications, data is often generated by sensors and end users at the network edge, and decisions are made using these collected data. Edge devices often require cloud services in order to perform intensive inference tasks. Consequently, the inference of deep neural network (DNN) model is often partitioned between the edge and the cloud. In this case, the edge device performs inference up to an intermediate layer of the DNN, and offloads the output features to the cloud for the inference of the remaining of the network. Partitioning a DNN can help to improve energy efficiency but also rises some privacy concerns. The cloud platform can recover part of the raw data using intermediate results of the inference task. Recently, studies have also quantified an information theoretic trade-off between compression and prediction in DNNs. In this paper, we conduct a simple experiment to understand to which extent is it possible to reconstruct the raw data given the output of an intermediate layer, in other words, to which extent do we leak private information when sending the output of an intermediate layer to the cloud. We also present an overview of mutual-information based studies of DNN, to help understand information leakage and some potential ways to make distributed inference more secure.

Citation Download Citation

Tiffany Tuor, Shiqiang Wang, Kin K. Leung, and Bong Jun Ko "Understanding information leakage of distributed inference with deep neural networks: overview of information theoretic approach and initial results", Proc. SPIE 10635, Ground/Air Multisensor Interoperability, Integration, and Networking for Persistent ISR IX, 106350K (4 May 2018); https://doi.org/10.1117/12.2306000

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available