Skip to main content
Log in

Secure image classification with deep neural networks for IoT applications

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

The Internet-of-Things (IoT) are used everywhere in our daily lives. IoT applications provide us with many useful functionalities such as preventing fires, detecting and tracking objects, controlling and reporting the changes in/outside the environments, and capturing images/videos in our homes, roads, and offices. For example, the images data gathered through the smart sensors of autonomous vehicles can serve in various applications such as traffic monitoring, prediction of road conditions, and classification of objects. Image classification with deep neural networks (DNNs) on the cloud is such a machine learning task and has great market potentials for IoT applications. Nevertheless, the deployment of these “smart” IoT devices and applications can raise the risks of security issues. It still suffers from the challenges of relieving IoT devices from excessive computation burdens, such as data encryption, feature extraction, and image classification. In this paper, we propose and implement an indistinguishability-chosen plaintext attack secure image classification framework with DNN for IoT Applications. The framework performs a secure image classification on the cloud without the IoT device’s constant interaction. We propose and implement a real number computation mechanism and a divide-and-conquer mechanism for the secure evaluation of linear functions in DNNs, as well as a set of unified ideal protocols for the evaluation of non-linear functions in DNNs. The information about the image contents, the private DNNs model parameters and the intermediate results is strictly concealed by the conjunctive use of the lattice-based homomorphic scheme and 2-PC secure computation techniques. A pre-trained deep convolutional neural network model, i.e., Visual Geometry Group (VGG-16), is used to extract the deep features of an image. The comprehensive experimental results show that our framework is efficient and accurate. In addition, we evaluate the security of our framework by performing the white-box membership inference attack which is believed to be the most powerful attack on DNNs models. The failure of the attack indicates that our framework is practical secure.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

References

  • Agrawal S, Freeman DM, Vaikuntanathan V (2011) Functional encryption for inner product predicates from learning with errors. In: Lee DH, Wang X (eds) Advances in cryptology—ASIACRYPT 2011. Springer, Berlin, pp 21–40

    Chapter  Google Scholar 

  • Araki T, Barak A, Furukawa J, Keller M, Lindell Y, Ohara K, Tsuchida H (2018) Generalizing the SPDZ compiler for other protocols. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pp 880–895

  • Bhardwaj S, Pandove G, Dahiya PK (2020) An efficient comparison of two indexing-based deep learning models for the formation of a web-application based IoT-cloud network. J Ambient Intell Human Comput. https://doi.org/10.1007/s12652-020-02500-2

  • Bogdanov D, Laur S, Willemson J (2008) Sharemind: a framework for fast privacy-preserving computations. In: Jajodia S, Lopez J (eds) Computer security—ESORICS 2008. Springer, Berlin, pp 192–206

    Chapter  Google Scholar 

  • Brakerski Z, Vaikuntanathan V (2014) Efficient fully homomorphic encryption from (standard) LWE. SIAM J Comput 43(2):831–871

    Article  MathSciNet  MATH  Google Scholar 

  • Brakerski Z, Gentry C, Vaikuntanathan V (2014) (Leveled) fully homomorphic encryption without bootstrapping. ACM Trans Comput Theory (TOCT) 6(3):1–36

    Article  MathSciNet  MATH  Google Scholar 

  • Chillotti I, Gama N, Georgieva M, Izabachene M (2016) Faster fully homomorphic encryption: bootstrapping in less than 0.1 seconds. In: International conference on the theory and application of cryptology and information security. Springer, Berlin, pp 3–33

  • Chillotti I, Gama N, Georgieva M, Izabachène M (2020) TFHE: fast fully homomorphic encryption over the torus. J Cryptol 33(1):34–91

    Article  MathSciNet  MATH  Google Scholar 

  • Dowlin N, Gilad-Bachrach R, Laine K, Lauter K, Naehrig M, Wernsing J (2016) Cryptonets: applying neural networks to encrypted data with high throughput and accuracy. Technical Report MSR-TR-2016-3, Microsoft Research

  • Du M, Wang K, Chen Y, Wang X, Sun Y (2018) Big data privacy preserving in multi-access edge computing for heterogeneous internet of things. IEEE Commun Mag 56(8):62–67. https://doi.org/10.1109/MCOM.2018.1701148

    Article  Google Scholar 

  • Elgamal T (1985) A public key cryptosystem and a signature scheme based on discrete logarithms. IEEE Trans Inf Theory 31(4):469–472

    Article  MathSciNet  MATH  Google Scholar 

  • Elhabob R, Sella I, Zhao Y, Zhu G, Xiong H (2018) A heterogeneous systems public key encryption with equality test in smart city. In: Proceedings of the 18th international conference on electronic business. ICEB, Guilin

  • Elhabob R, Zhao Y, Sella I, Xiong H (2019) Efficient certificateless public key cryptography with equality test for internet of vehicles. IEEE Access 7:68957–68969

    Article  Google Scholar 

  • Elhabob R, Zhao Y, Sella I, Xiong H (2020) An efficient certificateless public key cryptography with authorized equality test in IIoT. J Ambient Intell Human Comput 11(3):1065–1083

    Article  Google Scholar 

  • Eltayieb N, Elhabob R, Hassan A, Li F (2020) Secure mobile health system supporting search function and decryption verification. J Ambient Intell Human Comput. https://doi.org/10.1007/s12652-020-02321-3

  • Ferreira B, Rodrigues J, Leitão J, Domingos H (2015) Privacy-preserving content-based image retrieval in the cloud. In: 2015 IEEE 34th symposium on reliable distributed systems (SRDS), pp 11–20. https://doi.org/10.1109/SRDS.2015.27

  • Gentry C, Halevi S, Smart NP (2012) Fully homomorphic encryption with polylog overhead. In: Pointcheval D, Johansson T (eds) Advances in cryptology—EUROCRYPT 2012. Springer, Berlin, pp 465–482

    Chapter  Google Scholar 

  • Gilad-Bachrach R, Dowlin N, Laine K, Lauter K, Naehrig M, Wernsing J (2016) Cryptonets: applying neural networks to encrypted data with high throughput and accuracy. In: International conference on machine learning, pp 201–210

  • Goldreich O (2004) Foundations of Cryptography: Volume 2, Basic Applications. Cambridge University Press, New York

    Book  MATH  Google Scholar 

  • Goldreich O, Micali S, Wigderson A (1987) How to play ANY mental game. In: Proceedings of the 19th Annual Conference on Theory of computing. Vol. 87 ACM, New York, pp 218–229. https://doi.org/10.1145/28395.28420

  • Goldwasser S, Micali S, Rackoff C (1989) The knowledge complexity of interactive proof systems. SIAM J Comput 18(1):186–208. https://doi.org/10.1137/0218012

    Article  MathSciNet  MATH  Google Scholar 

  • Halevi S, Shoup V (2014) HElib-an implementation of homomorphic encryption. Cryptology ePrint Archive, Report 2014/039

  • Hassan A, Eltayieb N, Elhabob R, Li F (2018) An efficient certificateless user authentication and key exchange protocol for client-server environment. J Ambient Intell Human Comput 9(6):1713–1727

    Article  Google Scholar 

  • Hassan A, Wang Y, Elhabob R, Eltayieb N, Li F (2020) An efficient certificateless public key encryption scheme with authorized equality test in healthcare environments. J Syst Archit 109:101776. https://doi.org/10.1016/j.sysarc.2020.101776

  • Hastings M, Hemenway B, Noble D, Zdancewic S (2019) Sok: General purpose compilers for secure multi-party computation. In: 2019 IEEE symposium on security and privacy (SP), pp 1220–1237. https://doi.org/10.1109/SP.2019.00028

  • Hesamifard E, Takabi H, Ghasemi M, Wright RN (2018) Privacy-preserving machine learning as a service. Proc Priv Enhanc Technol 2018(3):123–142

    Google Scholar 

  • Hu S, Wang Q, Wang J, Qin Z, Ren K (2016) Securing sift: privacy-preserving outsourcing computation of feature extractions over encrypted image data. IEEE Trans Image Process 25(7):3411–3425

    Article  MathSciNet  MATH  Google Scholar 

  • Huang K, Liu X, Fu S, Guo D, Xu M (2019) A lightweight privacy-preserving CNN feature extraction framework for mobile sensing. IEEE Trans Dependable Secur Comput

  • Joy J, Rabsatt V, Gerla M (2018) Internet of vehicles: enabling safe, secure, and private vehicular crowdsourcing. Internet Technol Lett 1(1):e16

    Article  Google Scholar 

  • Juvekar C, Vaikuntanathan V, Chandrakasan A (2018) GAZELLE: a low latency framework for secure neural network inference. In: 27th USENIX security symposium (USENIX Security 18). USENIX Association, Baltimore, pp 1651–1669

  • Khayyam H, Javadi B, Jalili M, Jazar RN (2020) Artificial intelligence and internet of things for autonomous vehicles. In: Nonlinear approaches in engineering applications. Springer, Berlin, pp 39–68

  • Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in neural information processing systems 25. Curran Associates, Inc., Red Hook, pp 1097–1105

    Google Scholar 

  • Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun ACM 60(6):84–90. https://doi.org/10.1145/3065386

    Article  Google Scholar 

  • Li P, Li T, Yao ZA, Tang CM, Li J (2017) Privacy-preserving outsourcing of image feature extraction in cloud computing. Soft Comput 21(15):4349–4359

    Article  MATH  Google Scholar 

  • Liu J, Juuti M, Lu Y, Asokan N (2017a) Oblivious neural network predictions via miniONN transformations. In: Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, pp 619–631

  • Liu J, Yang J, Xiong L, Pei J (2017b) Secure skyline queries on cloud platform. In: 2017 IEEE 33rd international conference on data engineering (ICDE), pp 633–644

  • Liu F, Wang Y, Wang F, Zhang Y, Lin J (2019) Intelligent and secure content-based image retrieval for mobile users. IEEE Access 7:119209–119222. https://doi.org/10.1109/ACCESS.2019.2935222

    Article  Google Scholar 

  • Makri E, Rotaru D, Smart NP, Vercauteren F (2019) EPIC: efficient private image classification (or: learning from the masters). In: Matsui M (ed) Topics in cryptology—CT-RSA 2019. Springer, Cham, pp 473–492

    Chapter  Google Scholar 

  • Mao Q, Wang L, Tsang IW (2017) A unified probabilistic framework for robust manifold learning and embedding. Mach Learn 106(5):627–650

    Article  MathSciNet  MATH  Google Scholar 

  • Mohassel P, Rindal P (2018) ABY3: a mixed protocol framework for machine learning. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pp 35–52

  • Mohassel P, Zhang Y (2017) SecureML: a system for scalable privacy-preserving machine learning. In: 2017 IEEE symposium on security and privacy (SP), pp 19–38

  • Nasr M, Shokri R, Houmansadr A (2019) Comprehensive privacy analysis of deep learning: passive and active white-box inference attacks against centralized and federated learning. In: 2019 IEEE symposium on security and privacy (SP), pp 739–753

  • Ning Z, Hu X, Chen Z, Zhou M, Hu B, Cheng J, Obaidat MS (2017) A cooperative quality-aware service access system for social internet of vehicles. IEEE Internet Things J 5(4):2506–2517

    Article  Google Scholar 

  • Paillier P (1999) Public-key cryptosystems based on composite degree residuosity classes. In: International conference on the theory and applications of cryptographic techniques. Springer, Berlin, Heidelberg, pp 223–238

  • Polyakov Y, Rohloff K, Ryan GW (2018) Palisade lattice cryptography library. Cybersecurity Research Center, New Jersey Institute of Technology, Newark, NJ, USA, Technical Report

  • Rahim N, Ahmad J, Muhammad K, Sangaiah AK, Baik SW (2018) Privacy-preserving image retrieval for mobile devices with deep features on the cloud. Comput Commun 127:75–85

    Article  Google Scholar 

  • Riazi MS, Weinert C, Tkachenko O, Songhori EM, Schneider T, Koushanfar F (2018) Chameleon: a hybrid secure computation framework for machine learning applications. In: Proceedings of the 2018 on Asia conference on computer and communications security, pp 707–721

  • Rouhani BD, Riazi MS, Koushanfar F (2018) DeepSecure: scalable provably-secure deep learning. In: Proceedings of the 55th annual design automation conference. pp 1–6

  • Sadeghi AR, Schneider T, Wehrenberg I (2010) Efficient privacy-preserving face recognition. In: Lee D, Hong S (eds) Information, security and cryptology—ICISC 2009. Springer, Berlin, pp 229–244

    Chapter  Google Scholar 

  • SEAL (2019) Microsoft SEAL (release 3.3). https://github.com/Microsoft/SEAL. Microsoft Research, Redmond, WA

  • Shankar K, Elhoseny M, Kumar RS, Lakshmanaprabu S, Yuan X (2020) Secret image sharing scheme with encrypted shadow images using optimal homomorphic encryption technique. J Ambient Intell Human Comput 11(5):1821–1833

    Article  Google Scholar 

  • Sicari S, Rizzardi A, Grieco LA, Coen-Porisini A (2015) Security, privacy and trust in internet of things: the road ahead. Comput Netw 76:146–164

    Article  Google Scholar 

  • Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint. arXiv:1409.1556

  • Slaney M, Casey M (2008) Locality-sensitive hashing for finding nearest neighbors [lecture notes]. IEEE Signal Process Mag 25(2):128–131. https://doi.org/10.1109/MSP.2007.914237

    Article  Google Scholar 

  • Songhori EM, Hussain SU, Sadeghi A, Schneider T, Koushanfar F (2015) TinyGarble: highly compressed and scalable sequential garbled circuits. In: 2015 IEEE symposium on security and privacy, pp 411–428

  • Toor Y, Muhlethaler P, Laouiti A, De La Fortelle A (2008) Vehicle ad hoc networks: applications and related technical issues. IEEE Commun Surv Tutor 10(3):74–88

    Article  Google Scholar 

  • Wang Y, Hassan A, Duan X, Zhang X (2019a) An efficient multiple-user location-based query authentication approach for social networking. J Inf Secur Appl 47:284–294

    Google Scholar 

  • Wang Y, Hassan A, Liu F, Guan Y, Zhang Z (2019b) Secure string pattern query for open data initiative. J Inf Secur Appl 47:335–352

    Google Scholar 

  • Wang Y, Liu F, Pang Z, Hassan A, Lu W (2019c) Privacy-preserving content-based image retrieval for mobile computing. J Inf Secur Appl 49:102399

    Google Scholar 

  • Wang Y, Zhang S, Tang Y, Su Q, Chen B (2019d) Rational adversary with flexible utility in secure two-party computation. J Ambient Intell Human Comput 10(8):2913–2927

  • Yang Y, Wu L, Yin G, Li L, Zhao H (2017) A survey on security and privacy issues in internet-of-things. IEEE Internet Things J 4(5):1250–1258. https://doi.org/10.1109/JIOT.2017.2694844

    Article  Google Scholar 

  • Yao ACC (1986) How to generate and exchange secrets. In: 27th Annual symposium on foundations of computer science (SFCS 1986), pp 162–167

  • Yi X, Bertino E, Rao FY, Bouguettaya A (2016) Practical privacy-preserving user profile matching in social networks. In: 2016 IEEE 32nd international conference on data engineering (ICDE). IEEE, New York, pp 373–384

  • Zahur S, Evans D (2015) Obliv-C: a language for extensible data-oblivious computation. IACR Cryptol, ePrint Archive, Report 2015/1153

  • Zheng P, Huang J (2013) An efficient image homomorphic encryption scheme with small ciphertext expansion. In: Proceedings of the 21st ACM international conference on multimedia, pp 803–812

Download references

Acknowledgements

This work is supported by the National Key Research and Development Program of China (Grant No. 2018YFB0804702).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yong Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hassan, A., Liu, F., Wang, F. et al. Secure image classification with deep neural networks for IoT applications. J Ambient Intell Human Comput 12, 8319–8337 (2021). https://doi.org/10.1007/s12652-020-02565-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-020-02565-z

Keywords

Navigation