Weigao Su
Abstract
Motion sensors in modern smartphones have been exploited for audio eavesdropping in loudspeaker mode due to their sensitivity to vibrations. In this paper, we further move one step forward to explore the feasibility of using built-in accelerometer to eavesdrop on the telephone conversation of caller/callee who takes the phone against cheek-ear and design our attack Vibphone. The inspiration behind Vibphone is that the speech-induced vibrations (SIV) can be transmitted through the physical contact of phone-cheek to accelerometer with the traces of voice content. To this end, Vibphone faces three main challenges: i) Accurately detecting SIV signals from miscellaneous disturbance; ii) Combating the impact of device diversity to work with a variety of attack scenarios; and iii) Enhancing feature-agnostic recognition model to generalize to newly issued devices and reduce training overhead. To address these challenges, we first conduct an in-depth investigation on SIV features to figure out the root cause of device diversity impacts and identify a set of critical features that are highly relevant to the voice content retained in SIV signals and independent of specific devices. On top of these pivotal observations, we propose a combo method that is the integration of extracted critical features and deep neural network to recognize speech information from the spectrogram representation of acceleration signals. We implement the attack using commodity smartphones and the results show it is highly effective. Our work brings to light a fundamental design vulnerability in the vast majority of currently deployed smartphones, which may put people's speech privacy at risk during phone calls. We also propose a practical and effective defense solution. We validate that it is feasible to prevent audio eavesdropping by using random variation of sampling rate.
- Xuanang Feng, Hiroki Shimokubo, Eisuke Kita: Personal Identification Through Pedestrians' Behavior. Rev. Socionetwork Strateg. 12(2): 237--252 (2018)Google Scholar
Cross Ref
- Ivars Blums, Hans Weigand: A Financial Reporting Ontology for Market, Exchange, and Enterprise Shared Information Systems. PoEM 2019: 83--99.Google Scholar
- Liang Cai, Hao Chen: TouchLogger: Inferring Keystrokes on Touch Screen from Smartphone Motion. HotSec 2011.Google Scholar
- Sashank Narain, Triet D. Vo-Huu, Kenneth Block, Guevara Noubir: Inferring User Routes and Locations Using Zero-Permission Mobile Sensors. IEEE Symposium on Security and Privacy 2016: 397--413.Google Scholar
- Youngtae Yang, Byunggyu Lee, Jun Soo Cho, Suhwan Kim, Hyunjoong Lee: A Digital Capacitive MEMS Microphone for Speech Recognition With Fast Wake-Up Feature Using a Sound Activity Detector. IEEE Trans. Circuits Syst. II Express Briefs 67-II(9): 1509--1513 (2020).Google Scholar
- Yan Michalevsky, Dan Boneh, Gabi Nakibly: Gyrophone: Recognizing Speech from Gyroscope Signals. USENIX Security Symposium 2014: 1053--1067.Google Scholar
- Li Zhang, Parth H. Pathak, Muchen Wu, Yixin Zhao, Prasant Mohapatra: AccelWord: Energy Efficient Hotword Detection through Accelerometer. MobiSys 2015: 301--315.Google Scholar
- S. Abhishek Anand, Nitesh Saxena: Speechless: Analyzing the Threat to Speech Privacy from Smartphone Motion Sensors. IEEE Symposium on Security and Privacy 2018: 1000--1017.Google Scholar
- Jun Han, Albert Jin Chung, Patrick Tague: Pitchln: eavesdropping via intelligible speech reconstruction using non-acoustic sensor fusion. IPSN 2017: 181--192.Google Scholar
- S. Abhishek Anand, Chen Wang, Jian Liu, Nitesh Saxena, Yingying Chen: Spearphone: A Speech Privacy Exploit via Accelerometer-Sensed Reverberations from Smartphone Loudspeakers. arXiv preprint:1907.05972, 2019.Google Scholar
- Zhongjie Ba, Tianhang Zheng, Xinyu Zhang, Zhan Qin, Baochun Li, Xue Liu, Kui Ren: Learning-based Practical Smartphone Eavesdropping with Built-in Accelerometer. NDSS 2020.Google Scholar
- Wenjun Jiang, Chenglin Miao, Fenglong Ma, Shuochao Yao, Yaqing Wang, Ye Yuan, Hongfei Xue, Chen Song, Xin Ma, Dimitrios Koutsonikolas, Wenyao Xu, Lu Su: Towards Environment Independent Device Free Human Activity Recognition. MobiCom 2018: 289--304.Google Scholar
- Mingmin Zhao, Shichao Yue, Dina Katabi, Tommi S. Jaakkola, Matt T. Bianchi: Learning Sleep Stages from Radio Signals: A Conditional Adversarial Architecture. ICML 2017: 4100--4109.Google Scholar
- Isaac Griswold-Steiner, Zachary LeFevre, Abdul Serwadda: Smartphone speech privacy concerns from side-channel attacks on facial biomechanics. Comput. Secur. 100: 102110 (2021).Google ScholarDigital Library
- Soren Becker, Marcel Ackermann, Sebastian Lapuschkin, Klaus-Robert Müller, Wojciech Samek: Interpreting and Explaining Deep Neural Networks for Classification of Audio Signals. arXiv preprint:1807.03418, 2018.Google Scholar
- Jian Liu, Chen Wang, Yingying Chen, Nitesh Saxena: VibWrite: Towards Finger-input Authentication on Ubiquitous Surfaces via Physical Vibration. CCS 2017: 73--87.Google Scholar
- Sanorita Dey, Nirupam Roy, Wenyuan Xu, Romit Roy Choudhury, Srihari Nelakuditi: AccelPrint: Imperfections of Accelerometers Make Smartphones Trackable. NDSS 2014.Google Scholar
- Frederico Soares Cabral, Hidekazu Fukai, Satoshi Tamura: Feature Extraction Methods Proposed for Speech Recognition Are Effective on Road Condition Monitoring Using Smartphone Inertial Sensors. Sensors 19(16): 3481 (2019).Google ScholarCross Ref
- Ron Kohavi, George H. John: Wrappers for Feature Subset Selection. Artif. Intell. 97(1-2): 273--324 (1997).Google ScholarDigital Library
- Avrim Blum, Pat Langley: Selection of Relevant Features and Examples in Machine Learning. Artif. Intell. 97(1-2): 245--271 (1997)Google ScholarDigital Library
- Isabelle Guyon, Andre Elisseeff: An Introduction to Variable and Feature Selection. J. Mach. Learn. Res. 3: 1157--1182 (2003).Google ScholarDigital Library
- Anupam Das, Nikita Borisov, Matthew Caesar: Do You Hear What I Hear?: Fingerprinting Smart Devices Through Embedded Acoustic Components. CCS 2014: 441--452.Google Scholar
- Kyungho Joo, Wonsuk Choi, Dong Hoon Lee: Hold the Door! Fingerprinting Your Car Key to Prevent Keyless Entry Car Theft. NDSS 2020.Google ScholarCross Ref
- Stephen McAdams: Perspectives on the Contribution of Timbre to Musical Structure. Comput. Music. J. 23(3): 85--102 (1999).Google ScholarDigital Library
- Gao Huang, Zhuang Liu, Laurens van der Maaten, Kilian Q. Weinberger: Densely Connected Convolutional Networks. CVPR 2017: 2261--2269.Google Scholar
- "LibXtract". http://libxtract.sourceforge.net/Google Scholar
- Philip Marquardt, Arunabh Verma, Henry Carter, Patrick Traynor: (sp)iPhone: decoding vibrations from nearby keyboards using mobile phone accelerometers. CCS 2011: 551--562.Google Scholar
- Zhi Xu, Kun Bai, Sencun Zhu: TapLogger: inferring user inputs on smartphone touchscreens using on-board motion sensors. WISEC 2012: 113--124.Google Scholar
- Emiliano Miluzzo, Alexander Varshavsky, Suhrid Balakrishnan, Romit Roy Choudhury: Tapprints: your finger taps have fingerprints. MobiSys 2012: 323--336.Google Scholar
- Emmanuel Owusu, Jun Han, Sauvik Das, Adrian Perrig, Joy Zhang: ACCessory: password inference using accelerometers on smartphones. HotMobile 2012: 9.Google Scholar
- Chen Wang, Xiaonan Guo, Yan Wang, Yingying Chen, Bo Liu: Friend or Foe?: Your Wearable Devices Reveal Your Personal PIN. AsiaCCS 2016: 189--200.Google Scholar
- Anupam Das, Nikita Borisov, Matthew Caesar: Tracking Mobile Web Users Through Motion Sensors: Attacks and Defenses. NDSS 2016.Google Scholar
- Avesta Hojjati, Anku Adhikari, Katarina Struckmann, Edward Chou, Thi Ngoc Tho Nguyen, Kushagra Madan, Marianne Southall Winslett, Carl A. Gunter, William P. King: Leave Your Phone at the Door: Side Channels that Reveal Factory Floor Secrets. CCS 2016: 883--894.Google Scholar
- Chen Song, Feng Lin, Zhongjie Ba, Kui Ren, Chi Zhou, Wenyao Xu: My Smartphone Knows What You Print: Exploring Smartphone-based Side-channel Attacks Against 3D Printers. CCS 2016: 895--907.Google Scholar
- Jun Han, Emmanuel Owusu, Le T. Nguyen, Adrian Perrig, Joy Zhang: ACComplice: Location inference using accelerometers on smartphones. COMSNETS 2012: 1--9.Google Scholar
- Sarfraz Nawaz, Cecilia Mascolo: Mining users' significant driving routes with low-power sensors. SenSys 2014: 236--250.Google Scholar
- Sashank Narain, Triet D. Vo-Huu, Kenneth Block, Guevara Noubir: Inferring User Routes and Locations Using Zero-Permission Mobile Sensors. IEEE Symposium on Security and Privacy 2016: 397--413.Google Scholar
- He Wang, Ted Tsung-Te Lai, Romit Roy Choudhury: MoLe: Motion Leaks through Smartwatch Sensors. MobiCom 2015: 155--166.Google Scholar
- Tuo Yu, Haiming Jin, Klara Nahrstedt: WritingHacker: audio based eavesdropping of handwriting via mobile devices. UbiComp 2016: 463--473.Google Scholar
- Xiangyu Xu, Jiadi Yu, Yingying Chen, Qin Hua, Yanmin Zhu, Yi-Chao Chen, Minglu Li: TouchPass: towards behavior-irrelevant on-touch user authentication on smartphones leveraging vibrations. MobiCom 2020: 24:1-24:13.Google Scholar
- Nirupam Roy, Romit Roy Choudhury: Listening through a Vibration Motor. MobiSys 2016: 57--69.Google Scholar
- Sensor Overview: https://developer.android.com/guide/topics/sensors/.Google Scholar
- Uses-permission: https://developer.android.com/guide/topics/manifest/uses-permission-element.Google Scholar
- Ingo R. Titze and Daniel W. Martin: Principles of voice production. Acoustical Society of America Journal, vol. 104, p. 1148, 1998.Google ScholarCross Ref
- Ronald J. Baken and Robert F. Orlikoff: Clinical measurement of speech and voice. Cengage Learning, 2000.Google Scholar
- Kevin J. Coakley, Paul D. Hale: Alignment of noisy signals. IEEE Trans. Instrum. Meas. 50(1): 141--149 (2001).Google ScholarCross Ref
Index Terms
- Towards Device Independent Eavesdropping on Telephone Conversations with Built-in Accelerometer
Recommendations
Accelerometer-based smartphone eavesdropping
MobiCom '20: Proceedings of the 26th Annual International Conference on Mobile Computing and NetworkingIn this demonstration, we show that audio signals emitted by a smartphone speaker can be captured by the accelerometer on the same smartphone, and accelerometers on recently released smartphones can cover most of the fundamental frequency band of adult ...
Device-Independent Smartphone Eavesdropping Jointly Using Accelerometer and Gyroscope
Eavesdropping via inertial measurement units (IMUs) has brought growing concerns over smartphone users’ privacy. In such attacks, adversaries utilize IMUs, including accelerometers and gyroscopes, which require zero permissions for access to ...
Towards defending eavesdropping on NFC
Successful defense against eavesdropping on near-field communication (NFC) vastly depends on careful analysis of eavesdropping and adopting a suitable defense mechanism based on the analysis. However, such an analysis and such an adoption of a defense ...
Comments