skip to main content
research-article

Learning to Recognize Unmodified Lights with Invisible Features

Authors Info & Claims
Published:21 June 2019Publication History
Skip Abstract Section

Abstract

To enable accurate indoor localization at low cost, recent research in visible light positioning (VLP) proposed to employ existing ceiling lights as location landmarks, and use smartphone cameras or light sensors to identify the different lights using statistical visual/optical features. Despite the potential, we find such solutions are unreliable: the features are easily corrupted with a slight rotation of the smartphone, and are not discriminative enough for many practical light models with different size/shape/intensity. In this work, we propose Auto-Litell to resolve these critical challenges and make VLP truly robust. Auto-Litell builds a customized deep-learning neural network model to automatically distill the "invisible" visual features from the lights, which are resilient to smartphone orientation and light models. Moreover, Auto-Litell introduces a Light-CycleGAN to generate "fake" light images to augment the training data, so as to relieve human labors in data collection and labeling. We have implemented Auto-Litell as a real-time localization and navigation system on Android. Our experiments demonstrate Auto-Litell's high accuracy in discriminating the lights in the same building, and high reliability across a variety of practical usage scenarios.

References

  1. Amjad Almahairi, Sai Rajeshwar, Alessandro Sordoni, Philip Bachman, and Aaron C. Courville. 2018. Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data. (2018), 195--204.Google ScholarGoogle Scholar
  2. Jean Armstrong, Y Ahmet Sekercioglu, and Adrian Neild. 2013. Visible light positioning: a roadmap for international standardization. IEEE Communications Magazine 51, 12 (2013), 68--73.Google ScholarGoogle ScholarCross RefCross Ref
  3. Paramvir Bahl and Venkata N Padmanabhan. 2000. RADAR: An in-building RF-based user location and tracking system. In Proc. of IEEE INFOCOM, Vol. 2. 775--784.Google ScholarGoogle Scholar
  4. Huiwen Chang, Jingwan Lu, Fisher Yu, and Adam Finkelstein. 2018. Pairedcyclegan: Asymmetric style transfer for applying and removing makeup. In Proc. of IEEE CVPR.Google ScholarGoogle ScholarCross RefCross Ref
  5. Cheng-Lun Chen and Shao-Hua Lin. 2011. Intelligent color temperature estimation using fuzzy neural network with application to automatic white balance. Expert Syst. Appl. 38, 6, 7718--7728. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Yin Chen, Dimitrios Lymberopoulos, Jie Liu, and Bodhi Priyantha. 2012. FM-based indoor localization. In Proc. of ACM Mobisys. 169--182. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Yin Chen, Dimitrios Lymberopoulos, Jie Liu, and Bodhi Priyantha. 2012. FM-based indoor localization. In The 10th International Conference on Mobile Systems, Applications, and Services, MobiSys'12, Ambleside, United Kingdom - June 25 - 29, 2012. 169--182. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. 2016. Image Super-Resolution Using Deep Convolutional Networks. IEEE Trans. Pattern Anal. Mach. Intell. 38, 2 (2016), 295--307. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Chao Dong, Chen Change Loy, and Xiaoou Tang. 2016. Accelerating the super-resolution convolutional neural network. In ECCV. Springer, 391--407.Google ScholarGoogle Scholar
  10. Hany Farid. 2001. Blind inverse gamma correction. IEEE Transactions on Image Processing 10, 10 (2001), 1428--1433. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Ross Girshick. 2015. Fast r-cnn. In CVPR. 1440--1448. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR. 580--587.Google ScholarGoogle Scholar
  13. Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In AISTATS. 249--256.Google ScholarGoogle Scholar
  14. Daniel B Goldman. 2010. Vignette and exposure calibration and compensation. IEEE transactions on pattern analysis and machine intelligence 32, 12 (2010), 2276--2288. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In NIPS. 2672--2680. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. Mask r-cnn. In Proc. of IEEE ICCV. IEEE, 2980--2988.Google ScholarGoogle ScholarCross RefCross Ref
  17. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proc. of IEEE CVPR. 770--778.Google ScholarGoogle ScholarCross RefCross Ref
  18. Yu-Feng Hsu and Shih-Fu Chang. 2007. Image Splicing Detection using Camera Response Function Consistency and Automatic Segmentation. In Proc. of IEEE ICME. 28--31.Google ScholarGoogle ScholarCross RefCross Ref
  19. Yiqing Hu, Yan Xiong, Wenchao Huang, Xiang-Yang Li, Panlong Yang, Yanan Zhang, and XuFei Mao. 2018. Lightitude: Indoor Positioning Using Uneven Light Intensity Distribution. IMWUT 2, 2 (2018), 67:1--67:25. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167 (2015).Google ScholarGoogle Scholar
  21. Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. 2017. Image-to-Image Translation with Conditional Adversarial Networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017. 5967--5976.Google ScholarGoogle Scholar
  22. Zhenyuan Jia, Jinghao Yang, Wei Liu, Fuji Wang, Yang Liu, Lingli Wang, Chaonan Fan, and Kai Zhao. 2015. Improved camera calibration method based on perpendicularity compensation for binocular stereo vision measurement system. Optics express 23, 12 (2015), 15205--15223.Google ScholarGoogle Scholar
  23. Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. In ECCV. Springer, 694--711.Google ScholarGoogle Scholar
  24. Andrew Evan Kamholz, Eric A Schilling, and Paul Yager. 2001. Optical measurement of transverse molecular diffusion in a microchannel. Biophysical Journal 80, 4 (2001), 1967--1972.Google ScholarGoogle ScholarCross RefCross Ref
  25. Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv:1312.6114 (2013).Google ScholarGoogle Scholar
  26. Manikanta Kotaru, Kiran Raj Joshi, Dinesh Bharadia, and Sachin Katti. 2015. SpotFi: Decimeter Level Localization Using WiFi. In Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication, SIGCOMM 2015, London, United Kingdom, August 17-21, 2015. 269--282. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In NIPS. 1097--1105. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Ye-Sheng Kuo, Pat Pannuto, Ko-Jen Hsiao, and Prabal Dutta. 2014. Luxapose: Indoor positioning with mobile phones and visible light. In Proc. of ACM MobiCom. 447--458. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Wei-Sheng Lai, Jia-Bin Huang, Narendra Ahuja, and Ming-Hsuan Yang. 2017. Deep laplacian pyramid networks for fast and accurate superresolution. In Proc. of IEEE CVPR, Vol. 2. 5.Google ScholarGoogle Scholar
  30. Chongxuan Li, Taufik Xu, Jun Zhu, and Bo Zhang. 2017. Triple Generative Adversarial Nets. (2017), 4091--4101. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Liqun Li, Pan Hu, Chunyi Peng, Guobin Shen, and Feng Zhao. 2014. Epsilon: A Visible Light Based Positioning System.. In NSDI, Vol. 14. 331--343. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Xiang Li, Shengjie Li, Daqing Zhang, Jie Xiong, Yasha Wang, and Hong Mei. 2016. Dynamic-MUSIC: accurate device-free indoor localization. In Proc. of ACM UbiComp. 196--207. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Yi-Qun Li, Yi Dong, and Xiaofeng Xu. 2012. Color temperature tunable white light source. US Patent 8,203,260.Google ScholarGoogle Scholar
  34. Min Lin, Qiang Chen, and Shuicheng Yan. 2013. Network in network. arXiv:1312.4400 (2013).Google ScholarGoogle Scholar
  35. Makets and Markets. 2017. Indoor Location Market worth 40.99 Billion USD by 2022. https://www.marketsandmarkets.com/PressReleases/indoor-location.asp.Google ScholarGoogle Scholar
  36. Xudong Mao, Qing Li, Haoran Xie, Raymond Y. K. Lau, and Zhen Wang. 2016. Multi-class Generative Adversarial Networks with the L2 Loss Function. CoRR abs/1611.04076 (2016).Google ScholarGoogle Scholar
  37. Michael F Mathieu, Junbo Jake Zhao, Junbo Zhao, Aditya Ramesh, Pablo Sprechmann, and Yann LeCun. 2016. Disentangling factors of variation in deep representation using adversarial training. In NIPS. 5040--5048. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Kun Qian, Chenshu Wu, Zheng Yang, Yunhao Liu, and Kyle Jamieson. 2017. Widar: Decimeter-Level Passive Tracking via Velocity Monitoring with Commodity Wi-Fi. In Proceedings of the 18th ACM International Symposium on Mobile Ad Hoc Networking and Computing, Chennai, India, July 10-14, 2017. 6:1--6:10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Kun Qian, Chenshu Wu, Yi Zhang, Guidong Zhang, Zheng Yang, and Yunhao Liu. 2018. Widar2.0: Passive Human Tracking with a Single Wi-Fi Link. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys 2018, Munich, Germany, June 10-15, 2018. 350--361. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. John C Russ. 2016. The image processing handbook. CRC press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. 2018. Inverted residuals and linear bottlenecks: Mobile networks for classification, detection and segmentation. arXiv:1801.04381 (2018).Google ScholarGoogle Scholar
  42. Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. CoRR abs/1312.6034 (2013).Google ScholarGoogle Scholar
  43. Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2014).Google ScholarGoogle Scholar
  44. Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A Alemi. 2017. Inception-v4, inception-resnet and the impact of residual connections on learning.. In AAAI, Vol. 4. 12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proc. of IEEE CVPR. 1--9.Google ScholarGoogle ScholarCross RefCross Ref
  46. Roger Tsai. 1987. A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE Journal on Robotics and Automation 3, 4 (1987), 323--344.Google ScholarGoogle ScholarCross RefCross Ref
  47. Martin JL Turner, A Abbey, M Arnaud, M Balasini, M Barbera, E Belsole, PJ Bennie, JP Bernard, GF Bignami, M Boer, et al. 2001. The European photon imaging camera on XMM-Newton: the MOS cameras. Astronomy & Astrophysics 365, 1 (2001), L27--L35.Google ScholarGoogle ScholarCross RefCross Ref
  48. Dmitry Ulyanov, Andrea Vedaldi, and Victor S. Lempitsky. 2016. Instance Normalization: The Missing Ingredient for Fast Stylization. CoRR abs/1607.08022.Google ScholarGoogle Scholar
  49. Deepak Vasisht, Swarun Kumar, and Dina Katabi. 2016. Decimeter-Level Localization with a Single WiFi Access Point.. In Proc. of USENIX NSDI, Vol. 16. 165--178. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Allen Jong-Woei Whang, Yi-Yung Chen, and Yuan-Ting Teng. 2009. Designing uniform illumination systems by surface-tailored lens and configurations of LED arrays. Journal of display technology 5, 3 (2009), 94--103.Google ScholarGoogle ScholarCross RefCross Ref
  51. Bo Xie, Kongyang Chen, Guang Tan, Mingming Lu, Yunhuai Liu, Jie Wu, and Tian He. 2016. LIPS: A light intensity--based positioning system for indoor environments. ACM TOSN 12, 4 (2016), 28. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Bo Xie, Guang Tan, and Tian He. 2015. Spinlight: A high accuracy and robust light positioning system for indoor applications. In Proc. of ACM SenSys. 211--223. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Jie Xiong and Kyle Jamieson. 2013. ArrayTrack: A Fine-Grained Indoor Location System. In Proc. of USENIX NSDI. 71--84. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Han Xu, Zheng Yang, Zimu Zhou, Longfei Shangguan, Ke Yi, and Yunhao Liu. 2016. Indoor localization via multi-modal sensing on smartphones. In Proc. of ACM UbiComp. 208--219. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Qiang Xu, Rong Zheng, and Steve Hranilovic. 2015. IDyLL: indoor localization using inertial and light sensors on smartphones. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp 2015, Osaka, Japan, September 7-11, 2015. 307--318. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Jianwei Yang, Anitha Kannan, Dhruv Batra, and Devi Parikh. 2017. LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generation. CoRR abs/1703.01560 (2017).Google ScholarGoogle Scholar
  57. Zhice Yang, Zeyu Wang, Jiansong Zhang, Chenyu Huang, and Qian Zhang. 2015. Wearables can afford: Light-weight indoor positioning with visible light. In Proc. of ACM Mobisys. 317--330. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. DING Ying, FAN Jingtao, QUAN Wei, and HAN Cheng. 2017. Nonlinear compensation for optical vignetting in vision systems. Journal of Tsinghua University (Science and Technology) 57, 7 (2017), 702--706.Google ScholarGoogle Scholar
  59. Fisher Yu, Vladlen Koltun, and Thomas A Funkhouser. 2017. Dilated Residual Networks.. In CVPR, Vol. 2. 3.Google ScholarGoogle Scholar
  60. Chi Zhang and Xinyu Zhang. 2016. LiTell: robust indoor localization using unmodified light fixtures. In Proc. of ACM MobiCom. 230--242.Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Chi Zhang and Xinyu Zhang. 2017. Pulsar: Towards ubiquitous visible light localization. In Proc. of ACM MobiCom. 208--221. Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Jialiang Zhang, Chi Zhang, Xinyu Zhang, and Suman Banerjee. 2016. Towards a visible light network architecture for continuous communication and localization. In Proceedings of the 3rd Workshop on Visible Light Communication Systems, VLCS@MobiCom 2016, New York City, NY, USA, October 3-7, 2016. 49--54. Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Zhengyou Zhang. 1999. Flexible Camera Calibration by Viewing a Plane from Unknown Orientations. In ICCV. 666--673.Google ScholarGoogle Scholar
  64. Bo Zhao, Bo Chang, Zequn Jie, and Leonid Sigal. 2018. Modular Generative Adversarial Networks. (2018), 157--173.Google ScholarGoogle Scholar
  65. Jian Zhao, Lin Xiong, Panasonic Karlekar Jayashree, Jianshu Li, Fang Zhao, Zhecan Wang, Panasonic Sugiri Pranata, Panasonic Shengmei Shen, Shuicheng Yan, and Jiashi Feng. 2017. Dual-agent gans for photorealistic and identity preserving profile face synthesis. In NIPS. 66--76. Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. A. Zhou, X. Zhang, and H. Ma. 2017. Beam-forecast: Facilitating mobile 60 GHz networks via model-driven beam steering. In IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.Google ScholarGoogle Scholar
  67. Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. 2016. Learning deep features for discriminative localization. In Proc. of IEEE CVPR. 2921--2929.Google ScholarGoogle ScholarCross RefCross Ref
  68. Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. 2017. Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks. (2017), 2242--2251.Google ScholarGoogle Scholar
  69. Shilin Zhu and Xinyu Zhang. 2017. Enabling High-Precision Visible Light Localization in Today's Buildings. In Proc. of ACM MobiSys. 96--108. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Learning to Recognize Unmodified Lights with Invisible Features

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
      Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 3, Issue 2
      June 2019
      802 pages
      EISSN:2474-9567
      DOI:10.1145/3341982
      Issue’s Table of Contents

      Copyright © 2019 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 21 June 2019
      • Accepted: 1 April 2019
      • Revised: 1 February 2019
      • Received: 1 November 2018
      Published in imwut Volume 3, Issue 2

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader