Andi Xu
Anhong Guo
ABSTRACT
Blind or Visually Impaired (BVI) people often encounter flat, inaccessible interfaces. Current solutions lack cost-effectiveness, portability, and robustness in real-world settings. We introduce VizLens, a fully-automated, full-stack mobile application powered by computer vision algorithms. The system is deployed and publicly available through the Apple App Store (https://vizlens.org/). From May to August 2023, we had 665 users, who uploaded 1,320 interface images. We aim to use it to study usage patterns and possible challenges BVI users may encounter with flat interfaces through a large-scale study in real-world settings. With in-depth analysis of user data and activity logs, our study will provide insights into BVI users’ interface interests, preferred assistance modes, and potential challenges due to system limitations or users’ diverse abilities. Our goal is to enhance the understanding of how BVI users interact with inaccessible, flat interfaces, and inform future assistive technology design.
- Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. 2006. SURF: Speeded Up Robust Features. In Computer Vision – ECCV 2006, Aleš Leonardis, Horst Bischof, and Axel Pinz (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 404–417.Google Scholar
Digital Library
- Google Cloud. 2023. Google Cloud Vision. https://cloud.google.com/visionGoogle Scholar
- Aira Tech Corp. 2022. Aira. https://aira.io/Google Scholar
- Be My Eyes. 2023. Be My Eyes. https://www.bemyeyes.com/Google Scholar
- Giovanni Fusco, Ender Tekin, R.E. Ladner, and James Coughlan. 2014. Using Computer Vision to Access Appliance Displays. ASSETS / Association for Computing Machinery. ACM Conference on Assistive Technologies 2014. https://doi.org/10.1145/2661334.2661404Google ScholarDigital Library
- Anhong Guo, Xiang ‘Anthony’ Chen, Haoran Qi, Samuel White, Suman Ghosh, Chieko Asakawa, and Jeffrey P. Bigham. 2016. VizLens: A Robust and Interactive Screen Reader for Interfaces in the Real World. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (Tokyo, Japan) (UIST ’16). Association for Computing Machinery, New York, NY, USA, 651–664. https://doi.org/10.1145/2984511.2984518Google ScholarDigital Library
- Anhong Guo, Jeeeun Kim, Xiang ‘Anthony’ Chen, Tom Yeh, Scott E. Hudson, Jennifer Mankoff, and Jeffrey P. Bigham. 2017. Facade: Auto-Generating Tactile Interfaces to Appliances. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 5826–5838. https://doi.org/10.1145/3025453.3025845Google ScholarDigital Library
- Anhong Guo, Junhan Kong, Michael Rivera, Frank F. Xu, and Jeffrey P. Bigham. 2019. StateLens: A Reverse Engineering Solution for Making Existing Dynamic Touchscreens Accessible. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST ’19). Association for Computing Machinery, New York, NY, USA, 371–385. https://doi.org/10.1145/3332165.3347873Google ScholarDigital Library
- Apple Machine Learning. 2023. Apple Machine Learning Vision. https://developer.apple.com/documentation/vision/Google Scholar
- Chen Liang, Yasha Iravantchi, Thomas Krolikowski, Ruijie Geng, Alanson P. Sample, and Anhong Guo. 2023. BrushLens: Hardware Interaction Proxies for Accessible Touchscreen Interface Actuation. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (San Francisco, CA, USA) (UIST ’23). Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3586183.3606730Google ScholarDigital Library
- Tim Morris, Paul Blenkhorn, Luke Crossey, Quang Ngo, Martin Ross, David Werner, and Christina Wong. 2006. Clearspeech: A Display Reader for the Visually Handicapped. IEEE Transactions on Neural Systems and Rehabilitation Engineering 14, 4 (2006), 492–500. https://doi.org/10.1109/TNSRE.2006.881538Google ScholarCross Ref
- Vladimir Vezhnevets, Vassili Sazonov, and Alla Andreeva. 2004. A Survey on Pixel-Based Skin Color Detection Techniques. (03 2004).Google Scholar
Index Terms
- Deploying VizLens: Characterizing User Needs, Preferences, and Challenges of Physical Interfaces Usage in the Wild
Recommendations
Making Real-World Interfaces Accessible Through Crowdsourcing, Computer Vision, and Fabrication
W4A '17: Proceedings of the 14th International Web for All ConferenceThe world is full of physical interfaces that are inaccessible to blind people, from microwaves and information kiosks to thermostats and checkout terminals. Blind people cannot independently use such devices without at least first learning their layout,...
Crowd-AI Systems for Non-Visual Information Access in the Real World
CHI EA '18: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing SystemsThe world is full of information, interfaces and environments that are inaccessible to blind people. When navigating indoors, blind people are often unaware of key visual information, such as posters, signs, and exit doors. When accessing specific ...
Crowd-AI Systems for Non-Visual Information Access in the Real World
UIST '18 Adjunct: Adjunct Proceedings of the 31st Annual ACM Symposium on User Interface Software and TechnologyThe world is full of information, interfaces and environments that are inaccessible to blind people. When navigating indoors, blind people are often unaware of key visual information, such as posters, signs, and exit doors. When accessing specific ...
Comments