Ali Selman Aydin
ABSTRACT
People with low vision who use screen magnifiers to interact with computing devices find it very challenging to interact with dynamically changing digital content such as videos, since they do not have the luxury of time to manually move, i.e., pan the magnifier lens to different regions of interest (ROIs) or zoom into these ROIs before the content changes across frames.
In this paper, we present SViM, a first of its kind screen-magnifier interface for such users that leverages advances in computer vision, particularly video saliency models, to identify salient ROIs in videos. SViM's interface allows users to zoom in/out of any point of interest, switch between ROIs via mouse clicks and provides assistive panning with the added flexibility that lets the user explore other regions of the video besides the ROIs identified by SViM.
Subjective and objective evaluation of a user study with 13 low vision screen magnifier users revealed that overall the participants had a better user experience with SViM over extant screen magnifiers, indicative of the former's promise and potential for making videos accessible to low vision screen magnifier users.
- Federal Communication Act. 2019. Audio Description. https://www.fcc.gov/consumers/guides/video-descriptionGoogle Scholar
- AFB. [n.d.]. Screen Magnification Systems. https://www.afb.org/node/16207/screen-magnification-systems.Google Scholar
- Apple. [n.d.]. Apple iOS Accessibility Features. https://www.apple.com/accessibility/iphone/vision/.Google Scholar
- Apple. [n.d.]. Vision Accessibility - Mac - Apple. https://www.apple.com/accessibility/mac/vision/.Google Scholar
- Apple. 2019. Use Magnifier with your iPhone or iPad. https://support.apple.com/en-us/HT209517.Google Scholar
- Cagdas Bak, Aysun Kocak, Erkut Erdem, and Aykut Erdem. 2018. Spatio-temporal saliency networks for dynamic saliency prediction. IEEE Transactions on Multimedia 20, 7 (2018), 1688--1698.Google ScholarCross Ref
- Loris Bazzani, Hugo Larochelle, and Lorenzo Torresani. 2016. Recurrent mixture density network for spatiotemporal visual attention. arXiv preprint arXiv:1603.08199 (2016).Google Scholar
- Jean-Baptiste Bernard, Emilien Tlapale, Geraldine Faure, Eric Castet, and Pierre Kornprobst. 2008. Navisio: Towards an integrated reading aid system for low vision patients.Google Scholar
- Jeffrey P Bigham. 2014. Making the web easier to see with opportunistic accessibility improvement. In Proceedings of the 27th annual ACM symposium on User interface software and technology. ACM, 117--122.Google ScholarDigital Library
- Syed Masum Billah, Vikas Ashok, Donald E Porter, and IV Ramakrishnan. 2018. SteeringWheel: A Locality-Preserving Magnification Interface for Low Vision Web Browsing. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 20.Google ScholarDigital Library
- Ali Borji. 2018. Saliency prediction in the deep learning era: An empirical investigation. arXiv preprint arXiv:1810.03716 (2018).Google Scholar
- James V Bradley. 1958. Complete counterbalancing of immediate sequential effects in a Latin square design. J. Amer. Statist. Assoc. 53, 282 (1958), 525--528.Google ScholarCross Ref
- Souad Chaabouni, Jenny Benois-Pineau, and Chokri Ben Amar. 2016. Transfer learning with deep networks for saliency prediction in natural video. In 2016 IEEE International Conference on Image Processing (ICIP). IEEE, 1604--1608.Google ScholarCross Ref
- Pei-Yu Chi, Sally Ahn, Amanda Ren, Mira Dontcheva, Wilmot Li, and Björn Hartmann. 2012. MixT: automatic generation of step-by-step mixed media tutorials. In Proceedings of the 25th annual ACM symposium on User interface software and technology. ACM, 93--102.Google ScholarDigital Library
- Michael Christen and Mathias Abegg. 2017. The effect of magnification and contrast on reading performance in different types of simulated low vision. Journal of Eye Movement Research JEMR 10, 2 (2017).Google Scholar
- Ashley D Deemer, Bonnielin K Swenor, Kyoko Fujiwara, James T Deremeik, Nicole C Ross, Danielle M Natale, Chris K Bradley, Frank S Werblin, and Robert W Massof. 2019. Preliminary Evaluation of Two Digital Image Processing Strategies for Head-Mounted Magnification for Low Vision Patients. Translational vision science & technology 8, 1 (2019), 23--23.Google Scholar
- eSight. [n.d.]. Electronic Glasses for Blind People | From 20/200 Vision to 20/20! | eSight. https://www.esighteyewear.com.Google Scholar
- explore 5. [n.d.]. https://store.humanware.com/hus/explore-5-handheld-electronic-magnifier.html.Google Scholar
- American Foundation for the Blind. [n.d.]. CCTVs/VideoMagnifiers. https://www.afb.org/blindness-and-low-vision/using-technology/assistive-technology-products/video-magnifiers.Google Scholar
- American Foundation for the Blind. [n.d.]. Low Vision Optical Devices. https://www.afb.org/node/16207/low-vision-optical-devices.Google Scholar
- Google. [n.d.]. https://support.google.com/accessibility/android/answer/6006949?hl=en&ref_topic=9079043.Google Scholar
- Google. [n.d.]. Google Android Accessibility Features. https://support.google.com/accessibility/android/answer/6006949.Google Scholar
- Siavash Gorji and James J Clark. 2018. Going from image to video saliency: Augmenting image salience with dynamic attentional push. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7501--7511.Google ScholarCross Ref
- Elyse C Hallett, Wayne Dick, Tom Jewett, and Kim-Phuong L Vu. 2017. How Screen Magnification with and without Word-Wrapping Affects the User Experience of Adults with Low Vision. In International Conference on Applied Human Factors and Ergonomics. Springer, 665--674.Google Scholar
- Makoto J Hirayama. 2018. A book reading magnifier for low vision persons on smartphones and tablets. In Advanced Image Technology (IWAIT), 2018 International Workshop on. IEEE, 1--4.Google ScholarCross Ref
- Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735--1780.Google ScholarDigital Library
- Xun Huang, Chengyao Shen, Xavier Boix, and Qi Zhao. 2015. Salicon: Reducing the semantic gap in saliency prediction by adapting deep neural networks. In Proceedings of the IEEE International Conference on Computer Vision. 262--270.Google ScholarDigital Library
- i See. [n.d.]. https://irie-at.com/product/i-see-19/.Google Scholar
- IrisVision. [n.d.]. IrisVision. http://www.irisvision.com/.Google Scholar
- Lai Jiang, Mai Xu, Tie Liu, Minglang Qiao, and Zulin Wang. 2018. Deepvs: A deep learning based video saliency prediction approach. In Proceedings of the European Conference on Computer Vision (ECCV). 602--617.Google ScholarDigital Library
- Rudolph Emil Kalman. 1960. A new approach to linear filtering and prediction problems. Journal of basic Engineering 82, 1 (1960), 35--45.Google ScholarCross Ref
- Petros Koutras and Petros Maragos. 2019. SUSiNet: See, Understand and Summarize it. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 0--0.Google ScholarCross Ref
- Raja S Kushalnagar, Stephanie A Ludie, and Poorna Kushalnagar. 2011. Multiview platform: an accessible live classroom viewing approach for low vision students. In The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility. ACM, 267--268.Google ScholarDigital Library
- Qiuxia Lai, Wenguan Wang, Hanqiu Sun, and Jianbing Shen. 2019. Video Saliency Prediction using Spatiotemporal Residual Attentive Networks. IEEE Transactions on Image Processing (2019).Google Scholar
- Wei-Sheng Lai, Yujia Huang, Neel Joshi, Christopher Buehler, Ming-Hsuan Yang, and Sing Bing Kang. 2017. Semantic-driven generation of hyperlapse from 360 degree video. IEEE transactions on visualization and computer graphics 24, 9 (2017), 2610--2621.Google Scholar
- Stuart Lloyd. 1982. Least squares quantization in PCM. IEEE transactions on information theory 28, 2 (1982), 129--137.Google ScholarDigital Library
- VideoLAN media player. [n.d.]. https://www.videolan.org/index.html.Google Scholar
- Microsoft. 2019. Use Magnifier to make things on the screen easier to see - Windows Help. https://support.microsoft.com/en-us/help/11542/windows-use-magnifier.Google Scholar
- James Norris, Holger Schnädelbach, and Guoping Qiu. 2012. CamBlend: an object focused collaboration tool. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 627--636.Google ScholarDigital Library
- Junting Pan, Elisa Sayrol, Xavier Giro-i Nieto, Kevin McGuinness, and Noel E O'Connor. 2016. Shallow and deep convolutional networks for saliency prediction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 598--606.Google ScholarCross Ref
- Robert J Peters, Asha Iyer, Laurent Itti, and Christof Koch. 2005. Components of bottom-up gaze allocation in natural images. Vision research 45, 18 (2005), 2397--2416.Google Scholar
- Shrinivas Pundlik, Huaqi Yi, Rui Liu, Eli Peli, and Gang Luo. 2016. Magnifying smartphone screen using google glass for low-vision users. IEEE Transactions on Neural Systems and Rehabilitation Engineering 25, 1 (2016), 52--61.Google ScholarCross Ref
- Freedom Scientific. [n.d.]. MAGic® - Freedom Scientific. https://www.freedomscientific.com/products/software/magic/.Google Scholar
- Paul J Seakins, Jonathan D Cartwright, David J Haughey, David N Lovegrove, and Darryl J Best. 2009. Image magnifier for the visually impaired. US Patent App. 11/578,486.Google Scholar
- Vincent Sitzmann, Ana Serrano, Amy Pavel, Maneesh Agrawala, Diego Gutierrez, Belen Masia, and Gordon Wetzstein. 2018. Saliency in VR: How do people explore virtual environments? IEEE transactions on visualization and computer graphics 24, 4 (2018), 1633--1642.Google Scholar
- SMI. [n.d.]. SMI Eye Tracking Glasses 2 Wireless.Google Scholar
- Lee Stearns, Victor DeSouza, Jessica Yin, Leah Findlater, and Jon E Froehlich. 2017. Augmented reality magnification for low vision users with the microsoft hololens and a finger-worn camera. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, 361--362.Google ScholarDigital Library
- Lee Stearns, Leah Findlater, and Jon E Froehlich. 2018. Design of an Augmented Reality Magnification Aid for Low Vision Users. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, 28--39.Google ScholarDigital Library
- Video Studio. [n.d.]. https://www.videostudiopro.com/en/tips/basics/zoom-in-on-video/.Google Scholar
- Yu-Chuan Su and Kristen Grauman. 2017. Making 360 video watchable in 2d: Learning videography for click free viewing. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 1368--1376.Google ScholarCross Ref
- Yu-Chuan Su, Dinesh Jayaraman, and Kristen Grauman. 2016. Pano2Vid: Automatic Cinematography for Watching 360° Videos. In Proceedings of the Asian Conference on Computer Vision (ACCV).Google Scholar
- Mary Frances Theofanos and Janice Ginny Redish. 2005. Helping low-vision and other users with web sites that meet their needs: Is one site for all feasible? Technical communication 52, 1 (2005), 9--20.Google Scholar
- Wenguan Wang, Jianbing Shen, Jianwen Xie, Ming-Ming Cheng, Haibin Ling, and Ali Borji. 2019. Revisiting video saliency prediction in the deep learning era. IEEE transactions on pattern analysis and machine intelligence (2019).Google ScholarDigital Library
- WHO. [n.d.]. Low Vision Characterization. https://www.who.int/blindness/Change%20the%20Definition%20of%20Blindness.pdfGoogle Scholar
- Yuhang Zhao, Edward Cutrell, Christian Holz, Meredith Ringel Morris, Eyal Ofek, and Andrew D Wilson. 2019. SeeingVR: A Set of Tools to Make Virtual Reality More Accessible to People with Low Vision. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, 111.Google ScholarDigital Library
- Yuhang Zhao, Sarit Szpiro, and Shiri Azenkot. 2015. Foresee: A customizable head-mounted vision enhancement system for people with low vision. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility. ACM, 239--249.Google ScholarDigital Library
- Yuhang Zhao, Sarit Szpiro, Jonathan Knighten, and Shiri Azenkot. 2016. CueSee: exploring visual cues for people with low vision to facilitate a visual search task. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 73--84.Google ScholarDigital Library
- Zoomax. [n.d.]. https://www.zoomax.com/low-vision-products/easy-to-use-desktop-video-magnifier-Panda.html.Google Scholar
- Zoomtext. [n.d.]. Zoom Text Magnifier/Reader. https://www.zoomtext.com/products/zoomtext-magnifierreader/.Google Scholar
Recommendations
How People with Low Vision Access Computing Devices: Understanding Challenges and Opportunities
ASSETS '16: Proceedings of the 18th International ACM SIGACCESS Conference on Computers and AccessibilityLow vision is a pervasive condition in which people have difficulty seeing even with corrective lenses. People with low vision frequently use mainstream computing devices, however how they use their devices to access information and whether digital low ...
SaIL: saliency-driven injection of ARIA landmarks
IUI '20: Proceedings of the 25th International Conference on Intelligent User InterfacesNavigating webpages with screen readers is a challenge even with recent improvements in screen reader technologies and the increased adoption of web standards for accessibility, namely ARIA. ARIA landmarks, an important aspect of ARIA, lets screen ...
Design of an Augmented Reality Magnification Aid for Low Vision Users
ASSETS '18: Proceedings of the 20th International ACM SIGACCESS Conference on Computers and AccessibilityAugmented reality (AR) systems that enhance visual capabilities could make text and other fine details more accessible for low vision users, improving independence and quality of life. Prior work has begun to investigate the potential of assistive AR, ...
Comments