Ruoyan Kong
Ruixuan Sun
Charles Chuankai Zhang
Joseph A. Konstan
ABSTRACT
A single digital newsletter usually contains many messages (regions). Users’ reading time spent on, and read level (skip/skim/read-in-detail) of each message is important for platforms to understand their users’ interests, personalize their contents, and make recommendations. Based on accurate but expensive-to-collect eyetracker-recorded data, we built models that predict per-region reading time based on easy-to-collect Javascript browser tracking data.
With eye-tracking, we collected 200k ground-truth datapoints on participants reading news on browsers. Then we trained machine learning and deep learning models to predict message-level reading time based on user interactions like mouse position, scrolling, and clicking. We reached 27% percentage error in reading time estimation with a two-tower neural network based on user interactions only, against the eye-tracking ground truth data, while the heuristic baselines have around 46% percentage error. We also discovered the benefits of replacing per-session models with per-timestamp models, and adding user pattern features. We concluded with suggestions on developing message-level reading estimation techniques based on available data.
- Ioannis Arapakis, Mounia Lalmas, and George Valkanas. 2014. Understanding Within-Content Engagement through Pattern Analysis of Mouse Gestures. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management (Shanghai, China) (CIKM ’14). Association for Computing Machinery, New York, NY, USA, 1439–1448. https://doi.org/10.1145/2661829.2661909Google Scholar
Digital Library
- Guy Aridor, Duarte Gonçalves, Daniel Kluver, Ruoyan Kong, and Joseph Konstan. 2022. The Economics of Recommender Systems: Evidence from a Field Experiment on MovieLens. arXiv preprint arXiv:2211.14219 (2022).Google Scholar
- Olivier Augereau, Kai Kunze, Hiroki Fujiyoshi, and Koichi Kise. 2016. Estimation of english skill with a mobile eye tracker. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct. 1777–1781.Google ScholarDigital Library
- Martin Bohme, André Meyer, Thomas Martinetz, and Erhardt Barth. 2006. Remote eye tracking: State of the art and directions for future development. In Proc. of the 2006 Conference on Communication by Gaze Interaction (COGAIN). 12–17.Google Scholar
- Georg Buscher, Ralf Biedert, Daniel Heinesch, and Andreas Dengel. 2010. Eye Tracking Analysis of Preferred Reading Regions on the Screen. In CHI ’10 Extended Abstracts on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI EA ’10). Association for Computing Machinery, New York, NY, USA, 3307–3312. https://doi.org/10.1145/1753846.1753976Google ScholarDigital Library
- G Cardillo. 2006. Holm-Sidak t-test: a routine for multiple t-test comparisons. Disponible en Matlab Central, en la página de la Red Mundial: http://www. mathworks. de/matlabcentral/fileexchange/12786 (Consultado el 22 de Noviembre 2009) (2006).Google Scholar
- Mon Chu Chen, John R Anderson, and Myeong Ho Sohn. 2001. What can a mouse cursor tell us more? Correlation of eye/mouse movements on web browsing. In CHI’01 extended abstracts on Human factors in computing systems. 281–282.Google Scholar
- Gregory Funke, Eric Greenlee, Martha Carter, Allen Dukes, Rebecca Brown, and Lauren Menke. 2016. Which eye tracker is right for your research? performance evaluation of several cost variant eye trackers. In Proceedings of the Human Factors and Ergonomics Society annual meeting, Vol. 60. SAGE Publications Sage CA: Los Angeles, CA, 1240–1244.Google ScholarCross Ref
- Agostino Gibaldi, Mauricio Vanegas, Peter J Bex, and Guido Maiello. 2017. Evaluation of the Tobii EyeX Eye tracking controller and Matlab toolkit for research. Behavior research methods 49, 3 (2017), 923–946.Google Scholar
- David Hauger, Alexandros Paramythis, and Stephan Weibelzahl. 2011. Using browser interaction data to determine page reading behavior. In International conference on user modeling, adaptation, and personalization. Springer, 147–158.Google ScholarCross Ref
- Yunzhong He, Yuxin Tian, Mengjiao Wang, Feier Chen, Licheng Yu, Maolong Tang, Congcong Chen, Ning Zhang, Bin Kuang, and Arul Prakash. 2023a. Que2Engage: Embedding-based Retrieval for Relevant and Engaging Products at Facebook Marketplace. Proceedings of the ACM Web Conference 2023 (2023).Google ScholarDigital Library
- Yunzhong He, Cong Zhang, Ruoyan Kong, Chaitanya Kulkarni, Qing Liu, Ashish Gandhe, Amit Nithianandan, and Arul Prakash. 2023b. HierCat: Hierarchical Query Categorization from Weakly Supervised Data at Facebook Marketplace. Proceedings of the ACM Web Conference 2023 (2023).Google ScholarDigital Library
- Jeff Huang, Ryen White, and Georg Buscher. 2012a. User see, user point: gaze and cursor alignment in web search. In Proceedings of the SIGCHI conference on human factors in computing systems. 1341–1350.Google ScholarDigital Library
- Jeff Huang, Ryen W White, Georg Buscher, and Kuansan Wang. 2012b. Improving searcher models using mouse cursor activity. In Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval. 195–204.Google ScholarDigital Library
- Ruoming Jin, Dong Li, Benjamin Mudrak, Jing Gao, and Zhi Liu. 2021. On estimating recommendation evaluation metrics under sampling. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 4147–4154.Google ScholarCross Ref
- Ruoyan Kong, Zhanlong Qiu, Yang Liu, and Qi Zhao. 2021a. NimbleLearn: A Scalable and Fast Batch-mode Active Learning Approach. In 2021 International Conference on Data Mining Workshops (ICDMW). IEEE, 350–359.Google Scholar
- Ruoyan Kong, Charles Chuankai Zhang, Ruixuan Sun, Vishnu Chhabra, Tanushsrisai Nadimpalli, and Joseph A Konstan. 2022. Multi-Objective Personalization in Multi-Stakeholder Organizational Bulk E-mail: A Field Experiment. Proceedings of the ACM on Human-Computer Interaction 6, CSCW2 (2022), 1–27.Google ScholarDigital Library
- Ruoyan Kong, Haiyi Zhu, and Joseph A Konstan. 2021b. Learning to Ignore: A Case Study of Organization-Wide Bulk Email Effectiveness. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1–23.Google ScholarDigital Library
- Dmitry Lagun and Eugene Agichtein. 2015. Inferring searcher attention by jointly modeling user interactions and content salience. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. 483–492.Google ScholarDigital Library
- Meng-Lung Lai, Meng-Jung Tsai, Fang-Ying Yang, Chung-Yuan Hsu, Tzu-Chien Liu, Silvia Wen-Yu Lee, Min-Hsien Lee, Guo-Li Chiou, Jyh-Chong Liang, and Chin-Chung Tsai. 2013. A review of using eye-tracking technology in exploring learning from 2000 to 2012. Educational research review 10 (2013), 90–115.Google Scholar
- Seulki Lee, Bashima Islam, Yubo Luo, and Shahriar Nirjon. 2019. Intermittent learning: On-device machine learning on intermittently powered system. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 4 (2019), 1–30.Google ScholarDigital Library
- Yixuan Li, Pingmei Xu, Dmitry Lagun, and Vidhya Navalpakkam. 2017. Towards measuring and inferring user interest from gaze. In Proceedings of the 26th International Conference on World Wide Web Companion. 525–533.Google ScholarDigital Library
- Yiqun Liu, Ye Chen, Jinhui Tang, Jiashen Sun, Min Zhang, Shaoping Ma, and Xuan Zhu. 2015. Different Users, Different Opinions: Predicting Search Satisfaction with Mouse Movement Information. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (Santiago, Chile) (SIGIR ’15). Association for Computing Machinery, New York, NY, USA, 493–502. https://doi.org/10.1145/2766462.2767721Google ScholarDigital Library
- Moses Namara and Curtis John Laurence. 2019. What Do You See? An Eyetracking study of a Tailored Facebook Interface for Improved Privacy Support. In ACM Symposium on Eye Tracking Research & Applications (ETRA)(2019), Vol. 7.Google Scholar
- Francesco Di Nocera, Orlando Ricciardi, and James F Juola. 2018. Rapid serial visual presentation: Degradation of inferential reading comprehension as a function of speed. International Journal of Human Factors and Ergonomics 5, 4 (2018), 293–303.Google ScholarCross Ref
- Youssef Othman, Mahmoud Khalaf, Ahmed Ragab, Ahmed Salaheldin, Reham Ayman, and Nada Sharaf. 2020. Eye-To-Eye: Towards Visualizing Eye Gaze Data. In 2020 24th International Conference Information Visualisation (IV). IEEE, 729–733.Google Scholar
- Keith Rayner and Monica S Castelhano. 2008. Eye movements during reading, scene perception, visual search, and while looking at print advertisements. (2008).Google Scholar
- Christin Seifert, Annett Mitschick, Jörg Schlötterer, and Raimund Dachselt. 2017. Focus paragraph detection for online zero-effort queries: Lessons learned from eye-tracking data. In Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval. 301–304.Google ScholarDigital Library
- ABM Fahim Shahriar, Mahedee Zaman Moon, Hasan Mahmud, and Kamrul Hasan. 2020. Online product recommendation system by using eye gaze data. In Proceedings of the International Conference on Computing Advancements. 1–7.Google ScholarDigital Library
- Hongke Zhao, Qi Liu, Yong Ge, Ruoyan Kong, and Enhong Chen. 2016b. Group preference aggregation: A nash equilibrium approach. In 2016 IEEE 16th International Conference on Data Mining (ICDM). IEEE, 679–688.Google ScholarCross Ref
- Qian Zhao, Shuo Chang, F Maxwell Harper, and Joseph A Konstan. 2016a. Gaze prediction for recommender systems. In Proceedings of the 10th ACM Conference on Recommender Systems. 131–138.Google ScholarDigital Library
Index Terms
- Getting the Most from Eye-Tracking: User-Interaction Based Reading Region Estimation Dataset and Models
Recommendations
Personalized context-aware collaborative filtering based on neural network and slope one
CDVE'09: Proceedings of the 6th international conference on Cooperative design, visualization, and engineeringCurrently, context has been identified as an important factor in recommender systems. Lots of researches have been done for context-aware collaborative filtering (CF) recommendation, but the contextual parameters in current approaches have same weights ...
How Much Novelty is Relevant?: It Depends on Your Curiosity
SIGIR '16: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information RetrievalTraditional recommendation systems (RS's) aim to recommend items that are relevant to the user's interest. Unfortunately, the recommended items will soon become too familiar to the user and hence fail to arouse her interest. Discovery-oriented ...
Improving Similarity Measurement of User's Rating Value using Sigmoid Function in Personalization System
IMCOM '16: Proceedings of the 10th International Conference on Ubiquitous Information Management and CommunicationPersonalization system is a method of providing appropriate products on the users' preferences and requirements. Collaborative Filtering is the most popular technique in an effective, Personalized Recommendation System to find a specific user with ...
Comments