Skip to main content
SpringerLink
Log in
Book cover

Human-Harmonized Information Technology, Volume 2 pp 197–233Cite as

Reading-Life Log as a New Paradigm of Utilizing Character and Document Media

  • Chapter
  • First Online:

Abstract

“You are what you read.” As this sentence implies, reading is important for building our minds. We are investing a huge amount of time for reading to input information. However the activity of “reading” is done only by each individual in an analog way and nothing is digitally recorded and reused. In order to solve this problem, we record reading activities as digital data and analyze them for various goals. We call this research “reading-life log.” In this chapter, we describe our achievements of the reading-life log. A target of the reading-life log is to analyze reading activities quantitatively and qualitatively: when, how much, what you read, and how you read in terms of your interests and understanding. Body-worn sensors including intelligent eyewear are employed for this purpose. Another target is to analyze the contents of documents based on the users’ reading activities: for example, which are the parts most people feel difficult/interesting. Materials to be read are not limited to books and documents. Scene texts are also important materials which guide human activities.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Table 7.1 and Figs. 7.1, 7.3, 7.4, 7.5, 7.6, 7.7, 7.8, 7.9, 7.10, 7.11, 7.12, 7.13, 7.17, 7.18, 7.19, 7.20 are originally published in [1] and copyrighted by IEICE. They are granted to use in this article with the permission number 16KB0074. The research described in this chapter has been approved by the research ethics committee in Osaka Prefecture University.

References

  1. K. Kise, S. Omachi, S. Uchida, M. Iwamura, A trial for development of fundamental technologies for new usage of character and document media. J. Inst. Electron. Inf. Commun. Eng. 98(4), 311–327 (2015)

    Google Scholar 

  2. D. Karatzas, F. Shafait, S. Uchida, M. Iwamura, L. Gomez, S.R. Mestre, J. Mas, D.F. Mota, J.A. Almazan, L.P. de las Heras, ICDAR 2013 robust reading competition, in Proceedings of the 12th International Conference on Document Analysis and Recognition (ICDAR 2013) (2013), pp. 1484–1493

    Google Scholar 

  3. D. Karatzas, L. Gomez, A. Nicolaou, S. Ghosh, A. Bagdanov, M. Iwamura, J. Matas, L. Neumann, V.R. Chandrasekhar, S. Lu, F. Shafait, S. Uchida, E. Valveny, ICDAR 2015 robust reading competition, in Proceedings of the 13th International Conference on Document Analysis and Recognition (ICDAR 2015) (2015), pp. 1156–1160

    Google Scholar 

  4. S. Uchida, Text Localization and Recognition in Images and Video, Handbook of Document Image Processing and Recognition (Springer, 2014)

    Google Scholar 

  5. Q. Ye, D. Doermann, Text detection and recognition in imagery: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 37(7), 1480–1500 (2015)

    Article  Google Scholar 

  6. Y. Matsuda, S. Omachi, H. Aso, String detection from scene images by binarization and edge detection. IEICE Trans. Inf. Syst. (Japanese edition), J93-D (3), 336–344 (2010) (in Japanese)

    Google Scholar 

  7. R. Huang, P. Shivakumara, Y. Feng, S. Uchida, Scene character detection and recognition with cooperative multiple-hypothesis framework. IEICE Trans. Inf. Syst. E96-D (10), 2235–2244 (2013)

    Google Scholar 

  8. H. Takebe, S. Uchida, Scene character extraction by an optimal two-dimensional segmentation. IEICE Trans. Inf. Syst. (Japanese edition) (D), J97-D (3), 667–675 (2014) (in Japanese)

    Google Scholar 

  9. Y. Kunishige, Y. Feng, S. Uchida, Scenery character detection with environmental context, in Proceedings of the 11th International Conference on Document Analysis and Recognition (ICDAR 2011) (2011), pp. 1049–1053

    Google Scholar 

  10. A. Zhu, R. Gao, S. Uchida, Could scene context be beneficial for scene text detection?Pattern Recognit. 58C, 204–215 (2016)

    Google Scholar 

  11. A. Shahab, F. Shafait, A. Dengel, S. Uchida, How salient is scene text? in Proceedings of the 10th IAPR International Workshop on Document Analysis Systems (DAS2012) (2012), pp. 317–321

    Google Scholar 

  12. R. Gao, S. Uchida, A. Shahab, F. Shafait, V. Frinken, Visual Saliency Models for Text Detection in Real World. PLoS one 9(12), e114539 (2014)

    Article  Google Scholar 

  13. S. Wang, S. Uchida, M. Liwicki, Y. Feng, Part-based methods for handwritten digit recognition. Front. Comput. Sci. 7(4), 514–525 (2013)

    Article  MathSciNet  Google Scholar 

  14. S. Toba, H. Kudo, T. Miyazaki, Y. Sugaya, S. Omachi, Ultra-low resolution character recognition system with pruning mutual subspace method, in Proceedings of the 2015 International Conference on Consumer Electronics—Taiwan (ICCE-TW) (2015), pp. 284–285

    Google Scholar 

  15. M. Goto, R. Ishida, Y. Feng, S. Uchida, Analyzing the distribution of a large-scale character pattern set using relative neighborhood graph, in Proceedings of the 12th International Conference on Document Analysis and Recognition (ICDAR 2013) (2013), pp. 3–7

    Google Scholar 

  16. M. Goldstein, S. Uchida, A comparative study on outlier removal from a large-scale dataset using unsupervised anomaly detection, in Proceedings of the 5th International Conference on Pattern Recognition Applications and Methods (ICPRAM2016) (2016), pp. 263–269

    Google Scholar 

  17. T. Saito, H. Yamada, K. Yamamoto, On the data base ETL9 of handprinted characters in JIS Chinese characters and its analysis. IEICE Trans. Inf. Syst. (Japanese edition) (D), J68-D(4), 757–764 (1985) (in Japanese)

    Google Scholar 

  18. T. Nakai, K. Kise, M. Iwamura, Use of affine invariants in locally likely arrangement hashing for camera-based document image retrieval, in Document Analysis Systems VII, vol. 3872, Lecture Notes in Computer Science, (2006), pp. 541–552

    Google Scholar 

  19. S. Ahmed, K. Kise, M. Iwamura, M. Liwicki, A. Dengel, Automatic ground truth generation of camera captured documents using document image retrieval, in Proceedings of the 12th International Conference on Document Analysis and Recognition (ICDAR 2013) (2013), pp. 528–532

    Google Scholar 

  20. S.M. Lucas, A. Panaretos, L. Sosa, A. Tang, S. Wong, R. Young, K. Ashida, H. Nagai, M. Okamoto, H. Yamamoto, H. Miyao, J. Zhu, W. Ou, C. Wolf, J.-M. Jolion, L. Todoran, M. Worring, X. Lin, ICDAR 2003 robust reading competitions: entries, results and future directions. Int. J. Doc. Anal. Recognit. (IJDAR) 7(2–3), 105–122 (2005)

    Article  Google Scholar 

  21. S. Lucas, ICDAR 2005 text locating competition results, in Proceedings of the 8th International Conference on Document Analysis and Recognition (ICDAR2005), 1, (2005), pp. 80–84

    Google Scholar 

  22. A. Shahab, F. Shafait, A. Dengel, ICDAR 2011 robust reading competition challenge 2: reading text in scene images, in Proceedings of the 11th International Conference on Document Analysis and Recognition (ICDAR 2011) (2011), pp. 1491–1496

    Google Scholar 

  23. K. Wang, S. Belongie, Word spotting in the wild, in Proceedings of the 11th European Conference on Computer Vision (ECCV2010), Part I (2010), pp. 591–604

    Google Scholar 

  24. K. Wang, B. Babenko, S. Belongie, End-to-end scene text recognition, in Proceedings of the 13th International Conference on Computer Vision (ICCV2011) (2011), pp. 1457–1464

    Google Scholar 

  25. R. Gao, F. Shafait, S. Uchida, Y. Feng, A hierarchical visual saliency model for character detection in natural scenes. Camera-Based Document Analysis and Recognition, LNCS 8357, 18–29 (2014)

    Google Scholar 

  26. M. Iwamura, T. Matsuda, N. Morimoto, H. Sato, Y. Ikeda, K. Kise, Downtown osaka scene text dataset, in Proceedings of the 2nd International Workshop on Robust Reading (IWRR2016) (2016) (in printing)

    Google Scholar 

  27. Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, A.Y. Ng, Reading digits in natural images with unsupervised feature learning, in Proceedings of the NIPS Workshop on Deep Learning and Unsupervised Feature Learning (2011), p. 9

    Google Scholar 

  28. M. Iwamura, M. Tsukada, K. Kise, Automatic labeling for scene text database, in Proceedings of the 12th International Conference on Document Analysis and Recognition (ICDAR 2013) (2013), pp. 1397–1401

    Google Scholar 

  29. H. Saito, Y. Sugaya, S. Omachi, S. Uchida, M. Iwamura, K. Kise, Generation of character patterns from sample character images, in IEICE Technical Report, PRMU2010-287 (2011) (in Japanese)

    Google Scholar 

  30. T.S. Cho, S. Avidan, W.T. Freeman, The patch transform. IEEE Trans. Pattern Anal. Mach. Intell. 32(8), 1489–1501 (2010)

    Article  Google Scholar 

  31. S. Belongie, J. Malik, and J. Puzicha, Shape context: a new descriptor for shape matching and object recognition, in Advances in Neural Information Processing Systems (2000), pp. 831–837

    Google Scholar 

  32. M. Iwamura, T. Sato, K. Kise, What is the most efficient way to select nearest neighbor candidates for fast approximate nearest neighbor search? in Proceedings of the 14th International Conference on Computer Vision (ICCV 2013) (2013), pp. 3535–3542

    Google Scholar 

  33. M. Iwamura, T. Tsuji, K. Kise, Memory-based recognition of camera-captured characters, in Proceedings of the 9th IAPR International Workshop on Document Analysis Systems (DAS2010) (2010), pp. 89–96

    Google Scholar 

  34. Y. Lamdan, H.J. Wolfson, Geometric hashing: a general and efficient model-based recognition scheme, in Proceedings of the 2nd International Conference on Computer Vision (ICCV1988) (1988), pp. 238–249

    Google Scholar 

  35. N. Asada, M. Iwamura, K. Kise, Improvement of word recognition accuracy with spellchecker based on tendency of recognition error of characters, IEICE Technical Report, 110(467), PRMU2010-268, pp. 183–188 (2011) (in Japanese)

    Google Scholar 

  36. M. Iwamura, T. Kobayashi, K. Kise, Recognition of multiple characters in a scene image using arrangement of local features, in Proceedings of the 11th International Conference on Document Analysis and Recognition (ICDAR 2011) (2011), pp. 1409–1413

    Google Scholar 

  37. D.G. Lowe, Object recognition from local scale-invariant features, in Proceedings of the International Conference on Computer Vision (1999), pp. 1150–1157

    Google Scholar 

  38. T. Kobayashi, M. Iwamura, T. Matsuda, K. Kise, An anytime algorithm for camera-based character recognition, in Proceedings of the 12th International Conference on Document Analysis and Recognition (ICDAR 2013) (2013), pp. 1172–1176

    Google Scholar 

  39. T. Matsuda, M. Iwamura, K. Kise, Performance improvement in local feature based camera-captured character recognition, in Proceedings of the 11th IAPR International Workshop on Document Analysis Systems (DAS2014) (2014), pp. 196–201

    Google Scholar 

  40. X. Liu, J. Samarabandu, An edge-based text region extraction algorithm for indoor mobile robot navigation, in Proceedings of the 2005 IEEE International Conference on Mechatronics and Automation, 2, (2005), pp. 701–706

    Google Scholar 

  41. K. Kunze, H. Kawaichi, K. Kise, K. Yoshimura, The wordometer—estimating the number of words read using document image retrieval and mobile eye tracking, in Proceedings of the 12th International Conference on Document Analysis and Recognition (ICDAR 2013) (2013), pp. 25–29

    Google Scholar 

  42. S. Ishimaru, J. Weppner, K. Kunze, A. Bulling, K. Kise, A. Dengel, P. Lukowicz, In the blink of an eye—combining head motion and eye blink frequency for activity recognition with Google Glass, in Proceedings of the 5th Augmented Human International Conference (2014), pp. 150–153

    Google Scholar 

  43. Y. Shiga, T. Toyama, Y. Utsumi, A. Dengel, K. Kise, Daily activity recognition combining gaze motion and visual features, in PETMEI 2014: 4th International Workshop on Pervasive Eye Tracking and Mobile Eye-based Interaction, Proceedings of the 16th International Conference on Ubiquitous Computing (2014), pp. 1103–1111

    Google Scholar 

  44. K. Kunze, Y. Shiga, S. Ishimaru, K. Kise, Reading activity recognition using an off-the-shelf EEG—detecting reading activities and distinguishing genres of documents, in Proceedings of the12th International Conference on Document Analysis and Recognition (ICDAR2013) (2013), pp. 96–100

    Google Scholar 

  45. Y. Utsumi, Y. Shiga, M. Iwamura, K. Kunze, K. Kise, Document type classification toward understanding reading habits, in Proceedings of the 20th Korea-Japan Joint Workshop on Frontiers of Computer Vision, 3, (2014), pp. 11–17

    Google Scholar 

  46. K. Kunze, Y. Utsumi, Y. Shiga, K. Kise, A. Bulling, I know what you are reading: recognition of document types using mobile eye tracking, in Proceedings of the 17th Annual International Symposium on Wearable Computers (2013), pp. 113–116

    Google Scholar 

  47. O. Augereau, K. Kise, K. Hoshika, A proposal of a document image reading-life log based on document image retrieval and eyetracking, in Proceedings of the 13th International Conference on Document Analysis and Recognition (ICDAR2015) (2015), pp. 246–250

    Google Scholar 

  48. K. Kunze, H. Kawaichi, K. Yoshimura, K. Kise, Towards inferring language expertise using eye tracking, in CHI’13 Extended Abstracts on Human Factors in Computing Systems (2013), p. 6

    Google Scholar 

  49. K. Yoshimura, K. Kunze, K. Kise, The eye as the window of the language ability: estimation of english skills by analyzing eye movement while reading documents, in Proceedings of the 13th International Conference on Document Analysis and Recognition (ICDAR2015) (2015), pp. 251–255

    Google Scholar 

  50. H. Fujiyoshi, K. Yoshimura, K. Kunze, K. Kise, A method of estimating English skills using eye gaze information of answering questions of English exercises, IEICE Technical Report, 115(24), PRMU2015-10, pp. 49–54 (2015) (in Japanese)

    Google Scholar 

  51. K. Kunze, K. Masai, M. Inami, Ö. Sacakli, M. Liwicki, A. Dengel, S. Ishimaru, K. Kise, Quantifying reading habits: counting how many words you read, in Presented at the UbiComp’15: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (2015), pp. 87–96

    Google Scholar 

  52. S. Ishimaru, K. Kunze, K. Tanaka, Y. Uema, K. Kise, M. Inami, Smart eyewear for interaction and activity recognition, in Presented at the CHI EA’15: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (2015), pp. 307–310

    Google Scholar 

  53. T. Kimura, R. Huang, S. Uchida, M. Iwamura, S. Omachi, K. Kise, The reading-life log—technologies to recognize texts that we read, in Proceedings of the 12th International Conference on Document Analysis and Recognition (ICDAR 2013) (2013), pp. 91–95

    Google Scholar 

  54. A. Okoso, K. Kunze, K. Kise, Implicit gaze based annotations to support second language learning, in Proceedings of the 2014 ACM Conference on Pervasive and Ubiquitous Computing: Adjunct Publication (UbiComp2014) (2014), pp. 143–146

    Google Scholar 

  55. R. Biedert, G. Buscher, S. Schwarz, J. Hees, A. Dengel, Text 2.0, in Proceedings of the 28th ACM Conference on Human Factors in Computing Systems (CHI2011) (2011)

    Google Scholar 

  56. T. Toyama, W. Suzuki, A. Dengel, K. Kise, User attention oriented augmented reality on documents with document dependent dynamic overlay, in Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (ISMAR2013) (2013), pp. 299–300

    Google Scholar 

  57. K. Masai, Y. Sugiura, K. Suzuki, S. Shimamura, K. Kunze, M. Ogata, M. Inami, M. Sugimoto, AffectiveWear: towards recognizing affect in real life, in Presented at the UbiComp/ISWC’15 Adjunct: Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers (2015), pp. 357–360

    Google Scholar 

  58. S. Sanchez, T. Dingler, H. Gu, K. Kunze, Embodied reading: a multisensory experience, in Presented at the CHI EA’16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (2016), pp. 1459–1466

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Osaka Prefecture University, 1-1 Gakuencho, Naka, Sakai, Osaka, 599-8531, Japan

    Koichi Kise & Masakazu Iwamura

  2. Tohoku University, 6-6-05, Aramaki Aza Aoba Aoba-ku, Sendai, Miyagi, 980-8579, Japan

    Shinichiro Omachi

  3. Kyushu University, 744, Motooka, Nishi, Fukuoka, 819-0395, Japan

    Seiichi Uchida

  4. The University of Tokyo, 7-3-1 Hongo, Bunkyo, Tokyo, 113-8654, Japan

    Masahiko Inami

  5. Keio University, 4-1-1 Hiyoshi, Kohoku, Yokohama, Kanagawa, 223-8526, Japan

    Kai Kunze

Authors
  1. Koichi Kise
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shinichiro Omachi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Seiichi Uchida
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Masakazu Iwamura
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Masahiko Inami
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kai Kunze
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Koichi Kise .

Editor information

Editors and Affiliations

  1. Graduate School of Informatics, Kyoto University, Kyoto, Japan

    Toyoaki Nishida

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer Japan KK

About this chapter

Cite this chapter

Kise, K., Omachi, S., Uchida, S., Iwamura, M., Inami, M., Kunze, K. (2017). Reading-Life Log as a New Paradigm of Utilizing Character and Document Media. In: Nishida, T. (eds) Human-Harmonized Information Technology, Volume 2. Springer, Tokyo. https://doi.org/10.1007/978-4-431-56535-2_7

Download citation

  • DOI: https://doi.org/10.1007/978-4-431-56535-2_7

  • Published:

  • Publisher Name: Springer, Tokyo

  • Print ISBN: 978-4-431-56533-8

  • Online ISBN: 978-4-431-56535-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation