Abstract
Existing automatic image annotation (AIA) systems that depend solely on low-level image features often produce poor results, particularly when annotating real-life collections. Tag co-occurrence has been shown to improve image annotation by identifying additional keywords associated with user-provided keywords. However, existing approaches have treated tag co-occurrence as a static measure over time, thereby ignoring the temporal trends of many tags. The temporal distribution of tags, however, caused by events, seasons and memes, etc, provides a strong source of evidence beyond keywords for AIA. In this paper we propose a temporal tag co-occurrence approach to improve AIA accuracy. By segmenting collection tags into multiple co-occurrence matrices, each covering an interval of time, we are able to give precedence to tags which not only co-occur each other, but also have temporal significance. We evaluate our approach on a real-life timestamped image collection from Flickr by performing experiments over a number of temporal interval sizes. Results show statistically significant improvements to annotation accuracy compared to a non-temporal co-occurrence baseline.
This research was supported by the the European Community’s FP7 Programme under grant agreements nr 288024 (LiMoSINe).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Allan, J.: Topic detection and tracking, pp. 1–16. Kluwer Academic Publishers, Norwell (2002)
Athanasakos, K., Stathopoulos, V., Jose, J.M.: A framework for evaluating automatic image annotation algorithms. In: ECIR 2010 Milton Keynes, UK (2010)
Byde, A., Cayzer, S.: Personalized tag recommendations via tagging and content-based similarity metrics, New York (2) (2007)
Carneiro, G., Chan, A.B., Moreno, P.J., Vasconcelos, N.: Supervised learning of semantic classes for image annotation and retrieval. IEEE Transactions 29 (2007)
Cattuto, C., Benz, D., Hotho, A., Stumme, G.: Semantic Grounding of Tag Relatedness in Social Bookmarking Systems. In: Sheth, A.P., Staab, S., Dean, M., Paolucci, M., Maynard, D., Finin, T., Thirunarayan, K. (eds.) ISWC 2008. LNCS, vol. 5318, pp. 615–631. Springer, Heidelberg (2008)
Duygulu, P., Barnard, K., de Freitas, J.F.G., Forsyth, D.: Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part IV. LNCS, vol. 2353, pp. 97–112. Springer, Heidelberg (2002)
Fellbaum, C. (ed.): WordNet: An Electronic Lexical Database (Language, Speech, and Communication), illustrated edn. The MIT Press (1998)
Jaccard, P.: Étude comparative de la distribution florale dans une portion des alpes et des jura. Bulletin del la Société Vaudoise des Sciences Naturelles (1901)
Jeon, J., Lavrenko, V., Manmatha, R.: Automatic image annotation and retrieval using cross-media relevance models. In: SIGIR 2003, NY, USA, pp. 119–126 (2003)
Jin, S., Lin, H., Su, S.: Query expansion based on folksonomy tag co-occurrence analysis, pp. 300–305 (August 2009)
Joachims, T.: Learning to Classify Text Using Support Vector Machines: Methods, Theory and Algorithms. Kluwer Academic Publishers, Norwell (2002)
Kleinberg, J.: Bursty and hierarchical structure in streams. In: KDD 2002, pp. 91–101. ACM, New York (2002)
Lavrenko, V., Choquette, M., Croft, W.B.: Cross-lingual relevance models. In: SIGIR 2002, pp. 175–182. ACM, New York (2002)
Lavrenko, V., Manmatha, R., Jeon, J.: A model for learning the semantics of pictures. In: NIPS. MIT Press (2003)
Leskovec, J., Backstrom, L., Kleinberg, J.: Meme-tracking and the dynamics of the news cycle. In: KDD 2009, pp. 497–506. ACM, New York (2009)
Li, W., Sun, M.: Automatic Image Annotation Based on WordNet and Hierarchical Ensembles. In: Gelbukh, A. (ed.) CICLing 2006. LNCS, vol. 3878, pp. 417–428. Springer, Heidelberg (2006)
Llorente, A., Manmatha, R., Rüger, S.: Image retrieval using markov random fields and global image features. In: CIVR 2010, pp. 243–250. ACM, NY (2010)
Makadia, A., Pavlovic, V., Kumar, S.: Baselines for image annotation. Int. J. Comput. Vision 90(1), 88–105 (2010)
Manjunath, B.S.: Introduction to MPEG-7, Multimedia Content Description Interface. John Wiley and Sons, Ltd. (2002)
Monaghan, F., O’Sullivan, D.: Leveraging Ontologies, Context and Social Networks to Automate Photo Annotation. In: Falcidieno, B., Spagnuolo, M., Avrithis, Y., Kompatsiaris, I., Buitelaar, P. (eds.) SAMT 2007. LNCS, vol. 4816, pp. 252–255. Springer, Heidelberg (2007)
Santini, S., Gupta, A., Jain, R.: Emergent semantics through interaction in image databases 13(3), 337–351 (2001)
Sigurbjörnsson, B., van Zwol, R.: Flickr tag recommendation based on collective knowledge. In: WWW 2008, pp. 327–336. ACM, New York (2008)
Smeulders, A., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years 22(12), 1349–1380 (2000)
Vapnik, V.N.: The nature of statistical learning theory. Springer-Verlag New York, Inc., New York (1995)
Weinberger, K.Q., Slaney, M., Van Zwol, R.: Resolving tag ambiguity. In: MM 2008, pp. 111–120. ACM, New York (2008)
Whiting, S., Moshfeghi, Y., Jose, J.M.: Exploring term temporality for pseudo-relevance feedback. In: SIGIR 2011, pp. 1245–1246. ACM, New York (2011)
Yao, J., Cui, B., Huang, Y., Zhou, Y.: 2010 IEEE 26th International Conference on Data Engineering (ICDE), pp. 780–783 (March 2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
McParlane, P., Whiting, S., Jose, J. (2013). Improving Automatic Image Tagging Using Temporal Tag Co-occurrence. In: Li, S., et al. Advances in Multimedia Modeling. Lecture Notes in Computer Science, vol 7733. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35728-2_24
Download citation
DOI: https://doi.org/10.1007/978-3-642-35728-2_24
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35727-5
Online ISBN: 978-3-642-35728-2
eBook Packages: Computer ScienceComputer Science (R0)