WESONet: Applying semantic web technologies and collaborative tagging to multimedia web information systems

https://doi.org/10.1016/j.chb.2009.10.004Get rights and content

Abstract

The publication of different media types, like images, audio and video in the World Wide Web is getting more importance each day. However, searching and locating content in multimedia sites is challenging. In this paper, we propose a platform for the development of multimedia web information systems. Our approach is based on the combination between semantic web technologies and collaborative tagging. Producers can add meta-data to multimedia content associating it with different domain-specific ontologies. At the same time, users can tag the content in a collaborative way. The proposed system uses a search engine that combines both kinds of meta-data to locate the desired content. It will also provide browsing capabilities through the ontology concepts and the developed tags.

Introduction

The increase of information of different media types available in the world wide web is overwhelming. The production of digital media content is no longer a difficult issue and it is expected that more and more people will publish multimedia content on the web. Although it is easy to generate and publish this kind of data, it is very difficult to search and index it.

Automatically extracted descriptions like the dominant color, texture, etc. tend to be low-level and there is a need for semantic descriptions that are closer to the knowledge domain of the users. This semantic gap has already been acknowledged (Mich, Brunelli, & Modena, 1999). We consider that current, state-of-the-art multimedia analysis technologies are not yet able to produce the higher-level descriptions that are needed to facilitate multimedia retrieval at the user level. To solve that problem, it is necessary to manually add the descriptions. The people who could add these descriptions can be either the creators or the users of the system.

In production time, the creator usually knows some information about the video. For example if a person is recording a conference talk he may know who the speaker, the title and the chairman are. That data can be stored in a database which contains the information in a non-uniform way. One problem of using non-standard databases is that once the video is published on the web, the meta-data is lost. A better solution would be to embed that meta-data in the same video that is being produced. In this way, the meta-data is maintained with the video and it does not depend on a particular system or platform. MPEG-7 was proposed as a vocabulary that could embed multimedia descriptions (Salembier & Smith, 2002). In this paper we will use that standard as a means to publish videos with their corresponding meta-data.

Images and videos can have different meanings depending on the context and the users who view them (Aurnhammer, Hanappe, & Steels, 2006). For that reason, it is not feasible to let the creators alone to describe the multimedia asset. It was long assumed that manual annotations would not be provided by end-users because it is a tedious and time consuming process. However, recent developments like Flickr1, or YouTube2 have shown that the users can add descriptions to resources in the form of a freely chosen set of keywords (“tags”) or comments.

In this paper, we present an approach to develop multimedia web information systems which combine automatically obtained low-level descriptions, with higher-level descriptions obtained from the creators which link to ontology concepts and tags added collaboratively by the end-users.

The structure of the paper is as follows. We begin with a review of related work in Section 2. In Section 3 we describe the main semantic web technologies and their relationship with social networks. In Section 4 we propose an approach to add meta-data to multimedia systems. Section 5 describes a combination between ontologies and collaborative tagging. We also describe a hybrid search engine over these systems. Finally, Section 6 describes the architecture of the system that we are developing, which implements the proposed approach.

Section snippets

Related work

There have been numerous attempts to add semantic descriptions to multimedia files (Stamou and Kollias, 2005, Moenne-Loccoz et al., 2005). One of the first attempts to use MPEG-7 to store meta-data was made by Hunter (2001). It initially developed an RDFS ontology enriched with DAML-OIL constructs, which was subsequently translated to OWL. In the aceMedia project (Bloehdorn et al., 2005), a Visual Descriptor Ontology was developed for describing low-level audio–visual features which were

Semantic web technologies and social networks

Social networks are representations of the relationships between groups and individuals in a community. Analysis of social ties and social networks is an established field and has found numerous applications in several fields, like sociology, psychology and computer science (Wasserman & Faust, 1994).

Internet has enabled the creation of on-line virtual communities. The technological infrastructure of those communities facilitates the development of systems which can analyze the structure of the

Adding semantics to multimedia data

The semantic web approach breaks the concept of a web page as a unit of information enabling the creation of resource descriptions with finer granularity. For example, instead of the homepage of a person, it would be possible to refer to the phone number of that person.

In the case of textual information, that break-up is more affordable, although not easy, given that it is possible to access to paragraphs, words, etc. facilitating syntactic searches from keywords. In this way, traditional

Combining ontologies and collaborative tagging

The development of collaborative tagging sites has attracted a huge number of users. This kind of social software allows the users to tag a given resource with some text. Some examples of sites with collaborative tagging are Flickr, del.icio.us, Technorati, CiteULike, Buzznet, etc. They allow to share photos, URLs, blogs, article references and music titles. A tag is a free text string chosen by the user. Usually, these tools offer some recommendations about similar tags given to the same

WESONet project

The main goal of our project was the development of a video collection for our University. Most of the videos are about conferences and courses, although there are several videos about other subject matters. The number of videos available has increased in the last years given that there is a videoconference service provided by the University which publishes almost around 3 or 4 conferences each day. In the current system, it was quite difficult to find information given that there was not much

Conclusions

The creation of image and video libraries is a difficult challenge. It is necessary to develop hybrid approaches which combine collective tagging with ontology based tagging for a better integration and user experience.

Although our system is being developed as a prototype in a specific domain – a collection of images and videos from our University – we consider that it could be applied to generic domains, like YouTube or similar web sites. We also believe that the use and production of digital

References (22)

  • M. Aurnhammer et al.

    Augmenting navigation for collaborative tagging with emergent semantics

  • D. Berrueta et al.

    Searching over public administration legal documents using ontologies

  • S. Bloehdorn et al.

    Semantic annotation of images and videos for multimedia analysis

  • C. Cattuto

    Semiotic dynamics in online social communities

    The European Physical Journal C

    (2006)
  • Celma, O. (2006). Foafing the music: Bridging the semantic gap in music recommendation. In 5th international semantic...
  • F. Crestani

    Application of spreading activation techniques in information retrieval

    Artificial Intelligence Review

    (1997)
  • García, R., Celma, O. (2006). Semantic integration and retrieval of multimedia metadata. In 5th knowledge markup and...
  • Golbeck, J., Hendler, J. (2006). Filmtrust: Movie recommendations using trust in web-based social networks. In IEEE...
  • Halaschek-Wiener, C., Schain, A., Golbeck, J., Grove, M., Parsia, B., Hendler, J. (2005). A flexible approach for...
  • L. Hollink et al.

    Building a visual ontology for video retrieval

  • Hunter, J. (2001). Adding multimedia to the semantic web – building an mpeg-7 ontology. In International semantic web...
  • Cited by (31)

    • Survey of quality models of e-learning systems

      2018, Physica A: Statistical Mechanics and its Applications
      Citation Excerpt :

      In paper [37] the significant criteria in evaluating E-learning were found. In paper [38] was proposed a platform for the development of multimedia web information systems and the platform is based on the combination between semantic web technologies and collaborative tagging which is suitable for e-learning platforms as well. Findings in article [39] suggested that 3D virtual technologies may influence e-learning from different levels.

    • Implicit learner assessment based on semantic relevance of tags

      2016, Computers in Human Behavior
      Citation Excerpt :

      There is a new trend to use Wiki for assessing learners such as Kao & Chen (2013) which use collaborative learning and wiki to assess learners. Also in Gayo et al. (2010) an assessment system was developed that could work in complex new learning new learning environments especially self-regulated learning systems. Based on our findings, no work has been performed on using tags for learner assessment.

    • Dynamics of human categorization in a collaborative tagging system: How social processes of semantic stabilization shape individual sensemaking

      2015, Computers in Human Behavior
      Citation Excerpt :

      Research on improving information access with tags has been seeking different routes. For example some researchers have suggested enhancing semantic search technologies with social tags (e.g. Gayo, de Pablos & Lovelle, 2010; Specia & Motta, 2007). Another direction has been to employ recommender services that are based on tags, or that suggest suitable tags (Bollen & Halpin, 2009; Jäschke, Marinho, Hotho, Schmidt-Thieme, & Stumme, 2007).

    View all citing articles on Scopus
    View full text