Analyzing the understandability of Requirements Engineering languages for CSCW systems: A family of experiments

https://doi.org/10.1016/j.infsof.2012.06.001Get rights and content

Abstract

Context

A collaborative system is a special kind of software whose users can perform collaboration, communication and collaboration tasks. These systems usually have a high number of non-functional requirements, resulting from the users’ need of being aware of other users with whom to collaborate, that is, the workspace awareness.

Objective

This paper aims at evaluating two Requirements Engineering languages i* and CSRML (an extension of i*) in order to determine which is the most suitable one to specify requirements of collaborative systems, taking into account their special characteristics regarding collaboration and awareness.

Method

We performed a family of experiments comprising an original experiment and two replicas. They were performed by 30, 45 and 9 Computer Science students, respectively, from Spain and Argentina. These subjects filled in two understandability questionnaires once they analyzed the requirements models of two systems: an e-learning collaborative system and a conference review system with some collaborative aspects support. Both models were specified by using the evaluated languages.

Results

The statistical analysis of the family of experiments showed that the understandability was higher for the models specified with CSRML than for those specified with i*, especially for collaborative aspects. This result was also confirmed by the meta-analysis conducted.

Conclusions

CSRML surpasses i* when modeling collaborative systems requirements models due to the specific expressiveness introduced to represent collaboration between users and awareness and the new resorts to manage actors and roles.

Highlights

Understandability of i* and CSRML for CSCW requirements specification was analyzed. ► A family of experiments at three different universities was conducted. ► Participants tried to understand the requirements models of two different systems. ► The obtained results were summarized by means of meta-analysis techniques. ► CSRML resulted more understandable than i* to specify requirements of CSCW systems.

Introduction

Internet is continuously evolving not only in the way it is developed, but also in the way it is browsed and exploited. Internet sites have evolved from simple static sites created by a single developer, which just enabled visitors to read their contents, to complex dynamic sites, which can be created and modified by users in a collaborative way. It is worth taking a look at the top 1000 most visited pages in 2011 [1] and see that, except for search engines and news websites, almost all of the top 100 ones are collaborative webs. In addition, the just mentioned search engines and news web sites are becoming also collaborative by including new social features, such as the possibility of adding comments or recommendations to news or search results.

Nowadays, even classic applications like text processors are collaborative. For instance, Google Docs [2] enables several users to edit a text document simultaneously. These collaborative text processors are a good example of CSCW (Computer Supported Cooperative Work) systems [3], which are systems whose users can perform collaboration, communication and coordination tasks (3C). Collaborative systems, in a similar way to classical single-user systems, have to be specified by means of a set of requirements, whose accuracy and suitability are key to achieve the quality of the developed system. The main difference between the requirements of single-user systems and CSCW systems is the highly non-functional nature of the latter, because of the users’ need of being aware of the presence of other users with whom to perform the above mentioned 3C tasks, that is, the Workspace Awareness (WA).

WA was defined by Gutwin as “the up-to-the-moment understanding of another person’s interaction within a shared workspace” [4]. This WA involves knowledge about where others are working, what they are doing now, when a user performed some action, and what they are going to do next. Table 1, Table 2 show the WA elements related to the present and to the past, respectively.

In order to deal with the specification of these special type of systems, we conducted an initial test, by using DESMET [5], to check which is the most adequate Requirements Engineering technique to model both awareness and quality requirements of CSCW systems [6]. The analyzed techniques were Use Cases [7], [8], Goal-Oriented [9] and Viewpoints [10], and we concluded that Goal-Oriented techniques were the most promising ones for specifying this kind of requirements. Next, we conducted a second study comparing the suitability of three Goal-Oriented approaches for modeling CSCW systems requirements [11]. The compared proposals were NFR Framework [12], i* Framework [13] and KAOS Methodology [9]. The analysis of the results showed that i* was the most suitable proposal.

Despite the previous studies that determined that i* was the most promising technique, we identified several issues when modeling collaborative systems with this language, such as the lack of expressiveness for awareness requirements, imprecise 3C task representation, and a not fully suitable management of roles and actors. Due to these problems, the original i* language was extended to try to solve the above mentioned deficiencies by creating CSRML (Collaborative Systems Requirements Modeling Language) [14].

As Basili et al. [15] claim, families of experiments can be used to draw relevant conclusions that an experiment alone cannot provide. For this reason, in this paper, a family of experiments is presented whose general goal is to test which language, CSRML or i*, has a better understandability when modeling requirements of CSCW systems. This family of experiments has been conducted by following the Wholin et al.’s guidelines [16] and it has consisted of an experiment and two replicas, all of them carried out by students of Computer Science from three different universities. These students were asked to fill in two understandability questionnaires once they had analyzed the requirements models of an e-learning collaborative activity and a conference review system with collaborative aspects support. Both requirements models were specified with the tested languages.

As aforementioned, understandability was the criteria analyzed in the family of experiments. Understandability has been defined by the International Organization for Standardization (ISO) as “the capability of the software product to enable the user to understand whether the software is suitable, and how it can be used for particular tasks and conditions of use” [17]. Indeed, considering the requirements specification as one of the products of the software process, it becomes critical to provide a specification understandable by every stakeholder to develop a suitable software. This fact is also supported by the IEEE830-1998 standard for requirements specifications [18], since it considers an understandable specification as one of the means to decrease the development effort by reducing later redesign, recoding, and retesting. Moreover, understandability has also been considered as a quality attribute in several quality standards. For instance, it is one of the five quality characteristics that an engineering model must have along with abstraction, accuracy, predictiveness and inexpensiveness [19]. Furthermore, ISO includes understandability as one of the six sub-characteristics of the usability quality factor. More precisely, usability is defined as “the capability of the software product to be understood, learned, used and attractive to the user, when used under specified conditions” [17].

Based on these considerations, this paper is structured as follows: Section 2 presents several works related to CSCW and Empirical Software Engineering, focusing on the evaluation of Requirements Engineering models. Next, in Section 3, i* and CSRML are briefly introduced. In Section 4 the family of experiments is presented by describing how it has been carried out and its main results. Finally, some conclusions and future work are drawn in Section 5.

Section snippets

Related works

In traditional software applications, only one user is able to interact with the application. Therefore, there is no support for collaboration among users. Nevertheless, in the late eighties, Computer Supported Cooperative Work (CSCW [3], [20]) emerged as a new discipline to study collaborative systems, a special kind of software in which users can collaborate with each other by sharing resources, chatting, etc.

One of the key elements for the construction of CSCW systems is the proper

Languages overview

In this section the evaluated languages, namely CSRML and i*, are presented in order to facilitate the comprehension of the family of experiments. In Section 3.1, the i* language is briefly described and later, in Section 3.2 CSRML is introduced highlighting which concepts have been introduced to facilitate the modeling of requirements of CSCW systems.

The family of experiments

In this section, we present the context (Section 4.1), design (Section 4.2), conduction procedure (Section 4.3) and the analyzed results (Section 4.4) of the family of experiments (Fig. 4) performed in order to assess the understandability of CSRML language with regard to i*. As the three members of the family have been designed in a similar way, only a single description will be provided for all of them, presenting the results for each one in Section 4.4.

Conclusions and further work

This family of experiments has its foundations in two previous works [6], [11] that helped us to identify which RE technique is more suitable to model CSCW systems requirements. Nevertheless, these previous experiments led us to find that this ilk of specifications suffer from an important lack of expressiveness for some CSCW characteristics, such as user collaboration or awareness representation. To address these shortcomings, CSRML was proposed [14], an extension of i* to model CSCW systems

Acknowledgements

This work has been partially supported by the grant (PEII09-0054-9581) from the Junta de Comunidades de Castilla-La Mancha and by the grant (DESACO, TIN2008-06596-C02-01) from the Spanish Government. We would like to thank Leandro Antonelli (Software Engineering teacher at UNLP), Francisco Parreño (Statistics teacher at UCLM) and Oscar Dieste (Empirical Software Engineering expert at UPM) for their advices during the course of this study. We would also like to thank Symbia IT Corp. software

References (66)

  • V.B. Kampenes et al.

    A systematic review of effect size in software engineering experiments

    Information and Software Technology

    (2007)
  • Google Inc., Google DoubleClick Ad Planner 1000 Most-Visited Sites on the Web, 2011....
  • Google Inc., Google Docs, 2001. <http://docs.google.com> (accessed May...
  • K. Schmidt et al.

    Taking CSCW seriously

    Computer Supported Cooperative Work

    (1992)
  • C. Gutwin et al.

    A descriptive framework of workspace awareness for real-time groupware

    Computer Supported Cooperative Work

    (2002)
  • B.A. Kitchenham

    A methodology for evaluating software engineering methods and tools

  • M.A. Teruel, E. Navarro, V. Lopez-Jaquero, F. Montero, P. Gonzalez, An empirical evaluation of requirement engineering...
  • A. Cockburn

    Writing Effective Use Cases

    (2000)
  • G. Booch et al.

    The Unified Modeling Language User Guide

    (2005)
  • A. van Lamsweerde, Goal-oriented requirements engineering: a guided tour, in: Fifth IEEE International Symposium on...
  • A. Finkelstein et al.

    Viewpoints: a framework for integrating multiple perspectives in system development

    International Journal of Software Engineering and Knowledge Engineering

    (1992)
  • M.A. Teruel, E. Navarro, V. López-Jaquero, F. Montero, P. González, A comparative of goal-oriented approaches to...
  • L.M. Cysneiros et al.

    Non-functional requirements elicitation

  • J. Castro, M. Kolp, J. Mylopoulos, A requirements-driven development methodology, in: 13th Int. Conf. On Advanced...
  • M.A. Teruel, E. Navarro, V. López-Jaquero, F. Montero, P. González, CSRML: A Goal-Oriented Approach to Model...
  • V.R. Basili et al.

    Building knowledge through families of experiments

    IEEE Transactions on Software Engineering

    (1999)
  • C. Wohlin et al.

    Experimentation in Software Engineering: An Introduction

    (2000)
  • ISO/IEC, ISO/IEC 9126: Software Engineering – Product Quality,...
  • IEEE Computer Society, IEEE Std 830-1998 – IEEE Recommended Practice for Software Requirements Specifications,...
  • B. Selic

    The pragmatics of model-driven development

    IEEE Software

    (2003)
  • C.A. Ellis et al.

    Groupware: some issues and experiences

    Communications of the ACM

    (1991)
  • J.L. Garrido, AMENITIES: una Metodologia para el Desarrollo de Sistemas Cooperativos Basada en Modelos de...
  • A.I. Molina et al.

    CIAM: a methodology for the development of groupware user interfaces

    Journal of Universal Computer Science

    (2008)
  • Cited by (0)

    View full text