Analyzing the understandability of Requirements Engineering languages for CSCW systems: A family of experiments
Highlights
► Understandability of i* and CSRML for CSCW requirements specification was analyzed. ► A family of experiments at three different universities was conducted. ► Participants tried to understand the requirements models of two different systems. ► The obtained results were summarized by means of meta-analysis techniques. ► CSRML resulted more understandable than i* to specify requirements of CSCW systems.
Introduction
Internet is continuously evolving not only in the way it is developed, but also in the way it is browsed and exploited. Internet sites have evolved from simple static sites created by a single developer, which just enabled visitors to read their contents, to complex dynamic sites, which can be created and modified by users in a collaborative way. It is worth taking a look at the top 1000 most visited pages in 2011 [1] and see that, except for search engines and news websites, almost all of the top 100 ones are collaborative webs. In addition, the just mentioned search engines and news web sites are becoming also collaborative by including new social features, such as the possibility of adding comments or recommendations to news or search results.
Nowadays, even classic applications like text processors are collaborative. For instance, Google Docs [2] enables several users to edit a text document simultaneously. These collaborative text processors are a good example of CSCW (Computer Supported Cooperative Work) systems [3], which are systems whose users can perform collaboration, communication and coordination tasks (3C). Collaborative systems, in a similar way to classical single-user systems, have to be specified by means of a set of requirements, whose accuracy and suitability are key to achieve the quality of the developed system. The main difference between the requirements of single-user systems and CSCW systems is the highly non-functional nature of the latter, because of the users’ need of being aware of the presence of other users with whom to perform the above mentioned 3C tasks, that is, the Workspace Awareness (WA).
WA was defined by Gutwin as “the up-to-the-moment understanding of another person’s interaction within a shared workspace” [4]. This WA involves knowledge about where others are working, what they are doing now, when a user performed some action, and what they are going to do next. Table 1, Table 2 show the WA elements related to the present and to the past, respectively.
In order to deal with the specification of these special type of systems, we conducted an initial test, by using DESMET [5], to check which is the most adequate Requirements Engineering technique to model both awareness and quality requirements of CSCW systems [6]. The analyzed techniques were Use Cases [7], [8], Goal-Oriented [9] and Viewpoints [10], and we concluded that Goal-Oriented techniques were the most promising ones for specifying this kind of requirements. Next, we conducted a second study comparing the suitability of three Goal-Oriented approaches for modeling CSCW systems requirements [11]. The compared proposals were NFR Framework [12], i* Framework [13] and KAOS Methodology [9]. The analysis of the results showed that i* was the most suitable proposal.
Despite the previous studies that determined that i* was the most promising technique, we identified several issues when modeling collaborative systems with this language, such as the lack of expressiveness for awareness requirements, imprecise 3C task representation, and a not fully suitable management of roles and actors. Due to these problems, the original i* language was extended to try to solve the above mentioned deficiencies by creating CSRML (Collaborative Systems Requirements Modeling Language) [14].
As Basili et al. [15] claim, families of experiments can be used to draw relevant conclusions that an experiment alone cannot provide. For this reason, in this paper, a family of experiments is presented whose general goal is to test which language, CSRML or i*, has a better understandability when modeling requirements of CSCW systems. This family of experiments has been conducted by following the Wholin et al.’s guidelines [16] and it has consisted of an experiment and two replicas, all of them carried out by students of Computer Science from three different universities. These students were asked to fill in two understandability questionnaires once they had analyzed the requirements models of an e-learning collaborative activity and a conference review system with collaborative aspects support. Both requirements models were specified with the tested languages.
As aforementioned, understandability was the criteria analyzed in the family of experiments. Understandability has been defined by the International Organization for Standardization (ISO) as “the capability of the software product to enable the user to understand whether the software is suitable, and how it can be used for particular tasks and conditions of use” [17]. Indeed, considering the requirements specification as one of the products of the software process, it becomes critical to provide a specification understandable by every stakeholder to develop a suitable software. This fact is also supported by the IEEE830-1998 standard for requirements specifications [18], since it considers an understandable specification as one of the means to decrease the development effort by reducing later redesign, recoding, and retesting. Moreover, understandability has also been considered as a quality attribute in several quality standards. For instance, it is one of the five quality characteristics that an engineering model must have along with abstraction, accuracy, predictiveness and inexpensiveness [19]. Furthermore, ISO includes understandability as one of the six sub-characteristics of the usability quality factor. More precisely, usability is defined as “the capability of the software product to be understood, learned, used and attractive to the user, when used under specified conditions” [17].
Based on these considerations, this paper is structured as follows: Section 2 presents several works related to CSCW and Empirical Software Engineering, focusing on the evaluation of Requirements Engineering models. Next, in Section 3, i* and CSRML are briefly introduced. In Section 4 the family of experiments is presented by describing how it has been carried out and its main results. Finally, some conclusions and future work are drawn in Section 5.
Section snippets
Related works
In traditional software applications, only one user is able to interact with the application. Therefore, there is no support for collaboration among users. Nevertheless, in the late eighties, Computer Supported Cooperative Work (CSCW [3], [20]) emerged as a new discipline to study collaborative systems, a special kind of software in which users can collaborate with each other by sharing resources, chatting, etc.
One of the key elements for the construction of CSCW systems is the proper
Languages overview
In this section the evaluated languages, namely CSRML and i*, are presented in order to facilitate the comprehension of the family of experiments. In Section 3.1, the i* language is briefly described and later, in Section 3.2 CSRML is introduced highlighting which concepts have been introduced to facilitate the modeling of requirements of CSCW systems.
The family of experiments
In this section, we present the context (Section 4.1), design (Section 4.2), conduction procedure (Section 4.3) and the analyzed results (Section 4.4) of the family of experiments (Fig. 4) performed in order to assess the understandability of CSRML language with regard to i*. As the three members of the family have been designed in a similar way, only a single description will be provided for all of them, presenting the results for each one in Section 4.4.
Conclusions and further work
This family of experiments has its foundations in two previous works [6], [11] that helped us to identify which RE technique is more suitable to model CSCW systems requirements. Nevertheless, these previous experiments led us to find that this ilk of specifications suffer from an important lack of expressiveness for some CSCW characteristics, such as user collaboration or awareness representation. To address these shortcomings, CSRML was proposed [14], an extension of i* to model CSCW systems
Acknowledgements
This work has been partially supported by the grant (PEII09-0054-9581) from the Junta de Comunidades de Castilla-La Mancha and by the grant (DESACO, TIN2008-06596-C02-01) from the Spanish Government. We would like to thank Leandro Antonelli (Software Engineering teacher at UNLP), Francisco Parreño (Statistics teacher at UCLM) and Oscar Dieste (Empirical Software Engineering expert at UPM) for their advices during the course of this study. We would also like to thank Symbia IT Corp. software
References (66)
- et al.
User interface analysis for groupware applications in the TOUCHE process model
Advances in Engineering Software
(2009) - et al.
The impact of structural complexity on the understandability of UML statechart diagrams
Information Sciences
(2010) - et al.
Assessing the influence of stereotypes on the comprehension of UML sequence diagrams: a family of experiments
Information and Software Technology
(2011) - et al.
Impact of test-driven development on productivity, code and tests: a controlled experiment
Information and Software Technology
(2011) - et al.
A controlled experiment in assessing and estimating software maintenance tasks
Information and Software Technology
(2011) - et al.
An evaluation of methods for prioritizing software requirements
Information and Software Technology
(1998) - et al.
A controlled experiment to evaluate how styles affect the understandability of requirements specifications
Information and Software Technology
(2003) - et al.
Using acceptance tests as a support for clarifying requirements: a series of experiments
Information and Software Technology
(2009) - et al.
An empirical study of groupware support for distributed software architecture evaluation process
Journal of Systems and Software
(2006) - et al.
Evaluating requirements modeling methods based on user perceptions: a family of experiments
Information Sciences
(2011)