Skip to main content
SpringerLink
Log in

Comparing Goal-Oriented Analysis Techniques: A Controlled Experiment

  • Conference paper
  • First Online:
Model-Driven Engineering and Software Development (MODELSWARD 2021, MODELSWARD 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1708))

  • 176 Accesses

Abstract

Background: Goal models are usually used during the early phases of requirement elicitation, since they help to understand the motivations underlying the system to be developed. Goal-oriented analysis techniques help analysts reason and make decisions regarding the analyzed goal model. Aims: In this paper, we present an empirical evaluation of two goal-oriented analysis techniques: i) VeGAn, which we proposed in a previous work, and which follows a value-driven approach and a fuzzy logic approach and ii) GRL-Quant, which follows a value-neutral approach and a quantitative approach. Method: We conducted an experiment with a population of 64 Computer Science undergraduate students. The participants were asked to analyze a goal model with one of the techniques and to answer a questionnaire to assess their perceptions. The techniques were compared with respect to the accuracy of goal model element prioritization, the participants’ prioritization time, and their perceptions of the quality of the analysis results (perceived satisfaction), ease of use, usefulness, and intention to use. Results: The results of the experiment show that both techniques are very similar since no significant differences could be found in most of the variables analyzed. However, the participants perceived the results of VeGAn more satisfactorily than those of GRL-Quant, although the prioritization accuracy of GRL-Quant was better for one particular system. Conclusions: This paper provides new insights have emerged from this study, and also opportunities to improve both techniques. The experiment provides preliminary results on the usefulness of both goal-oriented analysis techniques, but further research is required in order to strengthen these results.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abrahão, S., Gravino, C., Insfran, E., Scanniello, G., Tortora, G.: Assessing the effectiveness of sequence diagrams in the comprehension of functional requirements: results from a family of five experiments. IEEE Trans. Software Eng. 39(3), 327–342 (2012). https://doi.org/10.1109/TSE.2012.27

    Article  Google Scholar 

  2. Abrahão, S., Insfran, E., Gonzalez-Ladron-de Guevara, F., Fernandez-Diego, M., Cano-Genoves, C., de Oliveira, R.P.: Assessing the effectiveness of goal-oriented modeling languages: a family of experiments. Inf. Softw. Technol. 116, 106171 (2019). https://doi.org/10.1016/j.infsof.2019.08.003

    Article  Google Scholar 

  3. Amyot, D., Ghanavati, S., Horkoff, J., Mussbacher, G., Peyton, L., Yu, E.: Evaluating goal models within the goal-oriented requirement language. Int. J. Intell. Syst. 25(8), 841–877 (2010). https://doi.org/10.1002/int.20433

    Article  Google Scholar 

  4. Van Solingen, R., Basili, V., Caldiera, G., Rombach, H.D.: Goal Question Metric (GQM) approach. Encycl. Softw. Eng. (2002). Wiley. ISBN: 9780471028956. https://doi.org/10.1002/0471028959.sof142

  5. Boehm, B.W.: Value-based software engineering: overview and agenda. Value-based Software Engineering, pp. 3–14 (2006)

    Google Scholar 

  6. Cano-Genoves, C., Abrahão, S., Insfran, E.: Experimental comparison of two goal-oriented analysis techniques. In: 10th International Conference on Model-Driven Engineering and Software Development, MODELSWARD 2022, pp. 242–251. SCITEPRESS (2022). https://doi.org/10.5220/0010847000003119

  7. Cano-Genoves, C., Insfran, E., Abrahão, S., Fernandez-Diego, M., González-L.G., F.: A value-based approach for reasoning with goal models. In: ISD2019 (2019)

    Google Scholar 

  8. Chen, C.T.: Extensions of the TOPSIS for group decision-making under fuzzy environment. Fuzzy Sets Syst. 114(1), 1–9 (2000). https://doi.org/10.1016/S0165-0114(97)00377-1

    Article  MATH  Google Scholar 

  9. Cooper, A.: The inmates are running the Asylum. In: Arend, U., Eberleh, E., Pitschke, K. (eds.) Software-Ergonomie 1999. Berichte des German Chapter of the ACM, vol. 53. Vieweg+Teubner Verlag, Wiesbaden (1999). https://doi.org/10.1007/978-3-322-99786-9_1

  10. Davis, F.D.: Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly, pp. 319–340 (1989). https://doi.org/10.2307/249008

  11. Dujmovic, J.J., Nagashima, H.: LSP method and its use for evaluation of java ides. Int. J. Approx. Reason. 41(1), 3–22 (2006). https://doi.org/10.1016/j.ijar.2005.06.006

  12. Ernst, N.A., Mylopoulos, J., Wang, Y.: Requirements evolution and what (research) to do about it. In: Lyytinen, K., Loucopoulos, P., Mylopoulos, J., Robinson, B. (eds.) Design Requirements Engineering: A Ten-Year Perspective. LNBIP, vol. 14, pp. 186–214. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-540-92966-6_11

    Chapter  Google Scholar 

  13. Frakes, W.B., Baeza-Yates, R.: Information retrieval: data structures and algorithms. Prentice-Hall, Inc. (1992). https://doi.org/10.5555/129687

  14. Horkoff, J., Yu, E.: Interactive analysis of agent-goal models in enterprise modeling. Int. J. Inf. Syst. Model. Design (IJISMD) 1(4), 1–23 (2010). https://doi.org/10.4018/jismd.2010100101

    Article  Google Scholar 

  15. Horkoff, J., Yu, E.: Analyzing goal models: different approaches and how to choose among them. In: Proceedings of the 2011 ACM Symposium on Applied Computing, pp. 675–682 (2011). https://doi.org/10.1145/1982185.1982334

  16. Horkoff, J., Yu, E.: Comparison and evaluation of goal-oriented satisfaction analysis techniques. Requirements Eng. 18(3), 199–222 (2013). https://doi.org/10.1007/s00766-011-0143-y

  17. Horkoff, J., Yu, E.: Interactive goal model analysis for early requirements engineering. Requirements Eng. 21(1), 29–61 (2016). https://doi.org/10.1007/s00766-014-0209-8

  18. Liaskos, S., Jalman, R., Aranda, J.: On eliciting contribution measures in goal models. In: 2012 20th IEEE International Requirements Engineering Conference (RE), pp. 221–230. IEEE (2012). https://doi.org/10.1109/RE.2012.6345808

  19. Liu, L., Yu, E.: Designing information systems in social context: a goal and scenario modelling approach. Inf. Syst. 29(2), 187–203 (2004). https://doi.org/10.1016/S0306-4379(03)00052-8

    Article  Google Scholar 

  20. Loewenthal, K., Lewis, C.A.: An introduction to psychological tests and scales. Psychology Press (2018)

    Google Scholar 

  21. Maxwell, K.D.: Applied statistics for software managers. Applied Statistics for Software Managers (2002)

    Google Scholar 

  22. Scanniello, G., Erra, U.: Distributed modeling of use case diagrams with a method based on think-pair-square: results from two controlled experiments. J. Vis. Lang. Comput. 25(4), 494–517 (2014)

    Article  Google Scholar 

  23. Souza, E., Moreira, A., Araújo, J., Abrahão, S., Insfran, E., Da Silveira, D.S.: Comparing business value modeling methods: a family of experiments. Inf. Softw. Technol. 104, 179–193 (2018)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the grant TIN2017-84550-R (Adapt@Cloud project) funded by MCIN/AEI/10.13039/501100011033 and the “Programa de Ayudas de Investigación y Desarrollo” (PAID-01-17) from the Universitat Politècnica de València.

Author information

Authors and Affiliations

  1. IUMTI, Universitat Politècnica de València, Valencia, Spain

    Carlos Cano-Genoves, Silvia Abrahão & Emilio Insfran

Authors
  1. Carlos Cano-Genoves
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Silvia Abrahão
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Emilio Insfran
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carlos Cano-Genoves .

Editor information

Editors and Affiliations

  1. University of Twente, Enschede, The Netherlands

    Luís Ferreira Pires

  2. ESEO, ERIS, Angers, France

    Slimane Hammoudi

  3. Model Driven Solutions, Herndon, VA, USA

    Edwin Seidewitz

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cano-Genoves, C., Abrahão, S., Insfran, E. (2023). Comparing Goal-Oriented Analysis Techniques: A Controlled Experiment. In: Pires, L.F., Hammoudi, S., Seidewitz, E. (eds) Model-Driven Engineering and Software Development. MODELSWARD MODELSWARD 2021 2022. Communications in Computer and Information Science, vol 1708. Springer, Cham. https://doi.org/10.1007/978-3-031-38821-7_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-38821-7_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-38820-0

  • Online ISBN: 978-3-031-38821-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation