Skip to main content
SpringerLink
Log in

Assessing the Performance of Automated Model Extraction Rules

  • Conference paper
  • First Online:
Advances in Information Systems Development

Part of the book series: Lecture Notes in Information Systems and Organisation ((LNISO,volume 26))

Abstract

Automated Model Extraction Rules take as input requirements (in natural language) to generate domain models. Despite the existing work on these rules, there is a lack of evaluations in industrial settings. To address this gap, we conduct an evaluation in an industrial context, reporting the extraction rules that are triggered to create a model from requirements and their frequency. We also assess the performance in terms of recall, precision and F-measure of the generated model compared to the models created by domain experts of our industrial partner. Results enable us to identify new research directions to push forward automated model extraction rules: the inclusion of new knowledge sources as input for the extraction rules, and the development of specific experiments to evaluate the understanding of the generated models.

A prior version of this paper has been published in the ISD2017 Proceedings (http://aisel.aisnet.org/isd2014/proceedings2017).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Pohl, K., Rupp, C.: Requirements Engineering Fundamentals, 1st ed. Rocky Nook (2011)

    Google Scholar 

  2. Yue, T., Briand, L.C., Labiche, Y.: A systematic review of transformation approaches between user requirements and analysis models. Requir. Eng. 16(2), 75–99 (2011)

    Article  Google Scholar 

  3. Deeptimahanti, D.K., Sanyal, R.: Semi-automatic generation of UML models from natural language requirements. In: Proceedings of the 4th India Software Engineering Conference, pp. 165–174. ACM, New York, NY, USA (2011)

    Google Scholar 

  4. Elbendak, M., Vickers, P., Rossiter, B.N.: Parsed use case descriptions as a basis for object-oriented class model generation. J. Syst. Softw. 84, 1209–1223 (2011)

    Article  Google Scholar 

  5. Ibrahim, M., Ahmad, R.: Class diagram extraction from textual requirements using natural language processing (NLP) techniques. In: 2010 Second International Conference on Computer Research and Development, pp. 200–204, (2010)

    Google Scholar 

  6. Popescu, D., Rugaber, S., Medvidovic, N., Berry, D.M.: Reducing ambiguities in requirements specifications via automatically created object-oriented models. In: Monterey Workshop, pp. 103–124 (2007)

    Google Scholar 

  7. Sagar, V.B.R.V., Abirami, S.: Conceptual modeling of natural language functional requirements. J. Syst. Softw. 88, 25–41 (2014)

    Article  Google Scholar 

  8. Yue, T., Briand, L.C., Labiche, Y.: aToucan: an automated framework to derive uml analysis models from use case models. ACM Trans. Softw. Eng. Methodol. 24(3), 13:1–13:52 (2015)

    Google Scholar 

  9. Arora, C., Sabetzadeh, M., Briand, L., Zimmer, F.: Automated checking of conformance to requirements templates using natural language processing. IEEE Trans. Softw. Eng. 41(10), 944–968 (2015)

    Article  Google Scholar 

  10. Arora, C., Sabetzadeh, M., Briand, L., Zimmer, F.: Extracting domain models from natural-language requirements: approach and industrial evaluation. In: Proceedings of the ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems, pp. 250–260. ACM, New York, NY, USA (2016)

    Google Scholar 

  11. Akbik, A., Broß, J.: Wanderlust: Extracting semantic relations from natural language text using dependency grammar patterns. In: WWW Workshop (2009)

    Google Scholar 

  12. Rosadini, B., Ferrari, A., Gori, G., Fantechi, A., Gnesi, S., Trotta, I., Bacherini, S.: Using NLP to detect requirements defects: an industrial experience in the railway domain. In: Requirements Engineering: Foundation for Software Quality—23rd International Working Conference, REFSQ 2017, Essen, Germany, Feb 27–Mar 2, 2017, Proceedings, pp. 344–360 (2017)

    Google Scholar 

  13. Fanmuy, G., Fraga, A., Lloréns, J.: Requirements verification in the industry. In: Proceedings of the Second International Conference on Complex Systems Design and Management, CSDM 2011, Paris, 7–9 Dec 2011, pp. 145–160 (2011)

    Google Scholar 

  14. Lucas, F.J., Molina, F., Toval, A.: A systematic review of UML model consistency management. Inf. Softw. Technol. 51(12), 1631–1645 (2009)

    Article  Google Scholar 

  15. Zave, P.: Classification of research efforts in requirements engineering. ACM Comput. Surv. 29(4), 315–321 (1997)

    Article  Google Scholar 

  16. Capobianco, G., Lucia, A. De, Oliveto, R., Panichella, A., Panichella, S.: On the role of the nouns in IR-based Traceability Recovery. In: Proceedings of the International Conference on Program Comprehension, pp. 148–157. IEEE (2009)

    Google Scholar 

  17. Runeson, P., Höst, M.: Guidelines for conducting and reporting case study research in software engineering. Empir. Softw. Eng. 14(2), 131–164 (2009)

    Article  Google Scholar 

  18. Robeer, M., Lucassen, G., v. d. Werf, J.M.E.M., Dalpiaz, F., Brinkkemper, S.: Automated extraction of conceptual models from user stories via NLP. In: 2016 IEEE 24th International Requirements Engineering Conference (RE), pp. 196–205, (2016)

    Google Scholar 

  19. Ferrari, A., Dell’Orletta, F., Spagnolo, G.O., Gnesi, S.: Measuring and improving the completeness of natural language requirements. LNCS (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics). 8396 LNCS, pp. 23–38 (2014)

    Google Scholar 

  20. Ambriola, V., Gervasi, V.: On the systematic analysis of natural language requirements with CIRCE. Autom. Softw. Eng. 13(1), 107–167 (2006)

    Article  Google Scholar 

  21. Ben Abdessalem Karaa, W., Ben Azzouz, Z., Singh, A., Dey, N., S. Ashour, A., Ben Ghazala, H.: Automatic Builder of Class Diagram ABCD: An Application of UML Generation from Functional Requirements. Softw. Pr. Exper. 46(11), 1443–1458 (2016)

    Google Scholar 

  22. Thakur, J.S., Gupta, A.: Identifying domain elements from textual specifications. In: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, pp. 566–577. ACM, New York, NY, USA (2016)

    Google Scholar 

Download references

Acknowledgements

This work has been partially supported by the Ministry of Economy and Competitiveness (MINECO) through the Spanish National R+D+i Plan and ERDF funds under the project Model-Driven Variability Extraction for Software Product Line Adoption (TIN2015-64397-R).

Author information

Authors and Affiliations

  1. Universidad San Jorge, Zaragoza, Spain

    Jorge Echeverría, Francisca Pérez & Carlos Cetina

  2. Universitat Politècnica de València, Valencia, Spain

    Óscar Pastor

Authors
  1. Jorge Echeverría
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Francisca Pérez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Óscar Pastor
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Carlos Cetina
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jorge Echeverría .

Editor information

Editors and Affiliations

  1. School of Sciences, University of Central Lancashire, Larnaca, Cyprus

    Nearchos Paspallis

  2. School of Sciences, University of Central Lancashire, Larnaca, Cyprus

    Marios Raspopoulos

  3. Cairnes School of Business and Economics, National University of Ireland Galway, Galway, Ireland

    Chris Barry

  4. Cairnes School of Business and Economics, National University of Ireland Galway, Galway, Ireland

    Michael Lang

  5. Faculty of Information Technology, Monash University, Melbourne, Australia

    Henry Linger

  6. Department of Information Systems, City University of Hong Kong, Kowloon, Hong Kong

    Christoph Schneider

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Echeverría, J., Pérez, F., Pastor, Ó., Cetina, C. (2018). Assessing the Performance of Automated Model Extraction Rules. In: Paspallis, N., Raspopoulos, M., Barry, C., Lang, M., Linger, H., Schneider, C. (eds) Advances in Information Systems Development. Lecture Notes in Information Systems and Organisation, vol 26. Springer, Cham. https://doi.org/10.1007/978-3-319-74817-7_3

Download citation

Publish with us

Policies and ethics

Search

Navigation