Skip to main content
SpringerLink
Log in

When simulated environments make the difference: the effectiveness of different types of training of car service procedures

  • Original Article
  • Published:
Virtual Reality Aims and scope Submit manuscript

Abstract

An empirical analysis was performed to compare the effectiveness of different approaches to training a set of procedural skills to a sample of novice trainees. Sixty-five participants were randomly assigned to one of the following three training groups: (1) learning-by-doing in a 3D desktop virtual environment, (2) learning-by-observing a video (show-and-tell) explanation of the procedures, and (3) trial-and-error. In each group, participants were trained on two car service procedures. Participants were recalled to perform a procedure either 2 or 4 weeks after the training. The results showed that: (1) participants trained through the virtual approach of learning-by-doing performed both procedures significantly better (i.e. p < .05 in terms of errors and time) than people of non-virtual groups, (2) the virtual training group, after a period of non-use, were more effective than non-virtual training (i.e. p < .05) in their ability to recover their skills, (3) after a (simulated) long period from the training—i.e. up to 12 weeks—people who experienced 3D environments consistently performed better than people who received other kinds of training. The results also suggested that independently from the training group, trainees’ visuospatial abilities were a predictor of performance, at least for the complex service procedure, adj R 2 = .460, and that post-training performances of people trained through virtual learning-by-doing are not affected by learning styles. Finally, a strong relationship (p < .001, R 2 = .441) was identified between usability and trust in the use of the virtual training tool—i.e. the more the system was perceived as usable, the more it was perceived as trustable to acquire the competences.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Ahlberg G et al (2007) Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 193:797–804. doi:10.1016/j.amjsurg.2006.06.050

    Article  Google Scholar 

  • Ai-Lim Lee E, Wong KW, Fung CC (2010) How does desktop virtual reality enhance learning outcomes? A structural equation modeling approach. Comput Educ 55:1424–1442. doi:10.1016/j.compedu.2010.06.006

    Article  Google Scholar 

  • Akins R, Tolson H, Cole B (2005) Stability of response characteristics of a Delphi panel: application of bootstrap data expansion. BMC Med Res Methodol 5:37

    Article  Google Scholar 

  • Alippi C, de Russis C, Piuri V (2003) A neural-network based control solution to air-fuel ratio control for automotive fuel-injection systems. Syst Man Cybern Part C Appl Rev IEEE Trans 33:259–268. doi:10.1109/TSMCC.2003.814035

    Article  Google Scholar 

  • Anastassova M, Burkhardt J-M (2009) Automotive technicians’ training as a community-of-practice: implications for the design of an augmented reality teaching aid. Appl Ergon 40:713–721. doi:10.1016/j.apergo.2008.06.008

    Article  Google Scholar 

  • Anastassova M, Burkhardt J-M, Mégard C, Ehanno P (2005) Results from a user-centred critical incidents study for guiding future implementation of augmented reality in automotive maintenance. Int J Ind Ergon 35:67–77. doi:10.1016/j.ergon.2004.08.005

    Article  Google Scholar 

  • Anderson JR (1982) Acquisition of cognitive skill. Psychol Rev 89:369–406

    Article  Google Scholar 

  • Arthur JW, Day EA, Bennett JW (1997) Dyadic versus individual training protocols: loss and reacquisition of a complex skill. J Appl Psychol 82:783–791

    Article  Google Scholar 

  • Arthur W Jr, Bennett W Jr, Stanush PL, McNelly TL (1998) Factors that influence skill decay and retention: a quantitative review and analysis. Hum Perform 11:57–101. doi:10.1207/s15327043hup1101_3

    Article  Google Scholar 

  • Bandura A (1992) Observational learning. In: Squire L (ed) Encyclopedia of learning and memory. Macmillan, New York

    Google Scholar 

  • Belardinelli C, Blümel E, Müller G, Schenk M (2008) Making the virtual more real: research at the fraunhofer IFF virtual development and training centre. Cogn Process 9:217–224. doi:10.1007/s10339-008-0216-0

    Article  Google Scholar 

  • Borsci S, Federici S, Lauriola M (2009) On the dimensionality of the system usability scale (SUS): a test of alternative measurement models. Cogn Process 10:193–197. doi:10.1007/s10339-009-0268-9

    Article  Google Scholar 

  • Borsci S, Kurosu M, Federici S, Mele ML (2013) Computer systems experiences of users with and without disabilities: an evaluation guide for professionals. CRC Press, Boca Raton. doi:10.1201/b15619-1

    Google Scholar 

  • Borsci S, Lawson G, Broome S (2015) Empirical evidence, evaluation criteria and challenges for the effectiveness of virtual and mixed reality tools for training operators of car service maintenance. Comput Ind 67:17–26. doi:10.1016/j.compind.2014.12.002

    Article  Google Scholar 

  • Bowman DA, Gabbard JL, Hix D (2002) A survey of usability evaluation in virtual environments: classification and comparison of methods. Presence Teleoperators Virtual Environ 11:404–424. doi:10.1162/105474602760204309

    Article  Google Scholar 

  • Brooke J (1996) SUS: A “quick and dirty” usability scale. In: Jordan PW, Thomas B, Weerdmeester BA, McClelland IL (eds) Usability evaluation in industry. Taylor & Francis, London, pp 189–194

    Google Scholar 

  • Carlson P, Peters A, Gilbert SB, Vance JM, Luse A (2015) Virtual training: learning transfer of assembly tasks visualization and computer graphics. IEEE Trans 21:770–782. doi:10.1109/TVCG.2015.2393871

    Google Scholar 

  • Christine Roy M, Dewit O, Aubert BA (2001) The impact of interface usability on trust in web retailers. Internet Res 11:388–398. doi:10.1108/10662240110410165

    Article  Google Scholar 

  • Fitts PM, Posner MI (1967) Human performance. Brooks/Cole, Monterey

    Google Scholar 

  • Flavián C, Guinalíu M, Gurrea R (2006) The role played by perceived usability, satisfaction and consumer trust on website loyalty. Inf Manag 43:1–14. doi:10.1016/j.im.2005.01.002

    Article  Google Scholar 

  • Garg AX, Norman GR, Eva KW, Spero L, Sharan S (2002) Is there any real virtue of virtual reality?: the minor role of multiple orientations in learning anatomy from computers. Acad Med 77:S97–S99

    Article  Google Scholar 

  • Good P (2000) Permutation tests: a practical guide for resampling methods for testing hypotheses. Springer, New York

    Book  MATH  Google Scholar 

  • Hall CR, Stiles RJ, Horwitz CD (1998) Virtual reality for training: evaluating knowledge retention. In: Virtual reality annual international symposium, 1998. Proceedings, IEEE 1998, pp 18–18, pp 184–189. doi:10.1109/VRAIS.1998.658488

  • Hamilton EC et al (2002) Comparison of video trainer and virtual reality training systems on acquisition of laparoscopic skills. Surg Endosc 16:406–411. doi:10.1007/s00464-001-8149-z

    Article  Google Scholar 

  • Hart SG, Staveland LE (1988) Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. In: Peter AH, Najmedin M (eds) Advances in Psychology, vol 52. North-Holland Press, Amsterdam, pp 139–183. doi:10.1016/S0166-4115(08)62386-9

    Google Scholar 

  • Jayaram S, Jayaram U, Kim YJ, DeChenne C, Lyons KW, Palmer C, Mitsui T (2007) Industry case studies in the use of immersive virtual assembly. Virtual Real 11:217–228. doi:10.1007/s10055-007-0070-x

    Article  Google Scholar 

  • Kennedy RS, Lane NE, Berbaum KS, Lilienthal MG (1993) simulator sickness questionnaire: an enhanced method for quantifying simulator sickness. Int Journal Aviat Psychol 3:203–220. doi:10.1207/s15327108ijap0303_3

    Article  Google Scholar 

  • Kolb DA (1984) Experiential Learning as the Science of Learning and Development. Prentice Hall, Englewood Cliffs

    Google Scholar 

  • Kolb AY, Kolb DA (2005) The Kolb Learning Style Inventory—Version 3.1 2005 Technical Specifications. Hay Group Holdings Inc, Boston

    Google Scholar 

  • Lewis JR (2014) Usability: lessons learned … and yet to be learned. Int J Hum Comput Interact 30:663–684. doi:10.1080/10447318.2014.930311

    Article  Google Scholar 

  • Lewis JR, Sauro J (2009) The factor structure of the system usability scale. In: Kurosu M (ed) Human centered design, vol 5619. Lecture notes in computer science. Springer, Berlin Heidelberg, pp 94–103. doi:10.1007/978-3-642-02806-9_12

  • Li JR, Khoo LP, Tor SB (2003) Desktop virtual reality for maintenance training: an object oriented prototype system (V-REALISM). Comput Ind 52:109–125. doi:10.1016/S0166-3615(03)00103-9

    Article  Google Scholar 

  • Lippert SK, Michael Swiercz P (2005) Human resource information systems (HRIS) and technology trust. J Inform Sci 31:340–353. doi:10.1177/0165551505055399

    Article  Google Scholar 

  • Mcknight DH, Carter M, Thatcher JB, Clay PF (2011) Trust in a specific technology: an investigation of its components and measures ACM Trans Manage. Inf Syst 2:1–25. doi:10.1145/1985347.1985353

    Google Scholar 

  • Michalos G, Makris S, Papakostas N, Mourtzis D, Chryssolouris G (2010) Automotive assembly technologies review: challenges and outlook for a flexible and adaptive approach. CIRP J Manufact Sci Technol 2:81–91. doi:10.1016/j.cirpj.2009.12.001

    Article  Google Scholar 

  • Milgram P, Kishino F (1994) A taxonomy of mixed reality visual displays. IEICE Trans Inform Syst E77-D:1321–1329

    Google Scholar 

  • Nagendran M, Gurusamy KS, Aggarwal R, Loizidou M, Davidson BR (2013) Virtual reality training for surgical trainees in laparoscopic surgery. Cochrane Database Syst Rev. doi:10.1002/14651858.CD006575.pub3

    Google Scholar 

  • Ottosson S (2002) Virtual reality in the product development process. J Eng Des 13:159–172. doi:10.1080/09544820210129823

    Article  Google Scholar 

  • Parry G, Newnes L, Huang X (2011) Goods, Products and Services. In: Macintyre M, Parry G, Angelis J (eds) Service Design and Delivery. Service Science: Research and Innovations in the Service Economy. Springer, New York, pp 19–29. doi:10.1007/978-1-4419-8321-3_2

    Chapter  Google Scholar 

  • Parsons TD, Larson P, Kratz K, Thiebaux M, Bluestein B, Buckwalter JG, Rizzo AA (2004) Sex differences in mental rotation and spatial rotation in a virtual environment. Neuropsychologia 42:555–562. doi:10.1016/j.neuropsychologia.2003.08.014

    Article  Google Scholar 

  • Peters M, Laeng B, Latham K, Jackson M, Zaiyouna R, Richardson C (1995) A redrawn Vandenberg and Kuse mental rotations test—different versions and factors that affect performance. Brain Cogn 28:39–58. doi:10.1006/brcg.1995.1032

    Article  Google Scholar 

  • Sasse M (2005) Usability and Trust in Information Systems. In: Mansell R, Collins B (eds) Trust and Crime in Information Societies. Edward Elgar, Cheltenham, pp 319–348

    Google Scholar 

  • Seth A, Vance J, Oliver J (2011) Virtual reality for assembly methods prototyping: a review. Virtual Real 15:5–20. doi:10.1007/s10055-009-0153-y

    Article  Google Scholar 

  • Sharples S, Cobb S, Moody A, Wilson JR (2008) Virtual reality induced symptoms and effects (VRISE): comparison of head mounted display (HMD), desktop and projection display systems. Displays 29:58–69. doi:10.1016/j.displa.2007.09.005

    Article  Google Scholar 

  • Stefanidis D, Korndorffer JR Jr, Sierra R, Touchard C, Dunne JB, Scott DJ (2005) Skill retention following proficiency-based laparoscopic simulator training. Surgery 138:165–170. doi:10.1016/j.surg.2005.06.002

    Article  Google Scholar 

  • Stefanidis D et al (2007) Construct and face validity and task workload for laparoscopic camera navigation: virtual reality versus videotrainer systems at the SAGES learning center. Surg Endosc 21:1158–1164. doi:10.1007/s00464-006-9112-9

    Article  Google Scholar 

  • Stone RT, Watts KP, Zhong P, Wei C-S (2011) Physical and cognitive effects of virtual reality integrated training. Hum Fact J Hum Fact Ergon Soc 53:558–572. doi:10.1177/0018720811413389

    Article  Google Scholar 

  • Stork A, Sevilmis N, Weber D, Gorecky D, Stahl C, Loskyll M, Michel F (2012) Enabling virtual assembly training in and beyond the automotive industry. In: Virtual Systems and Multimedia (VSMM), 18th international conference on, 2–5 Sept 2012, pp 347–352. doi:10.1109/VSMM.2012.6365944

  • Tang A, Owen C, Biocca F, Mou W (2003) Comparative effectiveness of augmented reality in object assembly. In: Proceedings of the SIGCHI conference on human factors in computing systems, Ft. Lauderdale, Florida, USA. ACM, 642626, pp 73–80. doi:10.1145/642611.642626

  • Wayne DB, Siddall VJ, Butter J, Fudala MJ, Wade LD, Feinglass J, McGaghie WC (2006) A longitudinal study of internal medicine residents’ retention of advanced cardiac life support skills. Acad Med 81:S9–S12

    Article  Google Scholar 

  • Webel S, Bockholt U, Engelke T, Gavish N, Olbrich M, Preusche C (2013) An augmented reality training platform for assembly and maintenance skills. Robot Auton Syst 61:398–403. doi:10.1016/j.robot.2012.09.013

    Article  Google Scholar 

  • Xu W, Gao F (2011) Research of virtual reality technology in automotive engine assembly teaching. In: Information technology and artificial intelligence conference (ITAIC), 2011 6th IEEE joint international, 20–22 Aug 2011, pp 167–169. doi:10.1109/itaic.2011.6030177

  • Yuviler-Gavish N, Gutierrez T, Webel S, Rodriguez J, Tecchia F (2011a) Design guidelines for the development of virtual reality and augmented reality training systems for maintenance and assembly tasks BIO web of conferences 1, pp 29–21, pp 29–24 doi:10.1051/bioconf/20110100029

  • Yuviler-Gavish N, Yechiam E, Kallai A (2011b) Learning in multimodal training: visual guidance can be both appealing and disadvantageous in spatial tasks. Int J Hum Comput Stud 69:113–122. doi:10.1016/j.ijhcs.2010.11.005

    Article  Google Scholar 

  • Yuviler-Gavish N, Gutiérrez T, Webel S, Rodríguez J, Peveri M, Bockholt U, Tecchia F (2015) Evaluating virtual reality and augmented reality training for industrial maintenance and assembly tasks. Interact Learn Environ 23:778–798. doi:10.1080/10494820.2013.815221

    Article  Google Scholar 

Download references

Acknowledgments

This paper was completed as part of Live Augmented Reality Training Environments (LARTE)—101509 project. The authors would like to acknowledge the Technology Strategy Board for funding the work.

Author information

Authors and Affiliations

  1. Diagnostic Evidence Cooperative of London, Department of Surgery and Cancer, Imperial College London, St. Mary’s Hospital, South Wharf Road, London, W2 1NY, UK

    Simone Borsci

  2. Human Factors Research Group, Faculty of Engineering, The University of Nottingham, Nottingham, NG7 2RD, UK

    Glyn Lawson & Davide Salanitri

  3. Jaguar Land Rover, Abbey Road, Whitley, Coventry, CV3 4LF, UK

    Bhavna Jha

Authors
  1. Simone Borsci
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Glyn Lawson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Davide Salanitri
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bhavna Jha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Simone Borsci.

Appendices

Appendix 1: Demographic profile of participants

Participants were recruited in the area of Nottinghamshire in UK. The overall cohorts were composed for the 69.7 % of European citizen, and 47.7 % of the participants were English native speakers. Professionals, and students from high schools and universities of the city of Nottingham were involved in the study. Demographics data (see Appendix 2, Q from 4 to 9a) showed 35 % of participants had a level of education from high school to college, whereas 43 % have achieved or are studying for a bachelor degree, and 22 % of participants have achieved (or are studying) for a master or higher levels of education (Q5). 55.4 % of participants reported that their experience with LEGO® Technic was quite low—from never to sometimes (Q6). 41.7 % of the participants have previous experience of VE systems (Q7); 63.1 % of participants reported a low level of experience with video games (Q8). 78.5 % of the cohort had never used the Lego® Digital Designer (LDD) system (Q9), and 21.5 % have from rare to fairly often experience in use of this tool. Among the participants, only 13.8 % had experienced sickness in the past using immersive or desktop virtual systems (Q7a–Q9a).

Participants were randomly distributed in three training groups of respectively: 22 (male 13, age M 32.09, SD 9.02), 21 (male 12, age M 28.52, SD 7.04), and 22 (male 14, age M 28.72, SD 7.31) people.

Appendix 2: Scenarios of first and second (target) procedures

  • Scenario of first procedure: The 4×4 Crawler car of a client does not work at all. Your manager promised to the client that you can conduct this procedure immediately, and the client is waiting for you. After a diagnostic test, your colleagues suggest to you to change the entire engine. Therefore, you have to remove the old engine from the car and place the new one as you learned during the training. After the procedure, please use the controller to test the car functioning.

  • Scenario of target procedure: A client reports to your head that his new car, a 4×4 Crawler, has some problems when he turns to the left. After a diagnostic, test your colleagues suggested to you that there is a malfunctioning of the left front damper of the car. Therefore, you have to remove the broken damper and place the new one, as you learned during the training. After the procedure please use the controller to test the car functioning.

Appendix 3: Demographic survey

figure a

Appendix 4: Selection of instructions kit

As a support to operate on the real car, you can use only one of the two types of instruction kit trainer showed to you. Please select the type you prefer to help you during the procedures. Once you put your choice you will be allowed to use only the type of kit you have selected, therefore take the most useful for you:

  • Video manual

  • Paper Manual

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Borsci, S., Lawson, G., Salanitri, D. et al. When simulated environments make the difference: the effectiveness of different types of training of car service procedures. Virtual Reality 20, 83–99 (2016). https://doi.org/10.1007/s10055-016-0286-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10055-016-0286-8

Keywords

Search

Navigation