Improving clinical reasoning by transferring the control of the path and the branches of the virtual patient to trainees using a non-linear, non-branched model

Non-linear, non-branched (NLNB) virtual patient (VP) models have no deﬁned paths or branches and emulate the "real world" where health professionals evaluate and manage their patients contemporaneously. This model has not been evaluated in prior studies but has potential to increase trainee interactivity and clinical reasoning. We evaluated NLNB VPs engagement by third-year medical students. After reading an initial clinical vignette students used a single textbox with an autocomplete feature to ﬁnd keywords and select associated actions needed to provide care for their virtual patients. As students found and selected actions their engagement varied signiﬁcantly based on the content the author provided. This commitment of time indicated the students’ interest in the author-provided content. Students used intrinsic motivation to gathered data and provide interventions. They used free text to add items to a problem list and the diﬀerential diagnosis. They were cognitively immersed. Sixty-four students engaged a mean of 2.69 NLNB VPs where actions were not categorized or listed. Students found actions to provide care for the virtual patient when well-deﬁned paths or branches were absent. NLNB VPs transfer control of the path and decisions to the trainees thereby increasing interactivity and potentially promoting critical thinking.


Introduction
Clinical reasoning is needed by physicians for every encounter with every patient. Clinical reasoning is exercised as data is gathered, evaluated, and assimilated to drive the next decision made by the clinician. This process is nonlinear. During urgent or emergent encounters the decision processes are in constant flux as the process of gathering, evaluating and assimilating data must take place contemporaneously with interventions by the physician. Even in the most benign physician-patient encounters data is gathered and interventions provided non-sequentially. A single finding may cause a paradigm shift.
Educational technology has been proposed as a method to teach clinical reasoning but defining the best methods for implementation has been elusive. It is difficult to develop education technology to mimic clinical reasoning in reallife environments. (Cook & Triola 2009) and (Bateman et al 2013) described the complexity of creating a virtual patient (VP) interface that is highly interactive and truly engages the learner. In spite of these challenges (VPs) show promise as effective modalities for practicing clinical reasoning skills. (Klinert et al 2015). VP construction to demonstrate what works has been studied using linear and branched models. (Bateman et al 2013) In linear models the path is well defined and usually follows the progression of a non-urgent patient encounter. In this model, trainees are directed by traditional components of the encounter such as history, physical examination, etc. This linear model may encourage a linear thought process. (Whalen 2014) To improve clinical reasoning branching models have a path that is less well-defined (non-traditional components) but branches defined as multiple choices. VP design should not stop with linear and branched designs but be further developed to promote clinical reasoning. (Cook & Triola 2009) Do trainees need defined paths or branches when they are not presents during real life physician-patient encounters? Another model, non-linear, non-branched, (NLNB) developed for this study has not been studied. It does not have defined paths or branches. Selections are not restricted by components or prior choices. NLNB VP navigation more closely approximates real life patient encounters where data is collected non-sequentially.
On initial investigation a NLNB VPs would seem to offer little educational benefit because trainees receive few directives and a skilled teacher does not predefined the structure. Fortunately for VP designers, medical trainees are accomplished adults who are search-savvy. Constructing questions and making choices are critical to the development of clinical reasoning skills. Self-directed learning is intensified when authors provide narratives to reinforce the trainee's decision or gentle redirect the trainer to gradually reveal the structure necessary to provide care for the VP. This unfolding of the case is a meaningful consideration for VPs. (Cook & Triola 2009) Virtual patient properties such as "real life", pathway flux, feedback, decision flux, consequence effect and others are relevant design components for VP contruction. (Bateman et al 2013) A NLNB VP incorporates these and provides the additional opportunity to improve interactivity as well as promote clinical reasoning as a primary goal. (Cook & Triola 2009) (Krupat 2011) (Berman et al 2016) suggested, "VPs can be designed to capitalize on intrinsic motivation, mastery goal orientation and achievement emotions to improve learning." When navigation is not possible without keyword searches, NLNB VPs capitalize on the trainee's intrinsic motivation. Because keyword searches are a critical part of a physicians' work this feature of VPs should not be undervalued. Keyword searches emulate the process physicians use to gather data, find literature to support or refute clinical decisions and write orders in the electronic health record.
This present study evaluated how trainees used keywords to find actions to benefit their patient after reviewing a brief clinical vignette. To further evaluate our trainee's interactions with the cases we programmatically monitored the time trainees spent with author-provided content and their navigation through the case. Trainees also created diagnoses for a differential diagnosis list using free text. The process of gathering, evaluating and assimilating data to

Software:
Because software to provide NLNB navigation while monitoring path and pace was not available, an Internet application was developed and designed by the principal investigator. (Clinical Gestalt 2016) Although standards exist for sharing medical education technology, (Medbiquitous 2010) an Internet application provides VPs that can be accessed from anywhere at anytime. Internet applications can be as robust as applications within native operating systems and provide a perpetual resource that uncouples instruction from physical space constraints, temporal constraints or instructor availability. (AAMC 2007) A VP case is the basic teaching module within the application. For the purposes of this study three pediatric hospitalists initially created six cases, writing them directly into the author interface. Later in the year one hospitalist authored an additional case, which was also included in the study. These cases were based on real-life cases encountered by the authors. They were not scripted or standardized.
To our knowledge, this is the first study evaluating student engagement with a NLNB, keyword-search-directed, internet-based VP application.

Recruitment:
During the academic year 2014-2015 sixty-four third year medical student volunteers were recruited for our study during their rotation on the general pediatric inpatient service. Each student completed a mean of 2.69 cases. This provided one hundred seventy-two (172) student-case interactions for analysis. Students were considered "engaged" when the student navigated to any action and selected it, creating a record in the database. Student participation did not influence their clerkship evaluation and no scores were recorded. Registration was anonymous. Faculty were blinded to all individual student data collected by the application. Students were told that work with the VP should not replace time at the bedside. Their work with the VP had no temporal or spatial protection. The application programmatically tracked the student's navigation and pace without intrusion into their cognitive processes.

Design:
Cases started with a clinical vignette. Students then searched for the most appropriate action to take next by finding keywords the author had associated with each action. On the second keystroke in the keyword search textbox a dropdown list of actions with matching associated keywords was made visible (autocomplete). As an example typing "pul" as in pulmonary will reveal a dropdown list with actions such as "Does JP have a cough", "Auscultate the chest" and "Order pulmonary function tests." Students could select either or retype the keyword and initiate a new search. When students selected an action from the list navigation to the action was initiated and results were made visible. To provide feedback for the students, authors assigned values of "optimal", "appropriate" or "suboptimal" to each result. An author narrative sometimes supplemented these. As students selected actions an accessible review list maintained the students' selections in chronological order making it easier to reflect on prior decisions and assimilate new data. Students repeated the search process for additional actions. Students also created a problem list and a differential diagnosis by typing into a textbox and pressing the enter key. Within the differential diagnosis and problem list items could be sorted or deleted. When students completed their work as indicated by clicking "done" they were provided with the author's differential diagnosis along with their own differential diagnosis and a critique of author's differential diagnosis. Selected actions and non-selected actions were displayed in two adjacent columns. A summary of key learning points and an additional author narrative completed the review of the case. Following the completion of their cases faculty met with small groups of students to discuss their decisions. This ensured congruity between the author's expert opinion and the students' path. Did they "get it" and did the authors adequately "explain it."

Disclosures:
This study was not funded and there are no author financial disclosures. After the conclusion of this study the PI received grants from East Carolina University, Brody School of Medicine and Vidant Medical Foundation to further develop the application.
This study was approved by the IRB and the pediatric clerkship directors. It conforms to the ethical requirements for research at The Brody School of Medicine at East Carolina University.

Student selections:
Authors assigned values of optimal, appropriate or suboptimal to each action. Students selected optimal actions from available optimal actions at a higher rate than suboptimal actions from available suboptimal actions (optimal selected/optimal available > suboptimal selected/suboptimal) indicating the students desire to find and take the most appropriate action for their VP. Table 1. Students selected 33-65% of the available actions within each case perhaps indicating a more thoughtful process than "click everything". Figure 3. The correlation coefficient between the number of available actions and the number of selected actions was 0.88. Figure 3. If more actions were available the students selected more actions.

Keywords:
Further analysis of author-provided keywords found a moderate positive correlation, 0.3-0.61 figure 1, between the number of keywords and the number of times students selected actions for all actions. Actions with more keywords were selected more often. Additional analysis of the correlation between the number of keywords for optimal actions and the number of times students selected these actions had the same weak to moderate correlation.

Student time variation:
Analysis of time spent with student-selected actions demonstrated the variability of student pace with authorprovided content. Students spent a mean of twenty-one seconds with all actions. This increased to thirty-six seconds when the author added a narrative. Students spent thirty-one seconds when the author added a table, usually laboratory values, forty-three seconds for a list, and forty-five seconds when images were included, usually radiographs. Figure 2. Two students exceeded 20 minutes in two separate actions. This probably represented a disruption in their interaction. These values were not calculated.

Differential diagnosis and problem lists:
Sixty-four students generated 150 differential diagnosis lists with a mean of 5. 13 (1-16)  Additionally forty-one students generated 76 problem lists with a mean of 6.04 (1-27) problems per list. Engagement with the differential diagnosis and the problem list required a change from point and click to typing free text. Our students were engaged.

Real world emulation:
NLNB VPs provide the appropriate platform for development of clinical reasoning. This study was innovative in the evaluation of how trainees engage a keyword-search-directed, NLNB case. This approach emulates the "real world" where health professionals have to evaluate and manage their patients contemporaneously. Students were required to search for actions, select an action, evaluate the result, assimilate the new data and use author-assigned values and narrative to navigate the cases. The "thrill of the hunt" in the absence of visible actions, a path or defined branches permitted students use their search skills and exercise critical thinking skills to select appropriate actions to manage their VPs.

Cognitive immersion:
Two examples support the notion that students were cognitively immersed in the case and chose what they believe to be the next best action for their VP. In one case most students chose vital signs or cardiovascular exam immediately in a four month-old child who had received little medical care and is now limp and dusky. This patient ultimately is found to have heart failure as a result of a large undiagnosed ventriculo-septal defect. This contrasts with another case where the most frequent initial actions for a stable two year old with vomiting and diarrhea were historical questions. They chose wisely.

Actions value assignment:
In this study actions were assigned values of "optimal", "appropriate" or "suboptimal". These value assignments have limitations. Our cases contained more optimal actions per case. In an informal survey ninety percent of our responding requested additional suboptimal actions. Adverse consequences validate the need to make good choices. The students' explanations often included the phrase "I learn more when I am wrong." It is unlikely that realistic cases will be created with actions that have equal assignment to each value category. Students selected optimal actions at a higher rate than expected based on availability of optimal actions, Table 1. In our design the optimal value assignment did not equate to critical. Students could stop when they believed their selections provided appropriate care for their patient. As an example obtaining a CBC may be considered optimal for the patient with fever but not critical. In our post-case sessions we learned that students often paid more attention to content than to the assigned value. After this study we changed the values to reflect a broader range from contraindicated and dangerous for the patient to critical for safety and management of patient etc. Additional studies, particularly with higher-level learners, could consider how values could provide better feedback when cases are used for formative or summative evaluation or if values are needed at all.
There was a strong correlation between the number of available actions and the probability an action would be selected. In spite of the strong correlation, the maximum mean rate of student selections for any individual case was sixty-five percent, Table 3, making it unlikely that students were selecting as many actions as possible just to find "the answer". Student responsiveness to content was purposeful despite the lack of grade assignment, temporal or spatial protection. They were driven by their own intrinsic motivation. Unfortunately, student fears that every activity is graded and concerns about post-case meetings may have provided additional non-positive motivation. While more actions from which to choose would seem desirable there is a risk that the trainee will resort to a trial and error method rather than critical thinking to find actions. (Young et al 2014) The number of available actions and the proper mix of optimal and suboptimal choices is an area for future study.

Keywords:
There was a positive correlation between the number of keywords for an action and student selection of those actions. Figure 1. This suggests that actions were selected more often simply because there were a greater number of keywords but keywords helped our students find actions not select them. Selection of the action required an additional step. Further study is needed to determine how the numbers, formality or completeness of keywords help trainees find actions. More keywords may reduce cognitive load because the student will be more likely to find the keyword residing in the working memory. The ideal cognitive load is a subject for further study.

Author content and narrative:
When tables, author narrative, lists or images were incorporated into actions students spent more time with these actions figure 2. The author's narrative within the results of actions provided immediate feedback while the summary at the end of the case provided delayed feedback. This combination may be superior to either alone for long-term retention.

Pros and cons:
NLNB VPs allow the trainee to forge their own path through each clinical case and display their clinical prowess by making appropriate decisions for each patient. While clear direction is not provided at the beginning of a NLNB VP case, the content and narratives provided by authors enhances clinical reasoning as the case twists, turns and ultimately unfolds. Teachers who aspire to write VP cases should be encouraged because the students paused presumably to evaluate and assimilate the content and narrative provided by the author. We would be remiss if we did not warn authors that NLNB cases present some challenges. Authors have no knowledge of which actions students have not selected; therefore, authors should not refer to other actions, results or narratives. Each action is isolated from the author's perspective but from the trainee's perspective. All actions are interpreted in the light of the data gathered by the student as they build a list of selected actions and results. This represents a different paradigm for many authors but offers greater opportunities for the trainee. Another limitation is the lack of order within an encounter. For many encounters the order of actions is variable but there are encounters where order is critical. Our design did not incorporate order into the case. This is another area needing research.

Clinical reasoning
Critical thinking entails the process of synthesizing data to come to a conclusion. (Krupat et 2011) The development of clinical reasoning is further enhanced when trainees build a problem list and a differential diagnosis.. The analysis of the data from these lists validated the student's motivation to critically analyze the data collected from their searches and synthesize them into lists created and maintained solely at the discretion of the student. When students completed the encounter they had an opportunity for self-reflection as they compared their differential diagnosis with the author's differential and read the author critique of the differential.

Tracking trainee activity:
While traditional predefined performance values may not apply to an educationally innovative VP, ( 2009) programmatically collecting data about the trainee path and pace through a VP helps evaluate the trainees intrinsic motivation, their adaptability to keyword-search navigation and their ability to deal with the cognitive load of the technology and the case content. When search boxes are placed in a "sea of gray" without enhancing images or animations the extraneous cognitive load is reduced.

Conclusion
As adult learners our trainees want to exercise their clinical reasoning skills and a NLNB VP promotes this opportunity to gather, evaluate and assimilate date and to drive their decisions. Third-year medical students do not need actions categorized or listed, nor do they need well-defined paths or branches to provide care their VP. This control of the trainee's education can be safely transferred to them.
There are not an ideal number of keywords or an ideal distribution of actions valued as optimal, appropriate or suboptimal. In fact, the number of keywords and the number of actions has the potential to skew the trainees' selections and requires additional study. Because each NLNB VP case is developed to teach principles that reside in the mind of the author no arbitrary number of actions or keywords should be assigned.
Educators can use simple keyword-search-directed, Internet-based, NLNB VPs to help trainees make decisions that emulate the complexity of unstructured clinical encounters. This is possible while reducing the expense and time to design, develop and deploy VPs. NLNB VPs can be developed at relatively low cost to institutions because the search is text driven. VPs provide opportunities for trainees to make decisions but NLNB VPs have increased interactivity and clinical reasoning. Trainees do more when you give them less.

Take Home Messages
Take home points In every encounter with patients physicians use clinical reasoning Non-linear, non-branched virtual patients provide a more realistic patient encounter and enhance clinical reasoning Using keywords to search for the next best thing to do for their patient is what physicians do Trainees use their selected to formulate a plan for future actions and develop a differential diagnosis Trainees do not need defined paths or branches. Navigation through a virtual patient case can be safely transferred to them.

Declarations
The author has declared the conflicts of interest below.
The principal investigator received two grants to continue the development of Clinical Gestalt after this study was completed. The grant was for expenses only. The PI has not received any personal benefit.