Criterion scores, construct validity and reliability of a web-based instrument to assess physiotherapists' clinical reasoning focused on behaviour change: ‘Reasoning 4 Change’

Background and aim: ‘Reasoning 4 Change’ (R4C) is a newly developed instrument, including four domains (D1–D4), to assess clinical practitioners' and students' clinical reasoning with a focus on clients' behaviour change in a physiotherapy context. To establish its use in education and research, its psychometric properties needed to be evaluated. The aim of the study was to generate criterion scores and evaluate the reliability and construct validity of a web-based version of the R4C instrument. Methods: Fourteen physiotherapy experts and 39 final-year physiotherapy students completed the R4C instrument and the Pain Attitudes and Beliefs Scale for Physiotherapists (PABS-PT). Twelve experts and 17 students completed the R4C instrument on a second occasion. The R4C instrument was evaluated with regard to: internal consistency (five subscales of D1); test-retest reliability (D1–D4); inter-rater reliability (D2–D4); and construct validity in terms of convergent validity (D1.4, D2, D4). Criterion scores were generated based on the experts' responses to identify the scores of qualified practitioners' clinical reasoning abilities. Results: For the expert and student samples, the analyses demonstrated satisfactory internal consistency (α range: 0.67–0.91), satisfactory test-retest reliability (ICC range: 0.46–0.94) except for D3 for the experts and D4 for the students. The inter-rater reliability demonstrated excellent agreement within the expert group (ICC range: 0.94–1.0). The correlations between the R4C instrument and PABS-PT (r range: 0.06–0.76) supported acceptable construct validity. Conclusions: The web-based R4C instrument shows satisfactory psychometric properties and could be useful in education and research. The use of the instrument may contribute to a deeper understanding of physiotherapists' and students' clinical reasoning, valuable for curriculum development and improvements of competencies in clinical reasoning related to clients' behavioural change.

How important do you think it is for you to use these methods in your practical work? Base your responses on how important you feel the method is for you to use, not on your ability. Select the option (0-10) that best matches your opinion. How certain are you that you can use the following methods in your practical work? Select the option (0-10) that best matches your opinion.

Not Certain
Highly certain at all #15 Guiding the client to independently monitor their target behaviour in its natural context, e.g. through a diary

☐ Physical and biomedical factors
Joseph's description of his physical resources and limitations and/or biomedical factors that are of significance for his complaints and situation.
For example, strength, balance, mobility, pain, breathing and condition, as well as pathological or pathophysiological changes, e.g. pain, sensitivity, ligament and muscle injury ☐ Psychological factors Joseph's description of his psychological resources and limitations that are of significance to his complaints and situation.
For example, thoughts, feelings, beliefs, expectations, fear and coping.

☐ Contextual factors
Physical and social environmental factors and/or personal factors that may affect Joseph's complaints and situation.
For example, assistive devices, exercise equipment, friends and relatives, and the client's sociocultural backgroun ☐ Behavioural skills Joseph's description of his ability to perform a target behaviour in an everyday activity.
A defined target behaviour in an everyday activity that the client experiences as difficult and which hinders the performance of the activity. Target behaviour is something that the client feels is important to be able to manage. For example, putting on their socks, working at their computer or running on uneven ground.

☐ Lifestyle-related risk factors
Joseph's health-related living habits that are of significance to his complaints and situation.
For example, physical activity, smoking, alcohol consumption habits and eating habits. ☐ 3. The difficulty walking on the spiral staircase is caused by her knee locking, which inhibits muscle control and leads to a feeling of instability.

Domain 3 Functional behavioural analysis
☐ 4. The difficulty walking on the spiral staircase is caused by the staircase being so steep that her knee needs to be significantly flexed, which causes more pain.
☐ 5. Anna-Lena's ability to manage to walk on the spiral staircase is encouraged by her strong will to fend for herself.
☐ 6. The difficulty walking on the spiral staircase is caused by a decreased ability to walk when it is dark because her balance is impaired and she feels unsteady and uncertain. Physical examination shows full active and passive mobility in the left knee and bilateral muscle strength in the quadriceps.

+2
The hypothesis greatly weakens The hypothesis somewhat weakens The hypothesis neither weakens or strengthens The hypothesis somewhat strengthens The hypothesis greatly strengthens 2. The difficulty walking on the spiral staircase is caused by a fear of falling, resulting in Anna-Lena largely avoiding taking the stairs at home.
Anna-Lena rates 2 on how certain she is about walking down the stairs at home on a scale of 0 = not at all certain to 10 = very certain.

+2
The hypothesis greatly weakens The hypothesis somewhat weakens The hypothesis neither weakens or strengthens The hypothesis somewhat strengthens The hypothesis greatly strengthens 3. The difficulty walking on the spiral staircase is caused by her knee locking, which inhibits muscle control and leads to a feeling of instability.
Anna-Lena feels that the best thing she can do to prevent the pain and ache in her knee from increasing is to be careful with all unnecessary movements of the knee. The hypothesis greatly weakens The hypothesis somewhat weakens The hypothesis neither weakens or strengthens The hypothesis somewhat strengthens The hypothesis greatly strengthens 4. The difficulty walking on the spiral staircase is caused by the staircase being so steep that her knee needs to be significantly flexed, which causes more pain.
Anna-Lena's own self-monitoring through a diary reveals that she thinks it is easier to walk down the stairs with her left leg first.

+2
The hypothesis greatly weakens The hypothesis somewhat weakens The hypothesis neither weakens or strengthens The hypothesis somewhat strengthens The hypothesis greatly strengthens 5. Anna-Lena's ability to manage to walk on the spiral staircase is encouraged by her strong will to fend for herself.
Anna-Lena doesn't think that physiotherapy will reduce her pain in any significant way. The hypothesis greatly weakens The hypothesis somewhat weakens The hypothesis neither weakens or strengthens The hypothesis somewhat strengthens The hypothesis greatly strengthens 6. The difficulty walking on the spiral staircase is caused by a decreased ability to walk when it is dark because her balance is impaired and she feels unsteady and uncertain The steps of the spiral staircase are made of wood and when Anna-Lena walks down them in the morning, she is usually wearing knitted socks.

+2
The hypothesis greatly weakens The hypothesis somewhat weakens The hypothesis neither weakens or strengthens The hypothesis somewhat strengthens The hypothesis greatly strengthens

Domain 4 Strategies for behaviour change
The case description begins with a hypothesis/assumption. The hypothesis/assumption describes conclusions on how different factors affect the client's ability to perform the target behaviour in an activity that she wants to be able to better manage.

Item 1: Do you believe that Samira's difficulty performing her target behaviour right now is mainly due to physical/biomedical, psychological or environmental factors (physical and/or social)?
Indicate the importance of these factors by assigning a percentage to each of these factors (the percentages should add up to 100%). E.g. Physical/biomedical factors 30%; psychological factors 10% and environmental factors 60%.

Per cent % Physical/biomedical factors Psychological factors Environmental factors (physical and/or social)
Item 2: To help Samira achieve her target behaviour, you need to use different treatments/interventions. What four treatments/interventions do you think are most important at this stage and those you want to prioritise to begin with?
Formulate your answers as briefly as possible. No more than 10 words per answer.

More information about Samira
You have now consulted with Samira a few times and she has taken walks directly after work four times in the last week. However, she isn't quite satisfied. She wants to begin exercising, but she also wants to pick up her children from their afterschool programme as soon as possible after work.
Samira's target behaviour: Performing regular physical activity by walking 30 minutes a day.
Item 3: Below are treatments/interventions to support Samira achieve her target behaviour. In your consultation with Samira, you acquire new information that may affect your choice of treatment/intervention. You should now determine how this new information influence your choice of treatment/intervention.

Increase positive outcome expectations through graded activity
At follow-up one week after this treatment/intervention, Samira says that she has taken two short evening walks this week after the children have fallen asleep. It feels good that the exercise is not affecting her time with her children. Supplement B. Design and system requirements and related features in layout and function of the web-based application of the Reasoning 4 Change instrument.
Design and system requirements a

Features in layout and function
Easy access to the instrument Access is provided via a web-link. At any point, the user is able to pause progress and continue at a later, more convenient time and/or with a more convenient device (e.g. the user starts off with a PC and later finishes on a tablet).

Secure personal log-in
The user is provided with a personal ID code and selects a personal user name and password. All passwords are stored in an encrypted format. Provide the user with clear instructions and definitions of key concepts and arrange for reading request A clickable button is used to confirm that the instructions and definitions have been read. A word list, including instructions and definitions is available anywhere in the instrument.
Disable the ability to change a decision already made in the reasoning process When the submit button is pressed for a set of items, the provided responses are locked.

Limit information displayed on the screen to support simplicity
In the long list of strategies for behaviour change in Domain 4, descriptions and examples are concealed and are only displayed when the drop-down list of options is activated. Previous descriptions of a case are concealed but are always available in pop-up menus.

Controlling the response process
Controlling for correct answering procedure: Slider controls are used to provide responses in percentages, thereby controlling a maximum of 100%. All items and required response options must be answered to submit and proceed. The selection of more response options than requested is not possible. Write-in answers are regulated by a maximum word count.

Facilitate administration
An administration tool is included in the web application which enables administration and charting of items, responses, user profiles etc. Facilitate the scoring process Scores of individual items and total scores for subscales and domains are automatically generated and exported as files that meet the requirements for statistical analyses.

Facilitate instant reinforcement
The user interface provides positive feedback when the user completes a domain. As the user completes a set of items within a domain, progress is shown. Visible immediate results enhance the user experience (Tidwell, 2011). The interface provides an easy and fast way to start or continue the instrument when logging in to reduce time spent on navigation. Additional measures to facilitate instant reinforcement include an authentication system that allows the user to remain logged in while the browser tab or the browser itself is closed, thus, making it faster to resume completing the instrument.

Facilitate safe exploration
The user interface is designed so that users can safely explore its various parts without consequences of data loss in the form of lost user responses. A user may rapidly scan the interface and choose the first option that she decides is in line with her goal, even if it is the wrong action (Tidwell, 2011). Thus, the interface is designed to handle user mistakes in an intuitive manner.
Reduce cognitive cost Visual simplicity and plainly worded labels are provided throughout the interface. Menu items in the top bar contain both icons and text to increase recognition.

Facilitate interface consistency
Information and controls are placed in consistent locations. As the user navigates through the domains with different form elements, the overall layout, interface design and actionable elements remain similar. The domain elements are placed in consistent locations with identical aesthetic styles. These design choices prevent user errors caused by biased perception (e.g. habituation) (Johnson, 2014). Provide strong visual structure and distinct information presentation Input forms and information presented to the user contains clear visual structure. Text is appropriately formatted, visual noise is reduced and form elements enhance visual structure (e.g. striped tables, percentage sliders). Colours throughout the interface are selected to facilitate clear text, forms and navigation. Orange is used for information, blue for clickable buttons, and warm yellow, blue green, purple and green to distinguish the domains. If information and forms are correctly visually structured, scanning and comprehension is made easier and quicker (Johnson, 2014). a What the application should provide for or do.