Measuring Primary Health Care Clinicians’ Skills for Depression Management

Introduction: Primary health care clinicians play an important role in the management of depression. Thus, it is very important to have a valid and reliable assessment of the competences needed to manage depression in primary health care, with the use of clinical simulation providing such an opportunity. Objective: The present study describes the assessment of primary health care clinicians’ depression-related skills through a series of objective structured clinical examination stations. Material and Methods: Clinicians from multi-professional teams for the management of depression at two primary health care clinics in Santiago, Chile, went through seven objective structured clinical examination stations, lasting 10 to 20 min each, to assess their depression-related skills. The clinical and communicative skills measured were in accordance with clinical guidelines. Standardized patients portrayed cases usually encountered in clinical practice, while expert raters evaluated clinicians’ performance with standardized checklists. Results: Psychosocial clinicians performed better than biomedical clinicians in the assessed skills. The most notable results were as follows: a high level of accomplishment in the relationship with patient, medical anamnesis, health checkup, and lab test requests; heterogeneous performance in patient management according to screening results, feedback to the patient, and registration in clinical records; and major deficiencies in the differential diagnosis of bipolar disorder. Discussion: The objective structured clinical examinations administered provided an opportunity to perform an in-depth examination of the depression-related skills of primary health care clinicians, where flaws in the screening and diagnosis procedures used by biomedical clinicians were detected. Given the significant involvement of these types of clinicians in depression management, undergraduate-level and continuing health education opportunities are needed.


INTRODUCTION
Nearly 20% of adults attending urban primary health care (PHC) clinics have depression (1), one of the main public health problems globally (2). Depression has been linked to an increased risk of chronic diseases (3)(4)(5), decreased health-related quality of life (6,7), impaired social role performance (8,9), and excess mortality (10). Although effective depression management strategies exist (11,12), the research-to-practice gap in the provision of mental health services has prevented a vast majority of depressed people from receiving the timely evidence-based, high quality care they need (13), leading to a higher risk of recurrence and worse outcomes (14,15). Moreover, a lack of mental health human resources is a major contributor to the underuse of best available evidence (16).
A promising solution to these informational and financial barriers in PHC settings is to reallocate clinical responsibilities to non-specialized health care personnel with due training and supervision (17). Thus, a team-driven approach to training would be necessary to increase working capacity through collaborative care (17), a basic component for the development of effective integrated behavioral health care (18). Accordingly, in one of the regions hit hardest by depressive disorders worldwide (2), the 2013 World Health Organization Report on Mental Health Systems in Latin America and the Caribbean demonstrated the urgent need to increase the availability of training/educational opportunities for undergraduate health care students and PHC clinicians to learn about mental health subjects (19).
These mental health training programs require valid and reliable assessments of the competences of trainees-instruments that can inform future learning objectives, also known as formative assessments (20). Regarding clinical simulation [i.e., the incorporation of standardized patients into objective structured clinical examination (OSCE) stations focused on specific tasks], it has proven to be one of the most reliable, consistent, and realistic ways to measure the clinical and communication skills of health care students and clinicians (20). However, no OSCEs for assessing PHC clinicians' management of depression have been systematized in Latin America and the Caribbean region.
In the context of a randomized clinical trial (RCT) that implemented a comprehensive training and supervision program to enhance depression management in PHC in a Southern Latin American country, the present study describes the assessment of PHC clinicians' depression-related skills through a series of OSCE stations.

Study Design
A descriptive secondary analysis of the intervention arm's training component of the cluster RCT "Comprehensive Technology-Assisted Training and Supervision Program to Enhance Depression Management in Primary Care" (21) was carried out at two PHC clinics in Santiago, Chile.
The participants were clinicians who were working in multiprofessional teams for the management of depression within the selected PHC clinics and gave written informed consent before entering the study. The Ethics Committee of the Faculty of Medicine, Universidad de Chile, granted approval for the study under record number 103-2012.

Objective Structured Clinical Examinations (OSCEs)
Between April and October 2014, the PHC clinicians participating in the intervention arm were enrolled in a training program on depression management that lasted 24 teaching hours. Prior to being trained, the PHC clinicians' depressionrelated skills were assessed through a series of OSCE stations.
Seven OSCE stations were designed by the authors through an iterative process of discussion. Based on the National Depression Program in Chile (22), which defines the coordination among PHC clinicians and introduces a stepped-care model for depression, the clinical and communicative skills needed for the management of this condition in PHC were defined and specified for each of type of clinician (Figure 1). For research purposes, these skills were divided into groups of basic and observable behaviors that were used to build a checklist. This made it possible to conduct a systematic and objective assessment in which expert raters had to indicate the "presence" or "absence" of each attribute in the evaluated clinicians.
Complementarily, the scripts for each clinical scenarioi.e., OSCE station-were developed, taking into account the skills and behaviors defined above and ensuring that conditions resembled the usual clinical practice (e.g., typical profiles of depressive patients and average consultation times). Thus, specific OSCE stations were designed for each type of clinician ( Table 1). To this end, the content of each clinical scenario was carefully reviewed by the authors and by experienced PHC clinicians; all modifications were carefully discussed by the whole research team.
The clinical cases presented in each OSCE station were portrayed by standardized patients, defined by Barrows (23) as "[persons] carefully coached to simulate an actual patient so accurately that the simulation cannot be detected by a skilled clinician (…) [they present] the gestalt of the patient being simulated; not just the history, but the body language, the physical findings, and the emotional and personality characteristics as well. " The authors reviewed the scripts for each clinical scenario together with the standardized patients, supervising their training and giving them pointers on their nonverbal communication.
The clinical simulations were performed in the Clinical Skills Center of the Faculty of Medicine, Universidad de Chile. Three OSCE stations were implemented simultaneously, devoting a total of three teaching hours to each selected PHC clinic (i.e., PHC team). In each OSCE station, a PHC clinician entered a bay (unidirectional mirror room) where they were informed of the clinical scenario and the expected objectives through instructions placed on their desk, with the stated premise that they should behave as they usually did in clinical practice. In parallel, standardized patients moved from the actors' room to the waiting room and waited for the coordinator's signal to enter the bay, effectively beginning the OSCE station, which lasted between 10 and 20 min depending on the skills being assessed. The interaction was observed through the mirror by an expert rater who had a checklist for the assessment of clinical and communicative skills. Finally, the interaction between the PHC clinician and the standardized patient was interrupted by the expert rater when the time allotted to the station was over; at this point, the standardized patient left for the actors' room and the expert rater gave feedback to the PHC clinician in the bay for 5 min.
The expert raters who evaluated the participants were experienced researchers and/or academics in the field of depression and medical education who also held degrees in topics such as mental health care in the public sector, social work, and education in the health care sciences field. The raters had no prior relationship with the participants. Lastly, to ensure the quality of their reports, the raters were informed and trained to become familiar with the OSCE stations, the procedures involved in the study, and the way in which they should assess the skills evaluated by using a checklist, which made it possible to generate results in a standardized, systematic, and objective manner.

Results Assessment
The PHC clinicians' performance was evaluated according to the "presence" or "absence" of a series of basic behaviors, grouped as clinical and communicative skills and specified for each OSCE station (Figure 1). A median of 22 basic behaviors (minimum = 15; maximum = 30) and a median of 5 clinical and communicative skills (minimum = 4; maximum = 8) were assessed per clinical scenario. The raters had a specific checklist for each OSCE station, in which the expected basic behaviors to be observed were laid out and defined. One point was assigned if the behavior was present and 0 if it was absent.  Station Task   1 Midwife must perform a screening for depression on a 19-year-old mother in the postpartum period with suspected depression, who attends her well-child check-up. Duration: 10 min 2 Nurse must perform a screening for depression on a 70-year-old patient with suspected depression who attends a routine geriatric check-up. Duration: 10 min 3 Physician must follow steps to detect and diagnose a moderate depressive episode on a 21-year-old woman who comes to a spontaneous consultation. Duration: 10 min 4 Physician must follow steps to detect and diagnose a moderate depressive episode and subclinical hypothyroidism on a 35-year-old man who is referred by another member of the PHC team. Duration: 10 min 5 Physician must create a treatment plan for a 27-year-old mother in the postpartum period who consults spontaneously and presents with a moderate depressive episode with low suicide risk. Duration: 15 min 6 The psychologist/social worker must apply problem solving techniques to help a 34-year-old woman, victim of domestic abuse, who is receiving pharmacological treatment for a depressive episode. Duration: 20 min 7 Occupational therapist must perform an evaluation of occupational performance on a 35-year-old man with a moderate depressive episode, referred by a PHC physician. Duration: 15 min

Statistical Analysis
The PHC clinicians' checklists were consolidated and grouped by OSCE station. Afterwards, the percentage of assessed "present" behaviors and skills was calculated for the whole of the participants in each OSCE stations. According to the percentage obtained, the following qualification rule was applied: a) "Absent": 0-39%; b) "Less than acceptable": 40-59%; c) "Acceptable": 60-74%; d) "More than acceptable": 75-100%. Descriptive analyses were assisted by SPSS 12.0.

Sample Characteristics
The sample was composed of 41 PHC clinicians (34 women and 7 men). In Table 2, the distribution of the type of PHC clinicians per participating clinic is specified.

Observed Clinical and Communicative Skills
The average number of competences assessed as "present" by station ranged from 85.7% for occupational therapists in Station 7 ("Diagnosis of occupational performance in a depressed man") to 57.6% for nurses in Station 2 ("Screening for depression in an elderly patient") ( Table 3). It should be noted that occupational therapists in Station 7 and psychologists/social workers in Station 6 ("Problem-solving techniques to help a woman to cope with depression"), with 80.5% of skills being assessed as present on average, were classed as "More than acceptable, " while nurses in Station 2 and physicians-with an average of 59.2% of skills being assessed as present in Station 3 ("Comprehensive diagnosis of depression in a depressed woman")-were ranked as "Less than acceptable. " In Figure 2, the average performance observed in each of the assessed skills is summarized by OSCE station. In Station 1 ("Screening for depression in a postpartum woman"), midwives performed at a "More than acceptable" level in the following skills: "Relationship with patient" (97.2%) and "Screening application" (76.7%), while "Registration in clinical record" (16.7%) was qualified as "Absent. " On the other hand, nurses' performance was assessed as "More than acceptable" in the "Relationship with patient" skill (94.4%) and "Absent" for the "Management according to screening results" (33.3%) and "Registration in clinical records" (0.0%) skills, in the context of Station 2 ("Screening for depression in an elderly patient"). In addition, physicians assessed in Station 3 ("Comprehensive diagnosis of depression in a depressed woman"), were classed as "More than acceptable" in the "Registration in clinical record" (85.7%), "Medical anamnesis and health checkup" (85.7%), and "Relationship with patient" skills (78.3%); however, they received an "Absent" qualification in the "Differential diagnosis: Bipolar disorder" skill (26.6%). Also, in Station 4 ("Diagnosis of a moderate depressive disorder and subclinical hypothyroidism in a depressed man"), physicians performed at a "More than acceptable" level in the "Medical anamnesis and health checkup" (88.5%), "Lab tests requests" (85.7%), "Relationship with patient" (80.2%), "Registration in clinical record" (78.6%), and "Differential diagnosis: Hypothyroidism" (76.2%) skills, while "Differential diagnosis: Bipolar disorder" (19.4%) was found to be "Absent. " Likewise, the physicians' performance was assessed as "More than acceptable" in the following skills: "Registration in clinical record" (92.9%), "Relationship with patient" (81.5%), and "Suicide risk assessment" (78.6%); however, they earned "Acceptable" ratings-the lowest that they obtained-in the "Feedback to the patient" (74.7%) and "Treatment plan" (71.9%) skills, in the context of Station 5 ("Treatment plan for a woman with postpartum depression with low suicide risk").
Psychologists and social workers who were evaluated in Station 6 ("Problem-solving techniques to help a woman to cope with depression") obtained "More than acceptable" ratings in the "Assessment of baseline and treatment" (92.9%), "Relationship with patient" (84.5%), and "Problem solving technique" (79.6%) skills, with the exception of the "Registration in clinical record" (50.0%) skill, where they performed at a "Less than acceptable" level. Lastly, in Station 7 ("Diagnosis of occupational performance in a depressed man"), occupational therapists obtained "More than acceptable" ratings in all the assessed skills: "Relationship with patient" (100.0%), "Registration in clinical record" (100.0%), "Occupational performance assessment" (83.3%), and "Occupational intervention plan" (75.0%).

Main Results
After administering a series of objective structured clinical examinations (OSCEs), it was observed that PHC clinicians in Santiago, Chile, demonstrated an acceptable level of clinical and communicative skills related to the management of depression.  Overall, psychosocial clinicians (psychologists, social workers, and occupational therapists) performed better than biomedical clinicians (physicians, midwives, and nurses) in the assessed skills, as the latter showed flaws linked to depression screening and diagnosis. The most notable results were the following: a high level of accomplishment in relationship with the patient, medical anamnesis, health checkup, and lab tests requests; a heterogeneous performance in patient management according to the screening results, feedback to the patient, and registration in clinical records; and major deficiencies in the differential diagnosis of bipolar disorder and the diagnosis of depression in the presence of medical comorbidity.

Strengths and Weaknesses of the Study
Skills for the management of depression in PHC clinicians were assessed through OSCEs, a method highlighted for its reliability and consistency in comparison to other ways of measuring clinical skills, such as clinical cases and/or questionnaires. The multi-station, against-the-clock scheme of OSCEs simulates the usual conditions of a PHC clinic, encouraging clinicians to focus on brief and precise diagnostic interviews and/or interventions. Given the problems brought up by the in vivo evaluation of clinical and communicative skills, OSCEs are the best choice (24).
However, this study has some important limitations. For instance, the small sample size affects the possibility of generalization.
In this regard, it should be noted that this is a method for assessing clinical skills that requires significant financial and logistical resources and that the clinics that participated were representative of PHC in Chile's capital, in administrative and professional terms. Also, future versions of this OSCE should attempt to include pre-test assessments of inter-rater agreement and-according to the informal commentary of an external collaborator-a more diligent review of the objectives declared in each station, the message delivered to the PHC clinicians, and the contents of the assessment checklist.
In addition, variations in clinical and communicative skills found between biomedical and psychosocial clinicians working in PHC may be artificial, since the participating health care professionals were subjected to different evaluation processes (i.e., multiple OSCE stations that required specific clinical skills). However, the reason for these differentiated procedures is that, in Chile, depression management in PHC is performed, by FIGURE 2 | Performance in each of the skills assessed, according to OSCE stations. Symbols: ,"Absent"; , "Less than acceptable"; , "Acceptable"; and , "More than acceptable." definition, by a multidisciplinary team whose members possess a distinctive set of clinical skills. Future studies, apart from increasing the number of participating clinicians, must conduct comparisons among those belonging to the same category, adjusting their results by covariates of interest.

Comparison With Prior Work
In Latin America, there are no published studies on the use of OSCEs for assessing and/or training skills for the management of depression in PHC. Previous experiences in the region have used theoretical and attitudinal tests as well as clinical case vignettes for the assessment of these skills (25,26); in addition, the literature describes comparisons of self-report questionnaires or clinical interviews by mental health specialists versus clinical evaluation by physicians to determine, specifically, the discriminatory capacity of the latter (25)(26)(27). In all these studies, the main conclusions are that there is significant room for improvement in the clinical skills of PHC clinicians.
Globally, OSCEs have been used both to evaluate and to train these skills in a variety of contexts. For example, in the case of nurses who work in outpatient cancer treatment centers, it has been utilized to observe their capacity to recognize and discuss depressive symptoms with patients (28). In addition to this, the studies by Falluco et al. (29,30), are the only precedents where PHC clinicians have been trained with an OSCE to improve their competence in risk assessment for adolescent depression and suicide. In this case, a training program was designed for these purposes, one of its components being the training of skills with standardized patients through an OSCE (29,30). This intervention improved the participants' confidence, knowledge, and clinical practice in relation to risk assessment for adolescent depression and suicide in PHC (29,30).
These studies have innovated in the evaluation of clinicians' depression management competences, extending previous uses of OSCEs in educational contexts. In the case of internal medicine residents, this tool has been used to compare how depressive symptoms can affect non-verbal communication and anamnesis in patients with physical comorbidity (31). In addition, OSCEs have been used in mental health training/education programs for nursing students and pediatrics residents, where their potential to improve clinical skills linked to depression management has been demonstrated (32).
Given that the use of OSCEs in the assessment of clinical skills related to depression management in PHC is relatively recent, there is a long road ahead for research and innovation in this area.

Implications for Practice and Research
Even though depression can be treated effectively by nonspecialized health care personnel (33), the detection and management of depression in PHC can be challenging and complex, seriously hindering timely access to treatment (34,35). In this context, assessment with OSCEs makes it possible to identify clinical and communicative skills of PHC clinicians with a significant margin for improvement, which can be then added to the future curricula of training programs aimed at these professionals.
In this regard, this study revealed that biomedical clinicians performed more poorly in areas such as screening and depression diagnosis. Notably, physicians and nurses not specialized in mental health share a large part of the direct intervention and coordination involved in depression management in PHC, however, the mental health training that these professionals receive amounts to less than 10% of their working hours (19). An education program focused on professional practice that combines additional hours of training in mental health with incentives to improve clinical practice, utilizing OSCEs as a formative tool (29,30), can significantly improve clinical outcomes.
Another aspect that required additional training was the comprehensive diagnosis of depression and its differential diagnosis in the presence of medical and/or psychiatric comorbidity. In this regard, the presentation of depression in conjunction with physical and/or psychiatric conditions represents a significant challenge for depression detection, diagnosis, and treatment by PHC physicians, who usually spend their consultation time exploring physical symptoms that mask emotional problems (34). Regrettably, comorbidity between depression and other conditions is the norm rather than the exception in PHC, and major efforts will be required for educational and health institutions, as decision makers, to integrate the complex management of depression to education curricula, continuing education, and clinical practice guidelines (36).
Additionally, in this study, the OSCE was able to reveal problems in clinical record keeping. This can negatively impact the continuity of care of patients with depression, becoming an obstacle to better interaction and formal communication between the members of the PHC team, which reduces the quality of teamwork. This has been identified in the literature as a significant barrier to structured and multi-professional management of depression in PHC (37).
Future studies should include bigger and more balanced samples of PHC clinicians, be able to determine how clinical and communicative skills for the management of depression vary by sex, age, hours of mental health training received in undergraduate or continuing health education, years of practice, and type of clinician, among other variables. There is evidence that skills for depression management may depend on certain socio-demographic factors (25) and on a person's attitude towards mental health topics (38). More extensive knowledge in this area will make it possible to personalize mental health training by adapting currently available educational opportunities to the needs of non-specialized healthcare personnel.
Considering the financial and logistical resources demanded by OSCEs, there may be attractive alternatives for their implementation in PHC, according to the emerging evidence. For instance, carefully trained, incognito, standardized patients making unannounced visits to PHC may avoid the interruptions in usual clinical practice that OSCEs require, thus facilitating the examination of actual practice habits (39). In parallel to this, current technology may enable the automatization, standardization, and mass implementation of evaluation processes through the use of tools such as virtual standardized patients or avatars, which have been recently introduced to medical education (32).
Finally, and given that limited consultation time in PHC could impose restrictions on information collection and registration, it is important for future clinicians' aids to focus on patient evaluation, integrating technologies to make it easier for them to record, store, and access clinical information in these contexts (40), thus freeing them from some of their significant administrative load.

CONCLUSIONS
The OSCE provided an opportunity to assess in depth the clinical and communicative skills of PHC clinicians for the management of depression. Although the clinicians had an acceptable level of competences, there are still areas that require further attention, such as the screening and diagnosis procedures used by biomedical clinicians. Given the significant involvement of these types of clinicians in the management of depression, better opportunities for mental health training should be available at both undergraduate and continuous health education.

DATA AVAILABILITY
The datasets generated for this study are available on request to the corresponding author.

ETHICS STATEMENT
The participants were those clinicians who were working in multiprofessional teams for the management of depression within the selected PHC clinics and gave informed consent before entering the study. The Ethics Committee of the Faculty of Medicine, Universidad de Chile, granted approval for the study under record number 103-2012.

AUTHOR CONTRIBUTIONS
PM coordinated the study, analyzed the data, and wrote the draft of the manuscript. GR conceived and designed the study, carried out parts of the data collection, and critically reviewed the manuscript. VM contributed analysis tools and critically reviewed the manuscript. RM analyzed the data and critically reviewed the manuscript. JC assisted in the coordination of the study, carried out parts of the data collection, and critically reviewed the manuscript. And VG participated in the discussions and helped write the manuscript.

FUNDING
This study was funded by CONICYT (National Fund for Science and Technology Research) under project FONDECYT no. 1100205, the Millennium Science Initiative of the Ministry of Economy, Development and Tourism, grant "Millennium Nucleus to Improve the Mental Health of Adolescents and Youths, Imhay" and the Fund for Innovation and Competitiveness (FIC) of the Chilean Ministry of Economy, Development, and Tourism, through the Millennium Science Initiative, Grant No. IS130005. These funding agencies did not contribute to the design or implementation of the study or to the data analysis process, nor did they influence the preparation, review, or approval of the final manuscript.