What Does Effective Teaching Look Like? Profession-Centric Perceptions of Effective Teaching in Pharmacy and Nursing Education

INTRODUCTION Much of the published literature surrounding teaching effectiveness is based on student perspectives. Explicit standards for what constitutes effective teaching in health professions education do not currently exist. Best practices for training and evaluation in teaching effectiveness could be better addressed with detail from expertbased sources. OBJECTIVE This study sought to identify elements of effective teaching by gathering perceptions of exemplary educators. In addition, profession-centric differences in perceptions of effective teaching were also evaluated. METHODS An iterative consensus-building method was used to gather the perceptions of nursing and pharmacy educators regarding effective teaching. Individual semi-structured interviews were initially used to gather example items within four pre-defined categories: effective methods, ineffective methods, active learning methods, and effective traits. These example items were then collated into an electronic survey that was then administered to the same participants to be rated on a numerical scale. RESULTS Ten educators from nursing and pharmacy participated in this study. Based on the participants’ rankings of the collated lists, a high level of consensus was observed for many items in each of the four categories. With other items, wide variability in ratings was also seen. Further analysis by discipline subgroups revealed patterns of item ranking that appear to be profession-centric. CONCLUSION Exemplary nursing and pharmacy educators revealed consensus regarding perceptions of certain elements of effective teaching. The results also suggested some elements where profession-centric perceptions exist. These results could be incorporated into best practices for effective teaching training and interprofessional teaching design.


Introduction
Academe has a responsibility to provide students with effective educators.In health professions education, if the process is ineffective, student learning is compromised and society may not be provided with quality professionals.A positive correlation between effective teachers' skills and student learning has been shown in academic performance and student outcomes in elementary and higher education (Goldhaber & Anthony, 2007;Trautmann, 2008).Specifically, researchers have found that "teacher quality is a significant, if not dominant, variable in achievement outcomes" in referring to overall student performance (Ganley, Quintanar & Loop, 2007).Effective teaching has also been positively correlated with student performance within health professions education (Cooke, Irby, Sullivan & Ludmerer, 2006;Hickok, 2006;McDonough & Bennett, 2006;Whitcomb, 2007).Although accreditation bodies for health professions education have standards that call for quality educators to deliver the curricula, these groups, such as the Accreditation Council for Pharmacy Education and the Commission on Collegiate Nursing Education, do not provide specific guidance on how to determine educator quality and/or teaching effectiveness.This leaves many educators and administrators in these fields wondering: What does effective teaching look like?
Characteristics of effective teachers have proved challenging to clearly identify.Anecdotally, a common answer when asked to describe a skilled educator is: "You know a good teacher when you see one." Many studies have explored students' perspectives of effective teaching (Jones, 2008;Lohman, 1996;Ramsden, 1992;Singh & Stoloff, 2007;Song, 2006;Walker, 2008).Students have endorsed descriptors such as "excitement" and "passion" as frequently as "knowledge" and "skill" when describing memorable teachers.The value of student evaluations in identifying effective teachers has been conflicting (Davidovitch & Soen, 2006;Edstrom, 2008;Marsh, 2007).Student evaluations have often been the standard for identification of good teachers or effective traits, however many question whether students are qualified to provide valuable subjective input (Barnes, 1985;Marsh, 2007;Solem & Foote, 2006).
Characteristics of effective teaching have not been extensively studied from the educator's perspective.However, a consistent finding from the available literature is that excellence in teaching includes knowledge as well as disposition of the educator (Jones, 2008;Lohman, 1996;Walker, 2008).Moreover, effective educators have mastered skills such as thorough comprehension in the field of study, understanding of the learning process and teaching techniques (Davidovitch & Soen, 2006;Ramsden, 1992).This leads to the next question: Does effective teaching look the same in different fields of study?Bain (2004) completed a longitudinal study to identify the shared characteristics of effective teachers across multiple disciplines in higher education by compiling over 15 years' worth of student interviews and surveys.Some tenets of effective teachers mirrored other findings such as knowledge in subject area, preparation and organization, and treating students with "simple decency" regardless of the discipline (Bain, 2004, p. 18).Bain also

Implications for Interprofessional Practice
• Different professions have similar perceptions of effective teaching in some areas; these similarities should be used to develop initial best practices for teaching training and interprofessional teaching design • Profession-centric perceptions should be identified and incorporated into interprofessional curriculum to proactively embrace commonality and celebrate differences uncovered additional commonalities of effective teaching such as approaching teaching as a serious responsibility, providing methodical and systematic assessments, expecting "more" from students routinely, and creating a "natural critical learning environment" (Bain, 2004, p. 18).Literature searches in health professions education revealed a paucity of publications focused on across-discipline or even discipline-specific evaluations of teaching effectiveness.For the most part, the literature that is available has been limited to discipline-specific perspectives of effective teaching within medical and dental education (Snell et al, 2000).
The purpose of this study was to identify elements of effective teaching methods and traits using a panel of exemplary educators from nursing and pharmacy education.A secondary objective was to evaluate potential profession-centric differences in perceptions of effective teaching.These results could be used in the development of effective teaching best practices for future training and evaluation of health professions educators.Profession-centric differences could be particularly informative for interprofessional education curricula that utilize team teaching approaches with an interdisciplinary faculty.

Design
This study utilized a consensus method to capture the perceptions of contemporary nursing and pharmacy educators to characterize effective teaching.Interviews were conducted using a modified Delphi method to identify elements within four categories: (1) effective teaching methods, (2) ineffective teaching methods, (3) active learning techniques, and (4) effective teaching traits.The modified Delphi method provided an opportunity to gather expert opinions from diverse geographic areas and, through the process of rating, reach consensus (Turoff & Hiltz, 1995).The modified Delphi method begins with open-ended submissions from the expert panel and, through a series of iterative cycles, identifies consensus (Custer, Scarcella & Stewart, 1999).In this study, two cycles were completed.
Exemplary educators in nursing and pharmacy education were identified to participate in the interviews.
Nursing and pharmacy were selected to represent two disciplines of health professions educators.Exemplary educators were identified using recommendations from professional organizations, published literature, and academic administrators.Criteria included more than three years teaching experience, currently teaching in professional programs, and public recognition of teaching expertise.A random sample of 10 exemplary educators was invited from the field of candidates to participate in the panel.Participation involved one telephone interview followed by an email and second telephone communication over a period of two to three weeks.
Semi-structured telephone interviews were conducted with individuals, independent of other participants.Telephonic communication was used to increase participation and allow descriptive responses.Interviews were completed using an investigator-developed Exemplary Educators Interview Survey (Appendix 1) that gathered participant demographics and perceptions of teaching methods and traits surrounding the aforementioned four categories.The interview questions were developed using a combination of published research (Davidovitch & Soen, 2006), training texts, and input from academic colleagues.The first phase of telephone interviews occurred at a prescheduled time with an expected duration of 45 minutes.The information obtained from the first phase of interviews was collated into the four major categories: effective teaching methods, ineffective teaching methods, active learning techniques, and traits of effective educators.When possible, each collated category list was condensed for duplicative statements.This condensed list (Appendix 2) was then emailed to the same participants that were interviewed in the first cycle.The second cycle of the modified Delphi process then had the same participants independently rate the collated list of items based on their personal perception and then return the completed ratings to the investigator.Interviewees rated the collated statements in the four categories, using a ten-point scale (1 to 10) with three narrative anchors: 1 = Strongly Agree, 5 = Neutral, and 10 = Strongly Disagree.The investigator subsequently contacted interviewees and completed a short telephone interview to clarify incomplete responses.Educators were instructed by the investigators to predicate their ratings based on their experience and observation in generalizations.All participants provided informed consent and the Institutional Review Boards of Pacific University, Washington State University, and A.T. Still University approved this study.The investigators analyzed responses provided by the participants during the first cycle to identify those that were similar or identical.As each participant was asked to provide three examples for each of the four categories, 30 discrete responses for each category were possible.This initial analysis resulted in a condensed list of responses in each category: Effective Teaching, 25 items; Ineffective Teaching, 23 items; Active Learning Methods, 25 items; Traits of Effective Teachers, 23 items.Appendix 2 lists the specific items in each of the four categories.
The overall and discipline-specific results from participants' ratings in the second cycle are shown in aggregate (median and ranges) in Figure 1 (following page).
In the overall results, participants were least variable in their ratings for examples of Traits of Effective Teachers, with median ratings for the individual examples ranging from 1.0 to 2.0 (Item Ranges: 0-2).Variability in the other categories was much greater: Effective Teaching Methods, (Median scores ranging from 1.0 to 4.0; Item Ranges: 1-5); Ineffective Teaching Methods (Median scores ranging from 1.0 to 3.5; Item Ranges: 1-8); and Effective Active Learning Methods, (Median scores ranging from 1.0 to 3.0; Item Ranges: 2-7).
Effective Teaching Methodology items for which there was a high level of consensus (i.e., a range of 1 or less for the individual item) included active engagement with students in class, displaying problem solving and decision-making, showing respect for students, and modeling professional behavior.Within the Ineffective Teaching Methodology statements, a high level of consensus was found (i.e., a range of 1 or less for the individual item) for arrogance and lack of availability to students, showing a lack of civility toward students, and displaying a lack of caring if students learn.Active Learning items that were found to have a high level of consensus included the use of facilitated discussions, case studies, and critical thinking exercises with evaluation and feedback.Active learning techniques wherein there was greater variability in ratings as to their effectiveness included computerized case studies and videotapes or web links to stimulate active learning.Less variability was noted for Effective Teaching Traits.Statements such as approachable, compassionate, and organized appeared to reflect personality styles of the educator.Other traits, however, such as openness to other opinions and encouragement of student thinking are more reflective of progression to student-centric modes of teaching methodology.
When evaluating the results by discipline subgroups, nursing educators provided identical ratings for 8 (33%) of the Effective Methods, 6 (25%) of the Ineffective Methods, 2 (8%) of the Active Learning techniques, and 17 (80%) of the Effective Traits.Pharmacy educators, however, did not demonstrate identical ratings for any of the Effective Methods, Ineffective Methods, or Active Learning items and held identical ratings for only 1 (4%) of the Effective Traits.Median scores and response ranges for these items (both overall and by discipline) are presented in Figure 1 with details of each numbered item corresponding to that listed in Appendix 2.

Discussion
This study revealed that perceptions of teaching efficacy among exemplary educators are broad and diverse yet common themes were also apparent.Each interviewee was enthusiastic to participate in this study, which may be reflective of their interest as exemplary educators.
While different perceptions of effective teaching were collected, once the examples were listed, the exemplary educators reached a high level of consensus in many areas.Nursing educators appeared to show higher consensus than pharmacy educators in discipline subgroup analysis.
The demographic data showed the propensity of these educators to seek teaching training after graduate education was completed, perhaps explaining their mastery of teaching.Almost one-third (30%) of the participants had moved into administrative roles as Deans, which may increase the attention to teaching effectiveness in those programs.As expected, interviewees were older, taught longer, and had advanced degrees.Similar characteristics were noted in Bain's (2004) analysis of effective college teachers.
While teaching awards currently exist, it was unfortunate more metrics were not available to identify exemplary educators.The challenges experienced in identifying and recruiting exemplary nursing faculty in particular were unexpected.Lack of recognition or perceived value of such may have been related to recruitment challenges.Nursing educators in the cohort received fewer national teaching recognitions than pharmacy educators, possibly indicative of the educational culture rather than re-

Figure 1 Effective Teaching Survey Results
Shapes represent median ratings while lines represent the range.Appendix 2 lists the corresponding details for the numbered methods and traits on the axes.flective of the individual's performance: if recognition is not provided, effective nursing educators may not seek participation in a study of this nature.Further research in this area may be warranted.
The first cycle of open-ended submissions produced a plethora of useful examples.The investigators expected more commonality in the distinct statements yet were delighted with the variation of responses.Interviewees emphasized there were exceptions to some examples provided, supporting the concept of individuality and innovation in teaching.The interviewees also found "it is easier to describe what not to do" when trying to describe an example of an effective teaching method.The premise of identifying elements of effective teaching methods and traits using a panel of exemplary educators from nursing and pharmacy education was met with great support whereas previous studies used student perspectives (Singh & Stoloff, 2007;Song, 2006).This attitude may be reflective of individuals who display a passion for teaching.
Although active learning is not a new concept, the number and creative nature of the examples provided by the educators were refreshing.One educator described the use of a specific publication revisited throughout the semester as a tool for understanding and applying statistics allowing students to learn application in tandem with critical thinking while contributing to a current topic.
Other examples, such as integrated practice laboratories, have been embraced but the educators emphasized the application of coaching and facilitation as much more effective than lecturing or directing.Most educators commented on the variability of activity success with the preparation and engagement of the educator.One educator recommended applying active learning sparingly: "Use it for burning questions and high-level concepts but be selective: keep them wanting more." This study was able to identify components of effective teaching methodology with a small cohort representing two health professions educators.The components of effective teaching that reflected high consensus, such as active engagement, problem solving, respectful, and open-minded, were similar to published studies (Bain, 2004;Singh & Stoloff, 2007) and could be incorporated into best practices for training and evaluation.It may appear these are obvious statements yet universally recognized best practices do not currently exist (Jones, 2008).Future studies may continue to accrue more insight by including other health professions and more participants.
Difference in rating variances observed between the two professions may be reflective of cultural differences that influence the perception of what is considered effective teaching within the given profession.The idea that learning cannot be separated from the culture within which one gains a skill or set of skills is not new.Lev Vygotsky, a Russian-born developmentalist, details how our social environment shapes thought in that the cultural relevance of information is transmitted to the learner along with the information itself (Vygotsky, 1978).Within the health care arena, professional culture has been identified as a potential barrier to successful interprofessioanl teamwork surrounding patient care, though these may extend to possibly also include curricular design and teaching efforts (Hall, 2005).
It is, therefore, possible that the culture of nursing profession is one wherein there is a strict protocol for not just what information is to be presented, but how learning is to be presented.If the culture of nursing is one in which deviation from the norm is not as highly regarded as in other professions (i.e., awards in recognition of innovation in teaching), the implicit message might exist that teaching effectiveness does not deviate from established cultural tradition.The perception of effective instruction is, in effect, inherently standardized.More importantly is that this is just a single example to possibly explain the complexity of the larger concept of professional culture and ultimately how it could impact even the early teaching and learning that occurs in interprofessional health care education.This would support the idea that interprofessional educational efforts should include "'professional' culture competency" training for students and faculty alike (Hamilton, 2011).
In that effective instruction is inherently entangled with the transmission of cultural norms and values, one would expect a developmental trajectory wherein individuals more acculturated to a given profession would differently interpret teaching effectiveness.Basically, those individuals who are either farther along in their training or are currently active members of the professional community will interpret teaching effectiveness in terms of the values of their professional culture.Those individuals, on the other hand, who are early in their professional training, will base their assessment of teaching effectiveness at the level of the individual rather than as an expres- Health & Interprofessional Practice | commons.pacificu.edu/hip1(1):eP1003 | 8 sion of the profession; they will base their assessment on a host of interpersonal and stylistic variables that, while important, are not reflective of the norms of the profession.These questions have not been addressed in the literature and future studies would be needed to determine if one would see different ratings of effectiveness.
There are some limitations to this study.The size and composition of the sample in this study should be taken into account when attempting to generalize the results.
In particular, only two professions were incorporated in the study.Additionally, the nature of the semi-structured telephone interview process introduced the possibility of interviewer bias, primarily through clarification dialogue surrounding the categorization of item examples into one of the four pre-defined domains.For example, the interviewer may have asked for clarification whether the item provided by the interviewee was really an "effective method" or instead an "effective trait, " with the final categorization based on the interviewee perspective.
This study was designed to provide results that were practical and readily applicable, yet also hypothesis-generating for future research.The modified Delphi method employed in this study allowed delineation of perceptions and expert opinions to produce items of consensus that could be readily adapted for faculty development and teaching evaluation.This is a necessary first step in the identification of areas that require more rigorous qualitative study methods to better elucidate reasons for large variation in perceptions of teaching effectiveness.
In a few areas, significant divergence in perceptions was observed between the two disciplines.Pre-requisite knowledge of both commonalities as well as differences in perceptions of teaching effectiveness could be important when members of these disciplines are working together to design educational programming.In a more general sense, these results hint at possible core philosophical differences between health professions educators with regard to ideas of effective teaching.Again, knowledge of these differences can be taken into account when curricula, such as those related to interprofessional education, are being developed by individuals from multiple disciplines.Proactive strategies could be considered to mitigate potential obstacles these differences could result in, thus making for a more cohesive and effective team to deliver interprofessional health education.

Conclusion
A consensus-building approach of exemplary educators revealed some perceptions of effective teaching that appeared to be consistent across nursing and pharmacy disciplines.Expert perceptions, which display consensus, could be incorporated into best practices for effective teaching training and interprofessional teaching design.Other perceptions, which appeared profession-centric, could be applied in interprofessional teaching design to proactively embrace commonality and celebrate differences.