The Effects of Engagement Activities on Pre-Physical Therapy Students Across One Semester of Class

Active engagement has been found to maximize learning, enhance personal development, and enrich communication in the classroom. This study examined how participants’ perceptions of engagement – measured by questions from the National Survey of Student Engagement (NSSE) changed in response to three problem-based learning (PBL) activities; peerled workshops, service learning, and journal clubs. Sixty-two undergraduate students enrolled in an introductory class to a career in physical therapy were recruited for three consecutive semesters. Participants were administered a shortened version of the NSSE at the beginning and end of their semester of class. Two NSSE benchmarks active and collaborative learning and supportive campus environment – were perceived by participants as improved, partially supporting the hypothesis. Participants reported increased engagement in six scalelets (specific subsets of educational experiences) course challenge, writing, active learning, collaborative learning, diversity, and interpersonal environment. Responses to several scalelets, however, showed a decline in engagement higher order thinking and information technology. This is the first study to look at changes in NSSE scores across only one semester of class. The study supports PBL, however, recognizes that class design, composition, and instructor may impact student perceptions.


Introduction
Active engagement has been found to maximize learning and enhance personal development in the classroom (Ahlfeldt, Mehta and Sellnow 2005;Carini, Kuh and Klein 2006;Chung et al. 2009;LaNasa, Cabrera and Transgrud 2009;Pike 2006a;Preszler 2009;Elder et al. 2011).Students reported a better understanding of course concepts and displayed greater active participation, when actively engaged (Ahlfeldt, Mehta and Sellnow 2005;Chung et al. 2009;Preszler 2009).Research has shown that student engagement is positively related to academic outcomes such as grades and persistence (Kuh 2001b;Kuh et al. 2008;Schuetz 2008;Salamonson, Andrew and Everett 2009).In fact, engaged students were more likely to pursue a second year of education at the same academic institution (Kuh et al. 2008).Student engagement has been shown to indicate a state of optimal student functioning and has been associated with a student's overall health and well-being (Bruce, Omne-Ponta and Gustavsson 2010).
Student engagement can be defined as both the time and energy students invest in educationally purposeful activities and the efforts institutions devote to using effective educational practices (Kuh 2001a).A variety of methods have been utilized in postsecondary education to actively engage studentsproblem-based learning, teambased learning, interactive lectures, group discussions, and peer teaching (Lake 2001;Clark et al. 2008;Elder et al. 2011).While research findings were generally positive, several studies documented outcomes which were less positive.For example, Butler, Phillman and Smart (2001) reported that the use of CARDS, writing answers during class to questions about presented material, did not stimulate intellectual activities outside of the classroom.Students' perception of learning and overall evaluation of the course instructor declined when comparing group discussions to traditional lectures in advanced physiology (Lake 2001).In a medical ethics course, students reported a decreased perception of cooperation and self-learning after participating in a team-based learning approach (Chung et al. 2009).Finally, nursing students reported they did not like their grades being dependent on group projects in team-based learning (Feingold et al. 2008).
Despite a number of studies about active engagement in the classroom, lecturing is still one of the most predominant methods of information dissemination in the education of health care professionals (Lake 2001).The use of lectures is particularly problematic in health care disciplines, as content-specific gaps may exist between instructors and students.These gaps may limit the ability of entry-level students to assimilate information presented by health care professionals and could further limit the ability of the students to generate their own knowledge base (Preszler 2009).Problembased learning (PBL) is one method designed to promote a learning environment that enhances communication and active learning, when lecture alone may not be sufficient for student knowledge (Lake 2001;Preszler 2009).PBL was first used in medical education in the 1950's and has become a prominent pedagogical method in medical and health-science programs (Hung, Jonassen and Liu 2008).PBL has been cited as being effective in facilitating students' problem solving skills and enhancing selfdirected learning (Schmidt 1983), a necessary component of health education.In particular, PBL is thought to be effective because learning is embedded in authentic tasks that apply to everyday concepts (Hung, Jonassen and Liu 2008).The characteristics of PBL thought to enhance learning include: (1) knowledge being stimulated by the problem and applied back to the problem, (2) student centered learning, (3) self-directed learning, (4) self-reflective approach, and (5) teachers are facilitators to the process (not knowledge disseminators) (Hung, Jonassen and Liu 2008).
Three techniques that promote PBL are small group discussions (Gedula 1997; Lake 2001), peer-led workshops (Preszler 2009), and service learning (Atler and Gavin 2010;Bridges et al. 2010).Small group discussions have led to enhanced learning for students in engineering, psychology, anatomy, and medical education courses (Gedula 1997; Lake 2001).These small group discussions allowed students to generalize from specific observations, apply concepts to new situations, and think critically (Lake 2001).
Peer-led workshops improved student performance, improved student retention, and improved the quality of student learning in an introductory biology course (Preszler 2009).Small cooperative groups were also found to help students solve challenging problems, evaluate case studies, and even improve general learning skills (Preszler 2009).Finally, service learning was shown to help students in both occupational and physical therapy reflect upon personal observations, plan a course of action, and evaluate outcomes of activities.Service learning provided health care students the opportunity to witness real-life issues and engage in active problem solving (Bridges et al. 2010).
As with many health care disciplines, the practice of physical therapy involves complex cognitive processes (Lake 2001).Authors of educational texts often argue that lecturing alone is inappropriate for the presentation of complex, detailed material, especially when that material must be learned and later used at high cognitive levels (Lake 2001).Health care professionals should have educational curricula designed to foster active, self-directed learning (Lake 2001).The purpose of this study was to examine how pre-physical therapy participants' perceptions of engagementmeasured by questions from the National Survey of Student Engagement (NSSE) -changed in response to three PBL activities -peer-led workshops, service learning, and journal clubs -designed to promote high levels of cognitive thinking.The hypothesis was that participants would report higher levels of engagement in five benchmarks identified by the NSSE, after participating in a class that utilized these problem-based activities.

Participants
Approval for the study was obtained from the Institutional Review Board at a Midwestern state university in the USA.A total of ninety undergraduate students, enrolled in PT 302 'Physical Therapy as a Profession', were recruited for three consecutive semesters: winter 2009, fall 2009, and winter 2010.To be included in the study, the student had to be enrolled in PT 302, be at least 18-years-old, and sign the informed consent.Students were excluded from the study if they did not meet all of the inclusion criteria.All ninety-students enrolled in the three classes met the inclusion criteria, and were therefore invited to participate in the study.
No invited student was in a physical therapy program at the time of the study.

Course Design
PT 302 was selected for analysis, as the class was designed for students considering a career in physical therapy and was taught by physical therapy faculty members.As mentioned previously, health care professionals typically engage in high levels of cognitive thinking.The addition of PBL activities to PT 302 was hypothesized to allow the participants an opportunity to engage in content specific, PBL before actual acceptance into a physical therapy program.The class met one-time-per-week over the course of 14 weeks, was held for approximately two hours, and provided a basic introduction to a career in physical therapy.The class was taught by two different instructors, one during the winter semesters and another during the fall semester.While the use of two instructors might be thought to impact student perceptions, previous research reported that changing class instructors did not lead to significant changes in learning gains, exam results, or student attitudes (Pollock 2005).In fact, the most significant change occurred with the addition or deletion of interactive engagement activities within the classroom (Pollock 2005).To further counteract the effect of different instructors, PT 302 was organized to include a number of guest speakers.The guest speakers were the same for all classes, regardless if the class occurred in the winter or fall semesters.
The design of PT 302 included a number of problem-based activities to engage students.First, the class included a 'Journal Club' where students were responsible for reading professional journal articles prior to class.During class time, the students discussed the articles in small groups of 4-5 students, organized and led by a different member of each group each week.Small group discussions have led to enhanced learning in engineering, physiological psychology, anatomy, physiology, and medicine (Gedula 1997;Lake 2001).A second activity involved service-learning.Students selfselected a four-hour service project that either involved working with a patient population receiving physical therapy that the student was unfamiliar with, or working with a national, regional, or local branch of the American Physical Therapy Association.
After the service project, the student completed a self-reflection paper which focused on the experience, the profession of physical therapy, and how the activity directly related to the course content.Service-learning was shown to positively change occupational therapy students' knowledge, skills, and confidence after as little as four hours of service provided to patients with neurological conditions (Atler and Gavin 2010).A third engagement activity involved small, self-selected groups of 3-4 students who participated in peer-led workshops.The workshops involved the 3-4 students evaluating an area of physical therapy practice, selecting appropriate reference material to represent that practice area, and developing/presenting a 30-minute lecture.During the 30-minute lecture, the 3-4 student leaders presented relevant information about the practice area and included an engagement activity to help the remainder of the class learn the material.Examples of engagement activities used during the student lectures included jeopardy games, crossword puzzles, and physical exercise practice.The use of peer-led workshops was shown to improve student performance, enhance student retention, and improve the quality of student learning in a biology course (Preszler 2009).Finally, PT 302 was structured without examinations or quizzes, since the intention was to allow students to explore the topics, ask questions freely, and not worry about memorizing specific content.The class was organized and graded-based upon the three problem-based activities.

Outcome Measure
In 2000, the National Survey of Student Engagement (NSSE) was introduced to colleges and universities across the United States (LaNasa, Cabrera and Transgrud 2009), to identify the level to which students reported being actively engaged in educational processes and what students gained from the educational experience overall (Kuh 2001b).The NSSE contains five benchmarks as indicators of student engagement: (1) academic challenge; (2) active and collaborative learning; (3) student/faculty interactions; (4) enriching educational experiences; and (5) supportive campus environments (LaNasa, Cabrera and Transgrud 2009).Field testing in 1999 revealed that the NSSE had a high degree of face validity with faculty and academic administrators.In addition, psychometric analyses found most items to be reliable (Ouimet et al. 2004).Since the original NSSE was eight-pages long and contained questions which were not being addressed during PT 302 (e.g.work on a research project with a faculty member), a shortened version of the NSSE was developed for this study to include activities that would be addressed in PT 302.A usage agreement was signed between Indiana University's Center for Postsecondary Research and the authors of this study prior to any data collection.Our shortened NSSE contained questions from each of the five benchmark indicators identified previously.Shortened versions of the compared to the full NSSE.Ahlfeldt et al. reported a 14-question survey tested student engagement adequately and had a solid structure when compared to the NSSE (Ahlfeldt, Mehta and Sellnow 2005).
Although institutional surveys -such as the NSSE -provide a variety of data to participating institutions, barriers can arise when attempting to use the data to make changes, as the results are often too general (Pike 2006a).Pike proposed that departments and colleges use scalelets to select areas of measure that are pertinent to that department or college.Specifically, a scalelet is a set of the NSSE survey questions that relate to a specific aspect of the educational experience (Pike 2006a).Pike created 12 scalelets based upon a selection of questions from the original NSSE survey, and reported that all 12 scalelets produced dependable group means (Pike 2006b).In addition, further investigation by Pike revealed the NSSE scalelet scores -conducted on groups of 25-50 students -had both convergent and discriminant validity allowing for greater explanatory power and richer detail than the NSSE benchmarks alone (Pike 2006a).For these reasons, the survey tool used in our study contained 11 of the scalelets developed by Pike (Appendix 1).Questions from one scaleletvaried experienceswere not selected for our survey as PT 302 did not address any of the topics in the scalelet.In addition, our survey used only 24 of the questions included in Pike's original scalelets, questions we felt to be most pertinent to PT 302.Thus, despite other researchers finding that shortened versions of the NSSE were valid and reliable, our survey tool was not tested prior to the study.In addition, no other study in the literature review examined student responses on a national survey of student engagement across only 14 weeks of class.In fact, Pike identified that more research is needed at both the college and department level, for time periods that are meaningful for these units (Pike 2006a).

Procedure
This study utilized a pretest -post-test design.During the first class period of the semester, participants were given a verbal description of the study, a copy of the informed consent, and our shortened version of the NSSE by a research assistant.The research assistant was responsible for all data collection; the class instructor was not present during survey completion.The research assistant separated informed consents from the surveys and entered all data.The procedure was repeated at the end of each semester.
The shortened NSSE used in this study contained three Likert scales.For all but four of the questions, a declining score on any of the Likert scales represented participating in the task more frequently, or improvement.For example, Question 1a asked the participant, 'In your experience during PT 302 or the last semester, how often did you ask questions in class or contribute to class discussion?' The answer options were: very often (1), often (2), sometimes (3), or never (4).A score that changed from a 2 to 1, for example, represented an improvement.The four reverse-scored questions also utilized Likert scales.Question 1f was worded in a negative fashion, 'How often did you come to class without completing readings?'A pre-survey score of 2 (often) improved if the score changed to 3 (sometimes) in the post-survey.The three remaining questions used reverse Likert scales.For example, 'Relationships with faculty members' improved if the score changed from (1) unfriendly to (7) available, helpful, sympathetic.

Data Analysis
SPSS version 14.0 was used for data analysis and significance was set at .05.
Descriptive statistics --comparing the participants on items such as age and gender -were calculated using both t-tests and chi-square tests.To determine if the three groups differed significantly on any demographic variable, a one-way analysis of variance (ANOVA) was calculated for age and Kruskal-Wallis one-way ANOVA by ranks were calculated for all other demographic data.Next, each of the 24 survey questions (Appendix 1) was analyzed to determine if the three student engagement activities outlined earlier had any impact on participants' perceived levels of engagement.examining the change values for the paired survey answers for each question, across each semester of evaluation, the interaction between time (pre versus post) and subject group (semester of evaluation) was examined.For questions where a significant difference was found among the overall mean ranking of participants in the three semester groups, a 'minimum significant difference' (MSD) was calculated.The MSD computation included a Bonferroni-type adjustment which resulted in a p-value of less than 0.017 (0.05 / 3 = 0.0167).

Findings
Although a total of 90 students enrolled during the three semesters, only 86 participants completed the survey.Of those 86 participants, 24 had missing answers on one or more items on the questionnaire.Therefore, data from 62 participants were used in statistical analyses.The participants were primarily full-time students (98.4%), female (62.9%), attending their senior year of college (64.5 %) (Table1).Although the average age of the subjects was 23.3 years old (+ 2.3 years), there was a statistically significant difference in age of 1.6 years between the male and female subjects [t(60) = 2.9, p < 0.01)].Specifically, the male participants were more likely to be 24 years or older, while the female participants were more apt to be 22 years old or younger [ 2 (4) = 11.3,p = 0.02].The participants in the three classes were not found to differ significantly in age, gender, enrolment status or reported grade point average.However, the participants in the fall semester 2009 were statistically different in college classification (p = .025).Fall participants had a greater number of seniors (n = 19) compared to participants in either winter semester (winter 2009 = 9, winter 2010 = 12).Group mean ranks for the pre-versus post-survey scores were compared for all three semesters.Significant differences were observed in nine of the 24 questions (Table 2).Examination of these nine questions revealed that participants felt they had more opportunities to participate in six of the tasks and fewer opportunities for the three remaining tasks.In general, participants felt more opportunities for the following:  Wilcoxon Signed-Ranks tests were also conducted to exam changes in group means, pre-versus post-survey, by semester of attendance.Fourteen questions were found to be statistically different for at least one semester (Table 3); nine were perceived to improve, five were perceived to occur less frequently.Two questions were found to be significantly different for all three semesters, regardless of the instructor of record; using e-mail to communicate with an instructor and memorizing facts.To determine if any question was significantly different between semesters, the change, calculated independently, between the pre-versus postsurvey scores across the three semesters, was compared using a Kruskal-Wallis ANOVA by ranks.If a significant difference was found, the absolute value of the difference was evaluated against a 'minimum significant difference' as explained earlier.The interaction (between semester of evaluation and pre-versus postscore) revealed no significant differences between the three semesters.4. Overall, our study examined 24 questions, finding 14 questions with significant differences from beginning to end of the study, and 10 questions without any changes.

Discussion
As mentioned earlier, PT 302 is an undergraduate-level course designed to explore a career in physical therapy.During the study, the students were primarily graded on three problem-based activities rather than on memorized facts.Participants were graded on written analysis of three professional journal articles, on evaluation and reflection of a service learning experience, and creation of a peer-led workshop.According to Bloom's Taxonomy of learning, levels of cognition progress from simple to complex in the following order; remembering, understanding, applying, analyzing, evaluating, and creating.While NSSE questions 2a -2e (higher order thinking skills) do not specifically replicate Bloom's Taxonomy, the cognitive skills being questioned do progress from simple to more complexmemorizing, analyzing, synthesizing, making judgments, and applying.Participants in all three semesters reported the coursework for PT 302 did not emphasize the simplest cognitive task of memorizing facts, ideas or methods (q.2a) (Table 4).Traditional classroom assessment techniques such as examinations and quizzes may have been seen by participants as requiring more memorization than problem-based activities used in this study.In addition, in the winter 2010 perceived fewer opportunities from pre-to-post survey for analyzing basic elements of ideas (q.2b) and synthesizing /organizing ideas (q.2c) (Table 4).The authors cannot explain why participants in the winter of 2010 perceived less engagement in analyzing and synthesizing ideas since both winter classes were taught by the same instructor and this cohort of participants did not differ significantly from participants in the other two semesters.However, previous researchers have documented that students may perceive they learn less despite having high academic outcomes (Lake 2001), and abilities of group co-members may impact the students' perception of overall learning and grades (Feingold et al. 2008).PBL, in general, has been criticized for its emphasis on facilitating higher order thinking and problem solving, often at the expense of lower level knowledge acquisition (Hung, Jonassen and Liu 2008).Students taught by traditional curriculum performed better in basic knowledge acquisition, while students in PBL performed better in clinical knowledge (hands-on skills) and reasoning (Albanese and Mitchel 1993;Vernon and Blake 1993).
Our participants, however, did not perceive any significant differences from preto post-survey in the highest levels of cognitive thinking (q.2d, q.2e) (Table 4).This finding is consistent with other studies where PBL was not found to be superior to traditional learning in acquisition of either basic skills or clinical knowledge (Berkson 1993;Colliver 2000).In fact, PBL resulted in the same acquisition of knowledge as traditional teaching (Berkson 1993;Colliver 2000).Furthermore, students engaged in PBL curricula had no difference in problem solving processes or abilities, compared to students in traditional curricula (Saarinen-Rahiika and Binkley 1998).The true test of the effectiveness of PBL may lie in the learners' ability to recall information at periods of time far removed from the learning situation.Norman and Schmidt (1992), for example reported that PBL classes foster better retention of knowledge over long periods of time, rather than during the initial acquisition of the knowledge.Furthermore, initial knowledge differences may be nullified by retention differences in student in PBL compared to students in more traditional learning formats (Saarinen-Rahiika and Binkley 1998).The present study did not examine long-term knowledge retention, and students may have not seen improved learning skills during the short time of the study.
The class structure may have also contributed to this study's findings.First, all students were taught by the same guest speakers (faculty members who still maintained a clinical practice), class size was kept small (n=30), and students were encouraged to interact without worry of being tested on the material.In a small genetics education class, for example, students reported being able to freely ask questions of an instructor was more conducive to learning than large lectures (Cameron 2003).The freedom to interact with the guests may have led our participants to feel improvement in interpersonal environment, through improved relationships with faculty members (q.5b), other students in the class (q.5a), and administrators within the School of Health Sciences (invited guests to final class presentations) (q.5c) (Table 4).Second, the class assignments were constructed to increase the course challenge.Students were graded on drafts of several papers, potentially contributing to the participants feeling they had enhanced opportunities for writing (q.1c) (Table 4).Students also participated in weekly journal clubs with grading dependent on participation and analysis during class time.
Participants, therefore, may have felt they could not come to class without completing the assigned readings (q.1f).Third, PT 302 was structured so that students worked within a variety of groups.Students participated in peer-led workshops which forced them to develop a presentation (active learning -q.1b), and complete work outside of class (collaborative learning -q.1h) (Table 4).Finally, the instructional design provided participants multiple opportunities for feedbackfinal papers were graded after reviewing drafts, presentations were graded the same day, and students provided feedback to each other during the journal club and peer-led workshops.The participants reported more opportunities for receiving prompt written or oral feedback from faculty (1m), and perhaps less need to e-mail (1i) or discuss class grades with the instructor (1j) (Table 4).Thus, it is logical that course interaction had mixed results, and the participants reported a lower frequency of using technology, specifically in the form of e-mailing the instructor (1i).However, Lim and Rodger reported that when students enter into dialogue and make their own judgments about what they and other classmates have done, feedback is enhanced (Lim et al. 2010).
Overall, participants in the fall 2009 perceived more opportunities for engagement in seven scalelet categories, compared to increased engagement in two scalelets for each of the winter semesters ( Two of the NSSE benchmarks were consistently perceived as having increased or unchanged opportunities: (1) active and collaborative learning, and (2) a supportive campus environment (Table 4).The level of academic challenge was improved in the scalelets of course challenge and writing.However, higher order thinking was perceived as occurring less frequently in the simplest form of cognitionmemorization -and among participants in the winter of 2010 who perceived less analysis and synthesis from the pre-to post survey, as explained previously.The benchmarks of student interaction and enriching educational experiences had mixed results.
Participants did report an improved perception of immediate feedback from instructors (q.1m), which may have caused the participants to perceive they emailed faculty less (1i) -information technology -and did not need to contact faculty to discuss grades (1j) -course interaction.Very few participants emailed the instructors during any of the semesters of PT 302.While many of the participants did meet with the instructors during office hours or after class, these meetings were not directed at questions about course grades but rather specific questions about the field of physical therapy.In addition, both the service learning project and journal club assignments asked students to reflect upon a wide variety of physical therapy populations.This reflection component may have led the participants to include diverse perspectives in class writing assignments and discussions -diversity (q.1e).The final two scaletes were unchanged, out-of-class interactions and support for student success (Table 4).

Limitations
While this study was conducted over three semesters, only 62 participants (69% of the available sample) completed the survey in its entirety.Participants who completed the survey may or may not be reflective of the remaining 28 students who opted not to answer all questions or participate in the study.For example, undergraduate students who did not complete the NSSE survey had slightly higher ratings of engagement compared to students who completed the NSSE, when invited for feedback (Kuh 2001b).In addition, students may have opted not to answer specific questions for reasons not identified by the survey.A second limitation of the current study is that the students who participated had obtained junior or senior status and, thus, the results cannot be generalized to students of freshmen or sophomore status.A third limitation is the lack of a control group.Perhaps the effects of active engagement activities could be better teased out from the course design -elements such as having no written examinationsby comparing two classes designed the same except for one element (e.g. one class has examinations and the other does not).
There are limitations to the questionnaire used in the study.4).The authors are unsure if these results are due to the use of a different instructor for the fall semester or the differences in the class composition of the fall cohort (significantly more senior students).However, it should be noted that the between semester data analysis revealed no statistical differences between any of the semesters.At least a moderate Cohen's effect size (d=.040)would have been needed to detect a statistically significant difference between the cohorts based upon the sample size.
Finally, traditional formats of testing learning outcomes may place students engaged in PBL activities at a disadvantage.Often testing is focused on immediate retention of knowledge, or memorization of facts.The use of pre-versus post-test questionnaires (including the NSSE) may not be the best method to assess the ability of these pre-health professionals in their ability to problem solve or critically think.Further work is needed to develop valid measures of knowledge and critically thinking for students in who engage in PBL activities.

Conclusion
This study supports the use of three PBL activities -peer-led workshops, journal club, and service learning -as methods to enhance student engagement in two of the five NSSE benchmarks, thereby partially supporting our hypothesis.Without using traditional classroom methods such as lecturing and examination, participants reported increased engagement in six scalelets -course challenge, writing, active learning, collaborative learning, diversity, and interpersonal environment -, decreased engagement in two scalelets -higher order thinking and informational technology -, mixed results in another scalelet -course interaction -, and no change in the last two scalelets measured by our survey -out-of-class interactions and support for student success.Although more content can be provided via traditional lecture, PBL may help to teach complex cognitive skills in health care pre-professional students.However, additional evaluation methods need to be developed.This is the first study to look at changes in student perceptions of engagement -measured by the NSSE-in only one semester.The study support PBL, however, recognizes that class design, instructor and class composition may impact student perceptions.
First, Wilcoxon Signed-Ranks tests were conducted to compare the pre-versus post-survey responses for all participants combinedregardless of semester of attendance -to determine both a direction of change, if any, and a relative difference in change, if any, for each question.Next, Wilcoxon Signed-Ranks tests were applied to the same preversus post-survey data, with the results run by semester of evaluation; winter 2009, fall 2009, and winter 2010.As a follow-up measure, a Kruskal-Wallis one-way ANOVA by ranks was employed to compare the pre-calculated change between the pre-and postsurvey scores for each participant, across the three semesters of data collection.By prepare in advance for class (complete several drafts of a project, work with other students outside of class, complete readings prior to class); develop relationships with other students and faculty; and receive prompt feedback from faculty on academic performance.The participants felt they engaged in fewer opportunities of contact of faculty by e-mail, inquire about academic performance; and memorize facts and ideas.
upon negative ranks, (b) based upon positive ranks, (c) sum of negative and positive ranks equal Light grey = reduced engagement, Dark grey = enhanced engagement Only one question was dedicated to out-of-class interactions and asked students about discussing career plans with faculty members.Participant responses may have not changed from pre-to post-survey since the students were already in the upper level of courses and had made a decision about their career choice -to take PT 302 a student had to have declared a minor in pre-physical therapy.Finally, support for student success was measured by responses to how often students had serious conversations with other students who were very different in terms of religion, politics, or values.The cohort of students taking PT were very homogenous and not diverse, full-time students (98%), from the local area (100%), and between the ages of 21-25 years (100%).Practical implicationsPBL activities were found to improve active and collaborative learning, and perceptions of a supportive campus environment, as expected.However, participants in the class felt they had the same or fewer opportunities for higher level thinking.Higher level thinking in the health professions is crucial as there is an expectation of increased memory retention, greater ability to apply knowledge to clinical contents, and the development of life-long learning habits among these professions (Saarinen-Rahiika and Binkley 1998).Based upon the results of this study, several changes are anticipated in our class.First, student preparation is critical to maximize learning in the PBL environment (Saarinen-Rahiika and Binkley 1998).Plans are in place to include a brief orientation to the PBL activities during the first day of class each semester.This will enable students to understand the expectations of the learning process and guide the students toward a more self-directed learning style.Second, since traditional examination has not been shown to adequately evaluate students in PBL programs (Saarinen-Rahiika and Binkley 1998), the course instructors are examining other methods to evaluate measures of knowledge and problem-solving.Consideration is being given to longterm follow-up of students who attended PT 302 to determine if retention from the class is observed 3-4 years later, during the final-year of successful admission into a physical therapy program.
Although scalelets have been found to have convergent and discriminant validity (Pike 2006a), our survey was not validated prior to use.By preselecting questions we felt would be pertinent to PT 302, a limited number of questions represented some of the scalelets (only 1-2 questions).In addition, by focusing on questions relevant to our class, bias may have occurred.Studies should also be conducted to determine what amount of change is considered a 'significant' change on the NSSE.Such information would allow future authors and classroom instructors to determine if results are truly significant without the use of a control group.Participants in the fall 2009 perceived more opportunities for engagement compared to participants in either winter semester (Table

Table 4 : Means for Significant Findings from Tables 2 and 3 -(by Benchmarks and Scalelets)
* Pre= pre-survey scores, Post = post-survey scores, [R] = reverse scored Grey = enhanced engagement, † = no significant difference in scores (Pike 2006a)all findings according to the five NSSE benchmarks and 12 scalelets identified by Pike(Pike 2006a).The means of significant findings are reported for participants: (1) for all semesters combined, and (2) by semester of attendance.As can be seen in Table4, students perceived more opportunities (or unchanged opportunities) in two benchmarks: active/collaborative learning and supportive campus environment.The remaining three benchmarks had mixed results (academic challenge, student interaction with faculty members, and enriching educational experiences).
this may have occurred during only one semester.Participants reported fewer opportunities or no change in higher order thinking and information technology, and reported mixed results in course interaction.No significant changes in perception were observed in scalelets for out-of-class interaction or support for student success.As mentioned previously, the scalelet for varied experiences was not addressed by our questionnaire and does not appear in Table (Pollock 2005)gh the participants had similar characteristics as far as age, gender, and reported grade point average, Ponta and Gustavsson 2010), changes in engagement due to class standing were not found in our literature review.Perhaps these seniors had more experiences to draw upon in either exposure to problem-solving activities or exposure to the field of physical therapy.Bruce, Omne-Ponta and Gustavsson (2010) did report that having prior experience in a health care sector led to increased active engagement in a nursing class.Another difference between the fall and winter cohorts was being taught by a different instructor.Although previous studies have not found significant changes in learning gains or student attitudes in classes with different instructors(Pollock 2005), results from the fall 2009 cohort did differ from the other semesters.For this reason, all plausible explanations should be considered.