Introducing and Evaluating a “Study Smarter, Not Harder” Study Tips Presentation Offered to Incoming Students at a Four- Year University

This paper: (1) briefly outlines a study tips presentation that uses both evidence from the cognitive and educational psychology literatures as well as demonstrations to teach students how to study more effectively, and (2) provides empirical evidence about whether this study tips presentation affects students’ study habits. We provide a brief overview of the presentation, a handout that summarizes the tips, and a reference list rich with sources that support the efficacy of these study approaches. We also summarize a study we conducted to evaluate the effectiveness of the presentation. Thirty-two students completed a questionnaire about their typical study strategies before and three months following the presentation. Additionally, 102 students who did not attend the presentation (control group) completed the study strategies survey, and their responses were compared to those from 74 students who had attended the presentation sometime between 3 months and 3 years and 3 months earlier. Finally, the 74 presentation attendees rated their memory for, utilization of, and perceived influence of the eight study tips. Results support the efficacy of the “Study Smarter, Not Harder” presentation as a way to improve students’ understanding and utilization of effective study approaches.

strategies to their academic success in the classroom (Bartoszewski & Gurung, 2015;Gurung, Daniel, & Landrum, 2012). Thus, research focused on improving students' study approaches has the potential to influence students' academic lives broadly and positively.
In perhaps one of the most comprehensive empirical overviews of the varied effectiveness of different study approaches, Dunlosky et al. (2013) evaluated ten common study strategies by examining their generalizability across settings, across learners, across materials and across types of outcome measures. They reviewed the educational and cognitive psychology literature pertaining to each and found that not all strategies are equally effective. In the conclusion of their article, they recommend that students be taught how to use study strategies to their advantage. This is not a new recommendation, as many past researchers have emphasized the importance of educating students about study techniques that work well (Marshak, 1984;Pressley, Goodchild, Fleet, Zajchowski, & Ellis, 1989). Despite this, we could find very few studies in the literature that involve teaching students study techniques and then evaluating whether the study tip training positively influences academic success. In 1999, Beidel, Turner, and Taylor-Ferreira published a study demonstrating that teaching test-taking strategies to elementary students with test anxiety can decrease their anxiety and improve their academic performance as measured by their grade point average. Additionally, Chen, Chavez, Ong, and Gunderson (2017) demonstrated that simply encouraging college students to self-reflect about (1) the resources they planned to use to prepare for exams, (2) why they were selecting those resources, and (3) how they would use them improved students' test scores in an introductory statistics course. Thus, evidence suggests that simple interventions can positively affect students' academic performance, but we were unable to find past research addressing whether direct instruction about effective study strategies can improve college students' approaches to studying in their everyday academic lives. Dunlosky and colleagues (2013) point out that one barrier to faculty incorporating lessons on study approaches into their classes may be that they, themselves, do not know which strategies are effective or how to teach students to implement them. Thus, our paper has two goals: (1) to briefly outline a study tips presentation that uses both evidence from the cognitive psychology literature as well as demonstrations to teach students how to study more effectively that faculty can use as a model for utilization with their own students, and (2) to provide empirical evidence about whether this study tips presentation affects students' study habits so that faculty can decide whether the effort to teach students to study more efficiently is worthwhile. Although these study tips are based on evidence from the Cognitive and Educational Psychology literatures, they are general study tips that are applicable to college students regardless of their background or major.
The first and second authors originally generated the "Study Smarter, Not Harder" presentation in response to a request from students in our department for an evening workshop that taught them how to study more effectively. A member of our Learning Resource Center (LRC) attended and asked us to create a short one-page overview of our study tips for inclusion in a newsletter distributed to all incoming students at our university. The following semester (August of 2012), the LRC asked us to share our presentation as part of a student orientation program on campus. Since then, we have regularly presented these study tips to students on our campus, largely during optional student orientation events: College 101 (August 2012, September 2013, and September 2014, Stress Less Week (April 2014), and the newer Become Your Best Bulldog portion of Orientation Week (August 2015 and August 2016). Across the same period of time, we have also shared the "Study Smarter, Not Harder" tips and associated evidence with faculty at a national conference (see Gingerich & Lineweaver, 2011) and at professional development workshops on our own campus and on other campuses in hopes that other faculty members would, in turn, share the tips with their students. Both our student and our faculty audiences have represented a broad array of disciplines. Although we have received positive evaluations and feedback from student and faculty attendees immediately after these sessions, we did not have any data that indicate whether students ultimately use the tips in their classes after attending our session. We designed this study to evaluate the effectiveness and usefulness of our presentation by asking students who have attended one of our sessions to complete a brief survey about their study habits and their perceptions of the study tips.
We hypothesized that "Study Smarter" informed students would report using study strategies that relate to the eight tips we present more often than and would perceive these strategies as more effective than strategies unrelated to the tips. We expected these views to be stronger after the presentation than before the presentation, and we expected them to be stronger in students who attended the presentation compared to a control group of students who did not. We also expected "Study Smarter" students to remember the eight study tips we presented, to report using them in their coursework, and to perceive them as contributing positively to their academic success. We conducted this study because we wanted to determine whether these tips are helpful to students and whether any effect of attending the presentation persists across time.

An Overview of the "Study Smarter, Not Harder" Presentation
We created the "Study Smarter, Not Harder" presentation to teach faculty and students eight tips for maximizing study time and study effort. The one-hour presentation includes empirical evidence in support of most of the tips combined with multiple demonstrations designed to illustrate each in a convincing and memorable way. The eight tips include (1) Pay Attention, (2) Skim Listen Read Repeat (SLRR), (3) Don't Rote Memorize, (4) Study A Little A Lot, (5) Quiz Yourself, (6) If at First You Don't Succeed, Try Something Else, (7) It's Never Too Early, and (8) Take Care of Yourself. (See Appendix 1 for the handout that briefly summarizes each of these tips.) Additionally, our reference list highlights many additional empirical articles that support each of the tips. For brevity and readability, we did not incorporate these references into the text of our article. This reference list is by no means exhaustive (the literature on many of these tips is quite extensive), but rather offers a sampling of studies in the literature that provide evidence that each of these tips should positively influence students' learning in academic settings. We offer this as a resource for faculty who would like to teach students to study more effectively. Another wonderful resource is a recent article by Putnam, Sungkhasettee, and Roediger (2016). A careful reading of their paper will reveal much of the same advice we offer here with additional empirical support for these tips. Finally, we also invite interested readers to email us (tlinewea@butler.edu or mhall2@butler.edu) if they would like an upto-date, complete version of our power point presentation that they can share with others or that they can modify for their own purposes.

Participants
One hundred and seventy-six students at a mid-sized private mid-western university participated in this study. The participants were divided into two groups. The first group was comprised of attendees (n = 74), students who had attended one of our "Study Smarter, Not Harder" presentations in the past and who agreed to participate in the study by completing our online questionnaires. We recruited attendees through a personal email invitation sent to 223 students who had attended a 2013, 2014, or 2016 presentation (no information was available for students who had attended the 2015 session, and 2012 attendees were likely to have graduated by the time we conducted this study). The second group included non-attendees (n = 102). We recruited these control group students from our general Psychology Department undergraduate participant pool through Sona, an online participant Lineweaver, Hall, Hilycord, and Vitelli Journal of the Scholarship of Teaching and Learning, Vol. 19, No. 2, March 2019. josotl.indiana.edu management system. Thus, all of our control group students were enrolled in a psychology course ranging from Introductory Psychology (primarily non-majors) to an upper level psychology course (primarily Psychology majors or minors) at the time of their participation. On average, the students were 19.5 years old (SD = 1.26), and were in their sophomore year. The majority of participants were female (82%). The attendee and non-attendee groups were not reliably different in age (t (174) = .53, p = .59), class year (t (174) = .37, p = .71), or gender, X 2 (n = 176) = .73, p =.69.

Materials
Demographic Questionnaire. This questionnaire assessed participants' demographic characteristics such as their age, gender, class year, and which presentation (if any) they had attended. It also asked the participants who had attended Become Your Best Bulldog in 2016 to indicate their name. This allowed us to link their baseline data to their post-presentation data.
Study Strategies Questionnaire. Created for the purpose of this study, this questionnaire asked all participants to indicate how effective they believed 20 study strategies to be (1="not effective" to 5="highly effective") and to specify how often they utilize the same 20 strategies (1="never" to 5="always") as part of their typical study approach. We drew several of these strategies from those described by Dunlosky et al. (2013) and supplemented them with study strategies that students frequently report using (e.g., looking over notes or using flashcards) and strategies that directly related to our presentation (e.g., skimming reading before class or changing ineffective strategies). Ten of the strategies related to tips from the presentation and ten did not. (Please see Appendix 2 for a full copy of the questionnaire.) We tallied participants' responses for tip-related and tip-unrelated strategies, with higher scores indicating a more positive perception of the effectiveness and a higher reported usage of the strategy type (possible range = 10 to 50).
"Study Smarter" Questionnaire. Only attendee participants completed this questionnaire, which was also created specifically for this study and which directly assessed the perceptions of each of the eight study tips from the "Study Harder, Not Smarter" presentation. Each of the attendees rated the eight tips on how well they remembered it (1="do not remember" to 4="vividly remember"), how often they use it (1="not at all" to 4="a lot"), and how much they believe it has positively influenced their academic success (1="not at all" to 4="a lot"). For the questions about frequency of use and perception of positive influence, we also included an option for students to indicate that they could not remember the tip well enough to rate it; in this case, we excluded these responses from analysis. (Please see Appendix 3 for a full copy of the questionnaire.) Possible scores on each of the three subscales ranged from 8 to 32, with higher scores indicating better memory, greater utilization, and more positive perceptions of the study tips.

Procedure
Each participant completed the study online. We used Limesurvey, an open source online survey application, to build the online assessment that encompassed all of the questionnaires. The majority of the participants only completed this online portion, which was available for one month from November 1 st , 2016 to December 1 st , 2016. However, a subset of the attendee group (i.e., those who attended the 2016 Become Your Best Bulldog "Study Smarter, Not Harder" presentation (n = 32)) also provided baseline data by filling out a hard copy of the Study Strategies Questionnaire immediately before the study tips presentation began.
When taking the online survey, attendees completed all three questionnaires in a specified order: first the Demographic Questionnaire, next the Study Strategies Questionnaire and, finally, the "Study Smarter" Questionnaire. Non-attendees followed the same procedure, except their survey did Lineweaver, Hall, Hilycord, and Vitelli Journal of the Scholarship of Teaching and Learning, Vol. 19, No. 2, March 2019. josotl.indiana.edu not include the "Study Smarter" Questionnaire. For attendees, the elapsed time from their attendance at the presentation to the time they completed the online survey ranged from three months (2016 attendees) to three years and three months (2013 attendees). The attendees whom we recruited through email received either a $5 Amazon gift card or extra credit in a psychology course in return for their time. Non-attendee control participants received extra credit in a psychology course for their participation.

Hypothesis 1: Perception and Usage of Common Study Strategies Before vs After the Presentation (Study Strategies Questionnaire)
To evaluate changes in attendees' perceptions of and usage of common study strategies that were related versus unrelated to the eight presentation tips from before to after the presentation, we ran two 2 (Strategy: Tip-Related vs Tip-Unrelated) x 2 (Time: Baseline vs Post-Presentation) repeatedmeasures ANOVAs. Both factors were within-subjects. Data were available only for the 32 attendees of the 2016 Become Your Best Bulldog session (i.e., the only presentation at which we administered the Study Strategies Questionnaire immediately before sharing the tips) who also participated in the follow-up online study (three months after the presentation). The dependent measures in the two analyses were (1) ratings of strategy effectiveness and (2) ratings of frequency of strategy use.
For strategy effectiveness, the main effect of Strategy neared significance (F (1, 31) = 3.54, p = .069, ηp 2 = .10), and the main effect of Time reached significance, F (1, 31) = 9.29, p = .005, ηp 2 = .23. However, these two main effects were qualified by a significant Strategy x Time interaction, F (1, 31) = 14.38, p = .001, ηp 2 = .32. See Figure 1, panel A. At baseline, before hearing the presentation, students who attended the presentation rated the tip-related strategies and the tip-unrelated strategies as similarly effective, F (1, 31) = 0.40, p = .53, ηp 2 = .01. However, approximately three months later, they believed that the tip-related strategies were significantly more effective than the tip-unrelated strategies, F (1, 31) = 32.02, p < .001, ηp 2 = .51. Interestingly, this was due to a decrease in the ratings of effectiveness of the tip-unrelated strategies from baseline to post-presentation, F (1, 31) = 15.08, p = .001, ηp 2 = .33; perceptions of the tip-related strategies were stable across the two assessments, F (1, 31) = 1.23, p = .275, ηp 2 = .04. For strategy effectiveness, students rated tip-related (solid lines) and tip-unrelated (dashedlines) strategies equivalently at baseline, but tip-unrelated strategies as less effective than tip-related strategies after the presentation. For strategy use, students reported using both tip-related and tipunrelated strategies more frequently after the presentation than at baseline.

Hypothesis 2: Attendees' vs Non-Attendees' Perceptions and Usage of Common Study Strategies (Study Strategies Questionnaire)
To determine whether students who attended the "Study Smarter, Not Harder" presentation had more positive perceptions of tip-related study strategies and were more likely to use these strategies in their everyday lives than students who did not attend the presentation, we ran two 2 (Strategy: Tip-Related vs Tip-Unrelated) x 2 (Group: Attendee vs Non-Attendee) repeated measures ANOVAs. Strategy was a within-subjects factor, whereas Group was a between-subjects factor. The dependent measures mirrored those from the previous set of analyses: 1) ratings of strategy effectiveness and 2) ratings of frequency of strategy use. Data were available for all 176 participants, but only reflected students' ratings after having attended the presentation; we did not include baseline data in these analyses. For effectiveness, there was a significant main effect of Strategy, F (1, 174) = 104.19, p < .001, ηp 2 = .38. Participants perceived tip-related study strategies (M = 32.36, SD = 3.07) as significantly more effective than tip-unrelated study strategies (M = 29.56, SD = 3.85), regardless of whether they had attended the presentation or not. The main effect of Group neared, but did not reach, significance, F (1, 174) = 3.25, p = .073, ηp 2 = .02. The interaction between Strategy and Group also failed to reach significance, F (1, 174) = 0.27, p = .61, ηp 2 = .002. Figure 2 illustrates attendees' and non-attendees' utilization ratings of tip-related and tipunrelated study strategies. For usage, the main effect of Strategy was again statistically significant (F (1, 174) = 16.34, p < .001, ηp 2 = .09). Unexpectedly, both groups reported greater use of tip-unrelated than tip-related study strategies, perhaps due to the popularity of the tip-unrelated strategies (e.g., highlighting, reviewing notes) we selected for comparison with the tip-related strategies. The main effect of Group was again not significant, F (1, 174) = 1.56, p = .21, ηp 2 = .01. Unlike for effectiveness, for strategy use, the interaction between Strategy and Group was statistically significant, F (1, 174) = 5.39, p = .021, ηp 2 = .03. As shown in Figure 2, presentation attendees reported significantly greater use of tip-related strategies than controls (F (1, 174) = 5.84, p =.017, ηp 2 = .03), but, not surprisingly, the two groups did not differ in their reported use of tip-unrelated strategies, F (1, 174) = 0.05, p = .83, ηp 2 = .000.

Figure 2. Reported usage of tip-related and tip-unrelated study strategies by students who attended a "Study Smarter, Not Harder" presentation (dark bars) versus controls (white bars).
Presentation attendees reported a greater use of tip-related strategies but not tip-unrelated strategies compared to students in the control group.
When we further examined attendees' vs non-attendees' reported usage of each study strategy in a series of one-way ANOVAs, we found that the two groups reported using each of the tipunrelated strategies to a similar extent, but attendees reported that they utilize four of the ten tiprelated strategies more often than controls. The differences between attendees and non-attendees selfreported usage neared statistical significance for two of these strategies (Getting Plenty of Sleep and Skimming Readings before Class), and reached statistical significance for the other two (Having Fun with Friends and Eating Healthy Foods). See Table 1 for group means and standard deviations, as well as for inferential statistics from this series of one-way ANOVAs. .747 *significant difference between groups (p < .05) a near significant difference between groups (p = .079; p = .054)

Hypothesis 3: Study Tip Memorability, Incorporation into Study Approach, and Perceived Influence on Academic Success ("Study Smarter" Questionnaire)
To examine whether presentation attendees remembered the eight specific study tips that we highlighted during the presentation, incorporated these tips into their typical study approach, and believed that these tips positively influence their academic success, we ran three series of one-sample t-tests. We set our critical value at 2.5. For tip memorability, the critical value of 2.5 was half way between a response of "Vaguely Remember" and "Largely Remember." For tip incorporation, the critical value of 2.5 represented a response half way between "Somewhat" and "Quite a Bit." Finally, for perceived positive influence of the tips, the critical value of 2.5 indicated a response between "Somewhat" and "Quite a Bit" when participants indicated the extent to which they felt the study tip had a positive influence on their academic success. Figure 3 shows the mean ratings for the Lineweaver, Hall, Hilycord,   tips denoted with superscript a were remembered worse (near significant effect), whereas those denoted with superscript b were remembered significantly better than the tips on average. All of the study tips surpassed the critical value for utilization and perceived influence with no significant differences among the eight tips.
We included data from all 72 presentation attendees in the tip memorability analysis. On average, presentation attendees remembered five of the eight study tips at the 2.5 level (i.e., half way between "Vaguely Remember" and "Largely Remember") or above. The tips they significantly remembered were: Skim, Listen, Read, Repeat, Don't Rote Memorize, Quiz Yourself, If at First You Don't Succeed, Try Something Else, and It's Never Too Early. Average ratings for the other three tips, Pay Attention, Distribute Practice, and Take Care of Yourself, did not exceed the 2.5 critical value set in the analysis.
Participants had the option of not rating their incorporation of each tip if they felt that they did not remember it well enough to say. We excluded data from these participants in the incorporation analysis. Thus, the number of participants contributing data to each of these one-sample t-tests varied, as shown in Table 2. Based on the responses of the students who remembered the tip well enough to rate it, the average ratings for incorporation of the tips significantly surpassed the 2.5 critical value (i.e., half way between "Somewhat" and "Quite a Bit") for all eight of the study tips. See Figure 3 and Table 2. Thus, students who attended the "Study Smarter, Not Harder" presentation reported including all of the study tips that they remembered into their typical study approach.
Finally, for perceived positive influence of the tips, participants again did not rate any tip they felt they did not remember well enough, so the number of participants in each analysis varied (see Table 2). Based on responses from those students who rated the perceived influence of the tips, average ratings for all of the tips significantly surpassed the critical value (i.e., half way between "Somewhat" and "Quite a Bit"). See Figure 3 and Table 2 for details. Thus, participants felt that all of the study tips they remembered were positively influencing their academic success. To directly compare the memorability, incorporation, and perceived influence of the eight tips, we ran three repeated-measures ANOVAs. In all three analyses, the independent variable was Tip (with eight levels representing each of the eight tips), and the dependent variables across the three analyses were participants' ratings of (1) how well they remembered the tip, (2) the extent to which they incorporate the tip into their typical study approach, and (3) the extent to which they believe the tip positively influences their academic success.
For memorability, the main effect of Tip reached statistical significance, F (7, 65) = 3.57, p = .003, ηp 2 = .28. Tests of within-subjects contrasts compared the average rating associated with each tip to the average mean rating of all of the tips combined and indicated that If At First You Don't Succeed, Try Something Else (F (1, 71) = 9.48, p = .003, ηp 2 = .12) and It's Never Too Early (F (1, 71) = 4.78, p = .032, ηp 2 = .06) were both remembered significantly better than the tips on average. See Figure 3. Pay Attention was remembered less well than the tips on average, although this difference only neared significance, F (1, 71) = 3.48, p = .066, ηp 2 = .05.
The main effect of Tip did not reach significance for either the extent to which students incorporated the eight tips into their study habits (F (7, 32) = 1.89, p =.105, ηp 2 = .29) or the extent to which students felt the tips positively influence their academic success, F (7, 35) = 1.39, p = .240, ηp 2 = .22. Note that the effect size for each of these two repeated measures analyses were similar to the effect size for tip memorability. However, because cases were excluded listwise in these analyses, the number of participants contributing data was greatly reduced (Incorporation: n = 39; Influence: n = 42), leading to non-significant F-tests.
To determine whether the memorability, likelihood of incorporation, and perceived positive influence of the study tips withstood time, we compared the ratings of students who attended College 101 in 2013 (n = 13), College 101 in 2014 (n = 19), Become Your Best Bulldog in 2015 (n = 4) and Become Your Best Bulldog in 2016 (n = 36) in a series of three multivariate ANOVAs with ratings for each tip on the "Study Smarter" Questionnaire as the dependent variables. Table 3 summarizes the mean ratings from attendees of each of the four sessions. Because Session was an independent variable with four levels, we used Wilks' Lambda multivariate statistic to determine statistical significance in order to protect against potential violations of the homogeneity of treatment difference variances assumption.
Tip memorability ratings differed significantly based on which presentation session students attended, F (24, 177.52) = 2.97, p = .001, ηp 2 = .28. Significant differences were apparent in how well Lineweaver, Hall, Hilycord,  attendees at more recent vs more remote sessions remembered Pay Attention (p < .001), Skim, Listen, Read, Repeat (p < .001), Don't Rote Memorize (p = .046), and It's Never Too Early (p = .028). For Pay Attention, the students who had attended the most recent session (just three months prior to the study) reported remembering the tip significantly better than those who had attended any of the sessions that had occurred more than a year earlier (all ps < .001). The students who attended the most recent session also reported remembering Skim, Listen, Read, Repeat and It's Never Too Early better than both the students who attended College 101 in 2013 (both ps < .042) and the students who attended College 101 in 2014 (both ps < .012), but not better than those who attended Become Your Best Bulldog in 2015 (both ps > .053). Finally, students who heard the presentation at the most recent Become Your Best Bulldog reported remembering Don't Rote Memorize significantly better than students who heard the presentation at College 101 in 2014 (p = .011), but not better than students who attended College 101 in 2013 (p = .052) or Become Your Best Bulldog in 2015, p =.55. No other group comparisons reached significance. The rated memorability of the other tips (Study A Little A Lot, Quiz Yourself, If At First You Don't Succeed, Try Something Else, and Take Care of Yourself) did not differ significantly based on which presentation students attended (all ps > .076).
The main effect of Session also reached significance for incorporation of the tips, F (24, 81.81) = 2.02, p = .01, ηp 2 = .36. The reported likelihood of incorporating Pay Attention (F (3, 35) = 5.54, p = .003, ηp 2 = .32), Skim, Listen, Read, Repeat (F (3, 35) = 3.10, p = .039, ηp 2 = .21), Quiz Yourself (F (3, 35) = 2.89, p = .049, ηp 2 = .20) and It's Never Too Early (F (3, 35) = 3.36, p = .030, ηp 2 = .22) varied based on which presentation the students attended. The pattern of mean differences was more complicated for incorporation than it was for memorability, and it did not appear to be the case that students who attended the presentation most recently were the most likely to incorporate the tips into their study approach. See Table 3 for details. .27 (0.87) Note: Tips followed by a superscript showed significant differences between groups based on the session attended: 1 p < .05, 2 p < .01 or 3 p < .001. Means and standard deviations with different superscripts differed from one another in post hoc analyses with Tukey's test (all ps < .05).
Unlike tip memorability and incorporation, the perceived positive influence of the study tips did not differ depending on which session students attended, F (24, 90.51) = 1.25, p =.24, ηp 2 = .24. Thus, students felt that the tips were positively influencing their academic success to the same extent regardless of whether they learned the tips three months earlier or three years and three months earlier.

Discussion
We had two primary goals in conducting this study and writing this manuscript: (1) to briefly outline a study tips presentation that uses both evidence from the cognitive and educational psychology literatures as well as demonstrations to teach students how to study more effectively regardless of their discipline, and (2) to provide empirical evidence about whether or not this study tips presentation affects students' study habits. Our hope is that faculty teaching a wide range of types of courses will find themselves better informed about how to help their students study effectively  and may even undertake the task of explicitly teaching students how to maximize their studying (Marshak, 1984;Pressley, Goodchild, Fleet, Zajchowski, & Ellis, 1989). Of course, we are also happy to provide a copy of this presentation to students who may themselves be interested in learning how to "Study Smarter, Not Harder." Our study, which involves both pre-presentation and post-presentation data, as well as a comparison of responses from presentation attendees to those of a non-attendee control group, consistently supports the efficacy of our one-hour study tips presentation. At the same time, several of our findings, particularly those that compare the eight tips to each other, are somewhat unexpected.
In general, we found strong support for our presentation's effectiveness. Without directly asking students about the study tips, we found that students who attended our presentation changed their views regarding the effectiveness of different types of study strategies from immediately before the presentation to three months after the presentation. Interestingly, rather than enhancing their already positive perceptions of tip-related strategies, the presentation appeared to challenge students to think more critically and to alter misconceptions they may have had about less effective study approaches (Bjork et al., 2013;). As such, their endorsement of tip-unrelated strategies decreased significantly following the presentation relative to before it, while their beliefs about the effectiveness of tip-related strategies remained stable. It is important to note that this fine-tuning of students' understanding of effective study strategies also corresponded with their transition from high school to college since most of them attended the "Study Smarter, Not Harder" presentation as part of their first-year orientation to the university. Thus, students may have had opportunities to put into practice and to gather their own evidence about what Lineweaver, Hall, Hilycord, (September, October, and November) that intervened between the presentation and their postpresentation assessments. This may account for the finding that students reported using both types of study strategies (those related to and those unrelated to the study tips) significantly more frequently after the presentation than before. Facing the increased rigor of the college classroom may have inspired students to study more ("study harder"), utilizing a wider range of approaches than they had previously. At the same time, after those three months, they were "smarter" in that they recognized that not all approaches are equally effective.
We documented more evidence that attendance at our presentation helps students "study smarter" when we compared the study habits of students who attended our presentation to those of students who had not. Although both attendees and non-attendees perceived tip-related strategies to be more effective than tip-unrelated strategies overall, the students who attended our presentation reported actually using the tip-related strategies more often than their non-attending peers. These findings are particularly noteworthy because the students who served as non-attendee controls in this study were all enrolled in Psychology courses, with some of them being upper-level Psychology majors who may have learned about some of the tips and the empirical evidence to support them in one or more of their prior Psychology classes. It seems likely that we would have found even larger differences between attendees and controls in these analyses if we had included a more academically diverse and general student population as a comparison group, such that they were more similar to our attendees in background and familiarity with psychological principles. Despite this disadvantage, "Study Smarter" informed students indicated that they incorporate Sleep Tight, Skim Listen Read Repeat, Have Fun, and Eat Right into their study approach to a greater extent than students who did not have the benefit of attending the presentation. This is particularly fascinating as three of these four study strategies relate to the Take Care of Yourself portion of the "Study Smarter" presentation. At the same time, when we asked attendees what they remembered from the presentation, they did not report a strong memory for the Take Care of Yourself tip-this was one of the three tips that was not remembered significantly well. Although future research would be necessary to replicate or to more clearly elucidate possible reasons for these seemingly contradictory findings, one possible explanation is that students learned the importance of taking care of themselves and came to view self-care as an important part of their academic success through their attendance at the presentation, but they did not explicitly recall learning it in that context.
Relatedly, we were surprised that students only remembered five of the eight study tips significantly well. We were also surprised at which of the tips failed to surpass this level of memorability. Students did not remember Pay Attention, the first study tip, and the tip we spend the most time discussing during the presentation, particularly well, with its memorability also decreasing significantly within a year after the presentation. We support this tip with both empirical evidence and with a demonstration that our audiences appear to enjoy. We wonder whether its position as first in the presentation or the extensive amount of time we spend on it undermines its effectiveness. Alternately, students may choose not to remember this tip because they resist the strong recommendation that they eliminate distractions and avoid dividing their attention while studying or because they feel that they are already paying attention fully since they were academically successful in high school, although it is likely that they are not avoiding distractions (Clay, 2009;. Students also did not report remembering Study a Little a Lot (the tip that encourages them to distribute their studying across time) or Take Care of Yourself (but see the previous paragraph) particularly well. Perhaps these three study tips are considered common sense or have been emphasized to students repeatedly by parents and teachers in the past, causing students to feel that they were already well informed on these suggestions prior to attending our presentation. This may have led them to pay less attention to these tips during the presentation, to share them less Lineweaver, Hall, Hilycord, and Vitelli Journal of the Scholarship of Teaching and Learning, Vol. 19, No. 2, March 2019. josotl.indiana.edu with others after the presentation (reducing their likelihood of being solidly encoded), or to fail to attribute them specifically to the presentation, resulting in less long-term retention and recognition of these tips as being from the "Study Smarter, Not Harder" presentation. Of course, whether or not students report remembering the tips is much less important than whether they use the study strategies, as it is the strategies that students use that predict their academic success (Bartoszewski & Gurung, 2015;Gurung, et al., 2012). Thus, even though students may not explicitly remember Take Care of Yourself as one of the tips from our presentation, our results show that they do view sleeping well, eating right, and having fun as important aspects of their academic approach, suggesting that they are "Studying Smarter" even if they do not know why.
In contrast to the tips that students did not remember well, students did explicitly recall five of the eight tips. Three of these five tips are very concrete: Don't Rote Memorize, Skim Listen Read Repeat, and Quiz Yourself. These may also represent new recommendations rather than simply reiterating what students have learned previously about studying. For example, few teachers and faculty teach students to use self-quizzing as a more effective study approach than re-reviewing material despite its well-established superiority for promoting learning . Two of these three tips (Skim Listen Read Repeat and Don't Rote Memorize) also involved fairly straightforward and short demonstrations, which could have contributed to their memorability (although Pay Attention also involved a demonstration, but students did not remember it well). In fact, we use an example from  to demonstrate the importance of putting new information into context during the Skim, Listen, Read, Repeat portion of the study tips presentation. One of the authors (T.L) experienced this demonstration in her Developmental Psychology class in 1990 and remembered it when it was time to create this presentation in 2010 (20 years later). Likewise, we were not surprised that students remembered a humorous YouTube video associated with It's Never Too Early called "I am worried about my grade" ( http://bit.ly/1PqS6Ho). We were, however, not expecting students to remember If At First You Don't Succeed, Try Something Else as well as they remembered other tips in the presentation. In actuality, students remembered this tip significantly better than the eight tips on average. This is a tip that we cover very briefly towards the end of the presentation, offering neither empirical evidence nor a demonstration in support of it. Perhaps this tip sticks with students because of its catchy and unexpected ending to a common phrase.
Finally, as further support for our presentation, students reported incorporating all of the tips they remembered into their study approach, and they felt that the tips they remembered contribute positively to their academic success. Students' positive perceptions of the tips were consistent across time, remaining significant more than three years after students attended the presentation. Thus, teaching students how to study, especially early in their collegiate career, has the potential to affect their academic success positively across their entire college trajectory. This suggests that providing instruction on how to effectively study may be most valuable during students' transition from high school to college when they have the most time to put those strategies to use. This may also be a time when students are most open to new ideas about studying as they prepare for the new academic demands placed on them in the college environment. Future studies would be necessary, though, to determine whether these speculations are correct and whether empirical evidence can support them.
Although our results reinforce the efficacy of our presentation and support our hypotheses, our study has some limitations that suggest directions for future research. First, our study only included college students as participants. We have not presented our study tips to students at other levels, and, thus, we do not know whether high school or even middle school students would benefit in the same way from explicit instruction into how to study more effectively. The only other study we could find in the literature that examined an intervention designed to teach students test-taking strategies in order to improve their performance (Beidel et al., 1999) was conducted with elementary Lineweaver, Hall, Hilycord, and Vitelli Journal of the Scholarship of Teaching and Learning, Vol. 19, No. 2, March 2019. josotl.indiana.edu students. Thus, there is a real gap in the literature with regards to the potential to improve the performance of students at other academic levels through study approach instruction.
A second limitation of our study is that our sample was self-selected. The students who attended the "Study Smarter, Not Harder" presentation elected to spend 60 minutes learning how to study more effectively rather than choosing to spend that time in another way. These students are unlikely to be representative of the full range of college students. In fact, it is likely that these students are more motivated than average and may, perhaps, be starting at a more advanced academic level than the general college student population. Thus, we cannot determine how struggling or remedial students might respond to this type of presentation. In fact, it is possible that the students who could gain the most from our presentation may be less likely to attend this type of session, especially given that struggling students may be overconfident in the study strategies they utilize (Bjork, et al., 2013;Hacker, et al., 2000). A future study either with a more broadly representative population or specifically targeting struggling students would help determine whether less academically successful students are able to gain as much, or even more, from this type of instruction in study skills.
Third, our study included a control group, but the control group did not experience any form of intervention, and we were not able to assign students to either the control group or the study skills intervention group due to our research design. Future studies could use a more experimental approach, randomly assigning students to either attend or not attend (wait list control group) the presentation or randomly assigning students either to the study skills intervention or to a comparison intervention such as training focused on time management skills.
Finally, we do not have any objective data regarding students' academic success. Thus, we cannot determine whether students who attend our presentation are actually more academically successful than those who do not. Chen et al. (2017) recently demonstrated that students who are more self-reflective about their approach to studying perform better on examinations. Thus, the "Study Smarter, Not Harder" presentation has the potential to improve students' performance both by making them more thoughtful about their studying and by informing them about which strategies are most likely to lead to success. However, only through collecting additional future data about students' GPA or performance in particular classes could we directly assess whether students' perceptions of how the study tips we share with them relate to their academic success are, indeed, accurate.
Despite these limitations, our study offers empirical evidence that college students remember, utilize, and believe their academic success benefits from explicit instruction in how to "Study Smarter, Not Harder." Attending an hour-long study tip presentation also gave students a better understanding of effective versus less effective study approaches. We encourage faculty to use the resources provided here and to request additional resources from us, if desired, in order to help their own students "Study Smarter, Not Harder." 3. Don't Rote Memorize: Try to make sense of the information you are learning. Relate it to everyday life and to personal experiences. Make sure you understand what you are learning and how it all fits together. If you don't understand it, ask your professor to explain it again during class or during office hours.
4. Study A Little A Lot: Study every subject several times a week. Review your notes the evening after each class or the next day rather than waiting until just before a quiz or examination to review what you have learned. Studying for an exam will take much less time if you have reviewed your notes several times in the interim than if you wait and cram. You may want to create a study schedule that sets aside specific times during the week for reviewing your notes from each class. 5. Quiz Yourself: When reviewing your notes, don't just reread them. Be sure to quiz yourself. Flashcards are one method of achieving this, but you can also simply look away from your notes and practice recalling the information on your own. Explain key concepts to your roommate or parents. If you can't do it, you are not yet ready for the exam. Keep quizzing and explaining until you are sure you can recall the information without relying on your book or notes.
6. If At First You Don't Succeed, Try Something Else: If you are not doing as well in a class as you would like, get help. Consult with the professor, use the learning resource center, utilize tutoring that is available, and find other students who are having greater success and ask them for tips. If you keep approaching the class in the same way, you will likely get the same result. Find another approach. 7. It Is Never Too Early: Semesters go fast. Do not fall behind. Keep up with the readings and the written assignments, and, if you are not able to, reprioritize your time. Once you fall behind, your work in all of your classes will start to suffer.
8. Take Care of Yourself: Sleep. Eat. Have fun. Taking care of yourself physically, mentally, and emotionally is even more essential to your success in college as studying is.
Reprinted from Study Smarter, Not Harder by Mandy Gingerich and Tara Lineweaver © 2010 All rights reserved. Changing strategies if yours are not working 1 2 3 4 5

Appendix 3. "Study Smarter" Questionnaire
The following questions ask about your memory for and perception of each of the study tips that were presented as part of the "Study Smarter, Not Harder" session you attended.
For each tip, please indicate the extent to which you remember the tip and the demonstrations or evidence that accompanied it.

Do Not Remember
Vaguely Remember Largely Remember Vividly Remember "Pay Attention" (with finding the city names in red and raising your hand to a sound multitasking demonstration) Finally, for each tip, please indicate the extent to which you feel it has positively influenced your academic success as a college student.