Pre-and In-Service Teachers ’ Perceptions of Learning through Research Workshops in a Hispanic-Serving Institution in USA : A Review of Two Years ’ Accumulative Data

The purpose of the study was to report the analysis results of two-year accumulative data from Research Academy workshops held for pre-service and in-service teachers in a southern state Hispanic-Serving Institution. Graduate students’ perceptions of learning through these professional development workshops were reported. Statistical analyses were performed. The study found consistent and significant gains of participants’ confidence in educational research skills and knowledge. Future directions, limitations, and implications were discussed.


Introduction
Professional development for teachers has been recognized as one of the core elements to improve quality of teaching and learning for students (Borko & Putman, 1995;Darling-Hammond, 1992;Darling-Hammond, 1997;Thompson & Zeuli, 1999).Advancing knowledge and skills is considered a main purpose of education-related professional development (Garet, Porter, Desimone, Birman, & Yoon, 2001;Shulman & Sparks, 1992).A group of faculty members with different educational backgrounds in a southern state university in the United States with 93% of its students being Hispanic established the College of Education Research Council (COERC) in Fall

Effective Professional Development
Some research studies examine the effectiveness of professional development workshops or seminars (e.g., Boyle, While, & Boyle, 2004;Lu, Ward, Overton, & Shin, 2014).Several research studies suggest that professional development mainly emphasizes the subject matter content, curriculum, or strategy use to teach students in an effective way and consequently to increase students' achievement (Cohen & Hill, 2000;Kennedy, 1998).Penuel, Fishman, Yamaguchi, and Gallagher (2007) examined the effects of different characteristics of a science program on teachers' perception about knowledge gains and ability to implement the program in the classroom.The results from a survey of 454 teachers highlighted the importance of coherent professional development experiences for successful teacher learning and program implementation.The study also underpinned that combined effects of time for teachers to implement the science program and technical support from a school or a district contributed to high-quality professional development and promoted the implementation of the science program.
Content focus has been argued to be the most influential factor leading to effective professional development (e.g., Banilower, Heck, & Weiss, 2007;Cohen & Hill, 2000;Cohen, 1990).Many studies, based on various methodologies such as case studies, quasi-experiments, longitudinal studies, and experimental designs, clearly supported the significance of content focus (Banilower, Heck, & Weiss, 2007;Cohen & Hill, 2000;Cohen, 1990;Desimone, Porter, Garet, Yoon, & Birman, 2002;Garet et al., 2001).In addition to content focus, Loucks-Horsley and Matsumoto (1999) considered immersion in inquiry as one of the essential elements for effective professional development, in which participants engage in an activity that they are likely to practice with their own students.
Regarding effective activities that contribute to the success of professional development, previous research studies have found and identified the following: participant engagement in structured and collaborative interactions around topics, coaching or mentoring arrangements in which participants have rich opportunities to work with more experienced teachers, facilitated networks between teachers and groups to explore and discuss topics of interest, sharing information, and pursuing common goals (Darling-Hammond, 1992;Hargreaves & Fullan, 1992;Lieberman, 1996;Loucks-Horsley, Styles, & Hewson, 1996;Richardson, 1996).
The effective activities or components of professional development may be associated with time involvement.Some researchers investigated if professional development sessions longer in duration were more effective since they allowed teachers to have more practice time and more time to reflect upon their teaching as part of an ongoing process (e.g., Shields, Marsh, & Adelman, 1998;Weiss, Montgomery, Ridgway, & Bond, 1998;Weiss, Banilower, & Shimkus, 2004).As a major characteristic of professional development, a handful of research studies examined the duration or length of the professional development and its association with the depth of teaching and teacher changes (Shields, Marsh, & Adelman, 1998;Weiss, Montgomery, Ridgway, & Bond, 1998;Weiss, Banilower, & Shimkus, 2004).Brinkerhoff (2006) examined the effects of long-duration professional development designed to develop teachers' technology skills.He analyzed data from the surveys of participants' self-assessment technology skills and self-efficacy, beliefs and feelings on the use of technology in classrooms, technology integration into the classroom, and teacher interviews.The professional development was designed to support teachers' mastery of technology skills and integration practices across two academic years and this long-duration academy allowed teachers to overcome the issue of insufficient time to acquire a skill while applying it in classrooms.The results from data analysis revealed that the participants significantly gained technology skills and computer self-efficacy, with little change to technology integration beliefs and practice.The article identified two major contributing factors to the success of the professional development: (1) the extended nature of the academy program, and (2) teacher participants volunteered to participate in the academic workshop program based on their own needs or interests.
Taken together, the core elements playing important roles in determining the effectiveness of professional development include the following: content focus (Cohen & Hill, 2000;Cohen, 1990;Garet et al., 2001), active learning (Banilower & Shimkus, 2004;Borko, 2004;Darling-Hammond, 1997), coherence, the extent to which knowledge gains from professional development are consistent with teachers' knowledge and beliefs (Elmore & Burney, 1996), duration including both the span of time over which professional development is spread and the amount of time spent in the professional development (Cohen & Hill, 2000;Fullan, 1993;Guskey, 2003;Supovitz & Turner, 2000), and collective participation in which participation from the same school, grade, or department might generate potential interaction and discourse among teachers as a powerful form of teacher learning (Banilower & Shimkus, 2004;Borko, 2004;Desimone, 2003;Fullan, 1993).

The Role of Research in Teacher Education
However, the role of research in teacher education is less explored while addressing the effectiveness of professional development focusing on subject matters such as science or mathematics or core elements for effective professional development.The main purpose of promoting research is to sharpen and maintain teachers' insight and professional curiosity (Rudduck, 1985).Effective teaching involves not only the ability to implement new strategies, develop effective tasks, and provide assessment tools tailored to different students' learning style but also the ability to apply and adopt new evidence-based research in specific classroom contexts (Stafford, 2006).This level of teacher quality cannot be accomplished unless teachers are knowledgeable about new research in their subject matter and new research methodologies and are eventually trained to be teacher researchers themselves for enhancing teaching practice.
Although research is widely recognized as an essential educational component, there remains a concern about how to efficiently implement research in educational settings.For example, research components and skills should be integrated into research within the teaching contexts for effectively creating an environment in which research knowledge or skills are needed to complete their immediate tasks (Lovat, Davies, & Plotnikoff, 1995).Despite several previous studies about the significance and subsequent influential factors of professional development, there is a relative paucity of studies designed to implement research components in the professional development, and let alone to evaluate the effects of long-durational research knowledge-and-skill integration workshops for pre-service or in-service teachers.
The paper augmented current literature in teacher's research-oriented professional development by describing a series of professional development workshops focusing on infusing research knowledge and skills into teaching context.The paper further explored the participants' perceptions of knowledge and confidence gains over the series of professional development and the relations between their achievement and other influential variables which might be related to their learning outcomes.

Research Academy Workshops
The Research Academy workshops were described and reported by Lu, Ward, Overton, and Shin (2014).The first two years of workshop series consisted of 13 workshops with various topics and details (Table 1).The APA/Academic writing workshop contained the most valid data points, with participants (N=64) while review of statistics has the fewest participants (N=19).The second years' workshops' order was changed based on the first year's empirical data, in which a new framework was formed with the order of APA writing, journal reading, research method, IRB process, qualitative research method, and proposal writing.The highest number of the participants in APA writing workshop indicated the graduate students' and novice researchers' urgent need for APA format, academic writing, and the basic structure of academic paper.Therefore this workshop was offered first in the second year.In a similar vein, the review of statistics session was provisionally eliminated from the workshop series due to the issues with time conflict and the participant's responses over the topic.-How to target a conference -Familiarizing with proposal templates/criteria -To write an proposal for a targeted conference The workshop series were designed and structured based on student needs and their calls for facilitating and internalizing their academic writing process in their real life context.The following figure (Figure 1) was set up at the second year and it represents the core contents and structure of the workshop series.Though research or academic writing is not considered a linear process, novice researchers or teachers can easily follow these sequences of workshops without losing their research foci.
Figure 1.Flow chart of the workshop series

Research Questions
With two years' accumulative quantitative data on all Research Academy's professional development workshops, the investigators asked the following Research Questions (RQs): 1) Are there any significant mean differences in participants' overall confidence level before and after each workshop and over the two years?
2) Are there any significant mean differences in participants' perceptions before and after each workshop and over the two years?
3) To what extent do the participant's overall confidence and other relevant variables impact their achievement test scores?

Participants
The total number of the participants was 492 consisting of COE graduate students who are currently enrolled in the southern state university and teachers or practitioners who already graduated from the university.These graduate students are all pre-and in-service teachers.The majority of the participants were females (female: 67%, male: 33%) and a substantial number of the participants were repeat attendees of the workshops.Their average age and years of teaching experience were 36.6 and 7.57, respectively.The participants' language background reflects on the strong Hispanic influence, showing approximately 47% with Spanish as their L1 while they also indicated either Spanish or bilingual preferences in reading and writing (16%) and in speaking (32%).Table 2 shows the demographics of participants participated in the study.

Instruments and Data Collection Procedures
Data collected from each workshop included demographic information, pre-and-post perception survey, pre-and-post content assessment designed by the council committee members, and satisfaction survey (See Lu, Ward, Overton, & Shin, 2014, for detailed description of the instruments and discussion regarding the effectiveness of the workshops by examining pre-and-post content assessment and satisfaction surveys of the workshops).Data collection procedures were all similar and instruments were given in order throughout the workshops: demographic data, pre-perception survey, pre-content assessment before the workshop session starts, and post-perception survey, post-content assessment and satisfaction survey after completing all the workshop sessions including PPT presentation, hands-on activities, and students-committee members' interaction sessions.The perception survey consists of a series of questions about specific skills and knowledge which were essentially covered or required to gain during a workshop.It also consists of an overall confidence rating in which the participants indicated the extent of agreement on a Likert-scale.

Data Analysis
In addition to exploratory data analysis for descriptive statistics, RQ1 and RQ2 were tested through a series of paired t-tests for within-subject effects and one-way ANOVA to compare pre-and post-tests and between year 1 and year 2 data.Regarding RQ3, an exploratory multiple regression analysis was utilized to identity which elements mainly influenced the performance of the participants (i.e., age, teaching experience, years in a program, pre-and-post overall confidence, and confidence increase, etc.).Research methods Qualitative

Overall Confidence
Participants were asked to rate their own confidence level before and after the workshops.Table 3 shows their ratings and mean comparison results.Overall there was significant increase in participants' level of confidence in Journal Reading, APA writing (first year), Research Methods (marginal sig.), Qualitative Research, Human Subject Research/IRB, and Preparing Proposals.Statistics Review workshop did not yield confidence increase.
Year comparisons indicated participants increased their confidence level in Preparing Proposals and decreased their confidence level in APA writing.The investigators examined each items on the survey used for participants' perceptions of APA knowledge and skills in the APA workshop.Table 4 and Figure 2 show data analysis results.It is shown that there were significant increases from pre-to post-surveys.The investigators examined each items used for participants' perceptions regarding Human Subject Research/IRB knowledge and skills in the IRB workshop.Table 5 and Figure 3 show data analysis results.It is clearly shown that there were significant increases from pre-to post-surveys.The investigators examined each item used for participants' perceptions of their proposal preparation ability and skills in the Proposal workshop.Table 6 and Figure 4 show data analysis results.It is clearly shown that there were significant increases from pre-to post-surveys.

Results of Achievement Scores (Pre-and -Posttests)
Table 7 shows the pre-and post-test results for participant achievement assessment in each workshop.From pretests to posttests, there were significant gains in their knowledge and skills in all workshops.Effect sizes were all medium to large.Year comparison shows participants performed better in year 2 in Research Methods and Qualitative Research.Note.The results of year comparison for the journal reading and APA workshops are not available due to different task types in year 1 and year 2.

Results of Multiple Regression Analysis for APA Workshop
The investigators conducted exploratory multiple regression analysis on the APA workshop, which contained the most valid data points, and the initial results were shown in Table 8.As teaching experience and overall confidence were significant predictors of achievement scores, the investigators re-ran the analysis for the modified model with step-wise method of entry for the significant predictors.Diagnostics analysis of collinearity was conducted and all Tolerance values were greater than .93,indicating high independency of the model's predictors.Table 9 and Figure 5 show the modified model and the regression equation.The model yielded an R=.547 (R 2 =.299; SE=3.83), and a significant model fit (F(2, 60)=15.771,MS Regression =231.86,p<.001).Therefore, about 30 % of the variance in achievement scores could be explained by the model with predictors of teaching experience and overall confidence.

General Discussion
Several points are worth mentioning from this two-year accumulative data.(A) All attendees of the workshops were graduate students from different graduate programs in the College of Education in this HSI with some years of teaching experience.There were more female than male participants, and most of the participants' first language (L1) is either Spanish or English.Many of the attendees had experience participating in previous Research Academy workshops (Table 2).(B) Most workshops successfully increased participants' overall confidence level, such as the journal reading, APA writing (1 st year), research methods (marginal sig.), qualitative research, IRB, and proposal writing workshops (Table 3).The medium to large effect sizes indicated the practical significance of these workshops.It is reasonable to assume that those who had already learned APA style/academic writing in their first year of graduate program did not find the APA workshop particularly helpful in increasing their confidence level in APA writing.In addition, the first year's statistics workshop did not yield significant confidence gains, which showed many students' fear of statistics and a general need for increasing their confidence in statistics and strengthening statistics knowledge and skills for graduate students in this HSI.(C) When examining participants' perception of each item from the APA workshop, the investigators found that there were significant increases in participants' ability to detect major sources of errors in APA, write in APA style, reference list, find resources, and proofreading skills (Table 4 & Figure 2).(D) When examining participants' perception of each item from the IRB workshop, the investigators found that there were significant increases in participants' understanding of major concepts, IRB protocols, human subject protection, IRB procedures, and finding resources for IRB (Table 5 & Figure 3).(E) When examining participants' perception of each item from the Proposal Writing workshop, the investigators found that there were significant increases in participants' ability to determine components of proposal writing and components of field-related proposal writing, differentiate between good and bad proposals, write a proposal, and seek conference information and resources (Table 6 & Figure 4).(F) The significant increase of participants' actual achievement test scores not only demonstrated their gains in research knowledge and skills through these workshops, but also provided evidence to explain their increase in confidence and their positive perception of learning (Table 7).Therefore, Latina/o graduate students at this HSI seemed to provide consistent and correct perceptions of their learning in regard to their actual learning achievements.(G) The modified multiple regression model revealed that teaching experience and overall confidence were significant predictors of achievement scores (Table 9).Those with fewer years of teaching experience tended to obtain better achievement scores in APA and those with higher confidence level tended to perform better in achievement tests (Table 9 & Figure 5).

Implications
Shifting from content-based, curriculum-based or strategy implementation activities (Cohen & Hill, 2000;Kennedy, 1998) to a more research-based approach, which placed emphasis on how to conduct classroom-related or field-based research, the paper demonstrated a new perspective that might be needed to examine the effectiveness of professional development designed to improve teachers' conceptual understanding of research knowledge and skills.
As these graduate students had provided positive perceptions of their learning research-related knowledge and skills through these workshops, the results of these effective workshops could potentially benefit not only HSIs but also other higher education institutions that emphasize teachers' research ability or educating future teacher researchers.Researchers and practitioners may utilize the study's model for implementing workshops for teachers to learn research-related knowledge and skills and to increase their confidence.
The outcome from the regression analysis could be considered an expected result, given the nature of learner's perception on the workshop.It is assumed that participants' confidence level from participating in a workshop positively influenced and thus predicted their achievement performance (posttest content assessment).Indeed, the higher the overall confidence level a participant had, the better the achievement score was.In addition, the less the participants' teaching experience was, the higher their achievement scores were.
The results implied that the pre-service or novice teachers were more capable of acquiring and adopting new knowledge regarding specific contents from the APA workshop.It might also reflect that pre-service or novice teachers are more urged to conduct time-efficient and cost-effective research in their field.Under the circumstance, the workshop series provided valuable opportunities for them to learn field-related research techniques and skills.

Limitations of the Study
Since some measurements used in the 2 nd years' workshops were improved or modified and thus different from the 1 st years', there were some discrepancies that did not allow investigators to make meaningful comparisons.Therefore, it would be more ideal to design and implement consistent measurements for longitudinal data analysis.In addition, though some participants had participated in more than two workshops or in both years, some participants were new and thus were totally different from those who participated in the first years' workshops.This limited some possible repeated-measures continuing data analysis for between year comparisons though the study mainly focused on the structure, perception, and effectiveness of the workshops.Last, though the study had reported many statistically and practically significant results, obtaining more valid data would help further analyses, such as a more comprehensive multiple regression analysis.

Future Directions for Research
Researchers may consider testing participants' long-term memory by assessing their retention after one week or one month of the workshop.As achievement scores were all tested at the end of each workshop, memories learners obtained might not be firmly stored in their long-term storage.With another round of test performed after one week or one month, the researchers will be able to investigate if these workshops also promoted long-term memories on research-related knowledge and skills.In addition, researchers may consider conducting more analyses in inferential statistics for other workshops other than the APA/academic writing workshop in the future when more valid datasets are collected.Unlike Lawless and Pellegrino (2007) in which they focus more on professional development in instructional technologies, future research may incorporate instructional technologies into teacher's PD training and investigate possible differences that may occur between these different delivery methods (e.g., Lawless & Pellegrino, 2007).Yet, similar to Lawless and Pellegrino's (2007) reporting knowns and unknowns, future studies may investigate what has not been known in teachers' professional development training.Last but not least, to probe these pre-service and in-service teachers' thoughts and opinions, qualitative inquiries, such as focus groups or interviews, may also help uncover what is needed and what aspects of the workshops can be improved or enhanced.

Conclusion
The study augmented current literature in teacher's professional development in research by examining the two-year accumulative data from Research Academy workshops held for mainly pre-service and in-service teachers in a southern state Hispanic-Serving Institution in USA.They study found that participating graduate students' perceptions of learning through this specific type of professional development workshops, namely the research workshops, were positive and effective.The study also found consistent and significant gains of confidence level of participants in research skills and knowledge in education.Mean comparisons from the two-year data were reported and the journal reading, APA writing, qualitative research, IRB/human subjects protection, and proposal writing workshops were among the most successful ones in increasing participants' confidence as well as learning outcomes.

Figure
Figure 5. Regression equation

Table 1 .
The order of the workshops in 2012-2014

Table 2 .
Summary of demographic data

Table 3 .
The summary of overall confidence results

Table 7 .
Summary of pre-and post-test results