Articles Effects of Modular Technology Education on Junior High Students ’ Achievement Scores

Background In the quest to improve public schools, an education in basic technological concepts and systems, or “technological literacy,” has been viewed as an important part of a school curriculum (Dugger & Yung, 1995, pp. 7-8). Proponents of technology education have claimed that technological knowledge may be critical to the future needs of all students in the United States. However, proponents of technology education have gone beyond technological literacy to cite even greater benefits for students educated in technology. Many technology educators have claimed that instruction in technological concepts is crucial in fully understanding the concepts in other academic subjects, particularly science and mathematics. These educators have argued that technology education allows students to apply the information that is received in other subjects to real-world situations, thereby increasing their comprehension of the subject matter (Dugger & Yung, 1995; LaPorte & Sanders, 1995; Lewis, 1999; Moss 1999). They also claim that technology education helps students to build and reinforce new patterns of knowledge that make better use of the information that is received in the classroom (Loepp, 1999). Repeatedly, experts in the technology education field have argued that technology education has the ability to strengthen students' achievement in other subjects by providing realistic and practical situations in which students can apply science, mathematics, and other skills. Some commercial curriculum vendors have adopted this argument as well, asserting that a given curriculum will help to boost student performance in mathematics, science, reading, or other areas. As high-stakes testing in basic skills continues to be implemented, it can be expected that educators and administrators will look more closely at any curriculum claiming achievement gains in core academic subjects. Technology education, then, may come under increased scrutiny for its purported benefits to core academic subjects. ____________________


Purpose and Research Questions
Despite anecdotal claims of achievement gains, we do not yet know what impact technology education has on student achievement in other academic subjects.Although some technology educators have provided personal accounts of success, there has been limited research on the effects of technology education on achievement in other academic subjects (Lewis, 1999).Most research to date has only focused on the effects of technology on achievement in physical science (e.g., Dugger & Johnson, 1992;Scarborough & White, 1994), perhaps because the conceptual relationships between technology and physical science tend to be closer than those to other subjects.While this research is certainly warranted, there are other core academic subjects that also may be affected by technology education.Reading and mathematics in particular are two core subjects that receive very little attention in the research surrounding technology education.The range of interaction between technology and other subjects has yet to be fully explored, and the effects of technology education on achievement in other subjects have not been thoroughly investigated (Zuga 1995).
The purpose of this study was to examine whether technology education improves students' achievement scores in the five areas of reading, language arts, mathematics, science, and social studies.Analysis of pre-and post-test scores allowed the analysis and discussion of the following research questions: 1. Do seventh grade students who have participated in one trimester of modular technology education show greater improvement on a test of achievement compared to similar students who did not participate in technology education?2. Do eighth grade students who have participated in one trimester of modular technology education show greater improvement on a test of achievement compared to similar students who did not participate in technology education?
Related Literature Technology education may continue as a separate subject or one that is integrated with other school subjects.Either way, the content covered by technology education necessarily overlaps that of other disciplines.Dugger (1993) examined the various fields to which technology is related -science, mathematics, and engineering -and found that technology had strong parallels with all of these fields.Some educators look beyond the interactions with mathematics, science, and engineering and expect technology to come into its own as a full academic subject with established parallels to other academic subjects (International Technology Education Association, 2000).It might be expected that a large body of research would define these links, and that some of this research would focus on the observed interaction of these subjects at the elementary and secondary levels.However, research on connections between technology and other academic subjects is sparse.Liedtke (1999) and Zuga (1995) both commented on a lack of published research on the integration of technology with other subjects.Zuga (1995Zuga ( , 1999) ) further noted a lack of research on the effectiveness of technology education in terms of contributing to a general education.
Several authors have identified the connection between technology and other academic subjects as a necessary area of inquiry.Lewis (1999) wrote: If schooling is to have desired meaning for children, then the various elements of the curriculum must cohere.Lessons learned in one subject must be amplified in others.To take its place squarely in school curricula, technology education must establish itself not just in its own right, but crucially in relation to other subjects.Thus, the relationship of technology to other subjects in the curriculum is a fruitful area of inquiry.(p.49) Moss (1999) concurred, noting that "[Technology] supplies the functional context for demonstrating the applications and enriching the meaning of many abstract concepts taught in mathematics and the physical sciences" (p.23).

The Effects of Technology Education on Achievement Gain in Science
Experimental research concerning the interaction of technology and other subjects can be broadly divided into two types: examination of the effects of technology education on other academic subjects, and examination of the effects of other academic subjects on technology education.The effects of interaction can be measured in multiple ways, including affects on achievement, motivation, or other factors.Unfortunately, most of these interactions have not been studied in published literature.The existing literature has almost exclusively focused on the effects of technology education on achievement in other subjects, particularly physical science.
Much of the existing research on the interaction between technology and physical science concerned an integrated physics-technology program called "Principles of Technology."The purpose of these studies was usually to determine whether the Principles of Technology curriculum was equivalent to a conventional physics course.The results of these studies were mixed.Dugger and Johnson (1992) and Hall (1989) found significant achievement gains in physics for students who participated in technology courses.Lewis (1990) and Nicholson (1991) did not.
Dugger and Johnson's 1992 study examined achievement gain for students enrolled in the Principles of Technology course in 15 Iowa high schools.Students were in three groups, those in a Principles of Technology course, a conventional physics course, and no physical science course.Each group was tested for pretest to posttest gain on a test of concepts specific to the Principles of Technology curriculum.The Dugger and Johnson study found significant achievement gain in the groups enrolled in both Principles of Technology and in conventional physics.However, the instrument used to measure achievement gain was written to test the specific curriculum of the Principles of Technology course.In essence, the test was a measure of the skills taught in the class and it is not surprising that the group enrolled in Principles of Technology performed considerably higher than the other group.Dugger and Johnson suggested further research comparing students on a standard test of physics achievement.
Two notable studies in fact used standardized physics tests and found different results.Lewis (1990) also compared gains in physics achievement for students enrolled in either a Principles of Technology curriculum (n=226) or a standard physics curriculum (n=251).Lewis's instrument was based on 30 test questions drawn from a test of physics achievement developed by the American Association of Physics Teachers and reflected content coverage found in both Principles of Technology and physics courses.Lewis found that the Principles of Technology course was comparable to conventional physics courses, in the positive sense, because the achievement of both groups was similar.Nicholson (1991) similarly measured achievement on a standardized physics test for students enrolled in a Principles of Technology course (n=178).
Nicholson measured only posttest achievement and did not employ a control group of physics students.Like Lewis, Nicholson used a standardized assessment test published by the American Association of Physics Teachers.Unlike Lewis, Nicholson found that the Principles of Technology student achievement was not comparable to physics student achievement and further concluded that the instrument selected was not valid for assessment of students enrolled in the Principles of Technology curriculum.
In a study with a slightly different focus, Brusic (1991) examined gains in science achievement for fifth-grade students when technology activities were integrated with traditional science instruction.Brusic conducted a quasiexperimental study of fifth graders in which the treatment and control groups participated in a unit of science.She found no statistical difference in achievement gain between the two groups, but found that the treatment group exhibited greater curiosity regarding the content than the control group did.Brusic noted that incorporation of technology activities might bring potential benefits in increasing students' level of engagement without sacrificing achievement gains.
The mixed results of these studies are not surprising, considering the wide scope of technology curriculum being studied and the varied instruments being employed.Validity of the instrument appears to be a large factor in the results obtained by these studies.Dugger and Johnson, for example, used an instrument that was specifically aligned with the curriculum being studied.Their study reported significant results, where other studies with more generic assessment instruments did not.

The Effects of Technology Education on Achievement Gain in Mathematics and Language Arts
Rogers (1990) conducted a study that examined the relationship between participation in industrial arts education and performance in mathematics courses.Rogers recorded students' participation in specific areas of industrial arts coursework such as drafting and construction, and determined mathematics performance by course grades as reported in the National Longitudinal Survey -Youth Cohort.Rogers concluded that industrial arts education did not significantly enhance students' grades in mathematics, also noting that the contribution of the correlation to the overall mathematics grade was very smallless than one percent.Rogers' study focused on traditional industrial arts.Current developments in the area of technology education suggest the need for a study focused on updated technology curriculum.Ilott and Ilott (1988) studied the interaction of technology and language arts among a small group (n = 7) of elementary students participating in a summer technology enrichment program.The study examined language use in the transmission of verbal instructions among students participating in the enrichment program.The study was qualitative in design and did not measure changes in language usage or skill achievement as a result of experience with technology education.

Conclusions from the Review of Literature
It appears that most research efforts on the interaction of technology and other subjects have focused on the evaluation of specific curricula.Some of these studies have shown parity with existing curricula, and researchers have concluded that the new curriculum does as good of a job as the old one.Most of these studies have focused on physical science.
There are few published studies that specifically examine the potential of technology courses to enhance learning in other academic areas.Rogers found that technology enrollment correlated to lower math grades, but his study did not investigate causes that may lie beneath this correlation.Brusic found that technology activities in a fifth-grade classroom led to increased curiosity, but not to higher achievement.No research exploring the claim that modular technology education improves student achievement in other disciplines could be located.This current study attempted to contribute to the body of research by examining the effects of modular technology education on achievement gain in five different academic areas.

Methodology
The intent of this research study was to determine whether students who have participated in modular technology education show greater achievement gain in reading, language arts, mathematics, science, and social studies than students who have not participated in technology education.This study examined seventh and eighth grade students at one junior high school.A causalcomparative analysis was the most applicable to the research questions and was also useful in minimizing disruption to the normal activities of the selected school and students.Participating students were given a pre-test to measure achievement in each of the following five subject areas: reading, language arts, mathematics, science, and social studies.After the pre-test, each student was randomly assigned to one of three groups: those exposed to a full unit of technology education, a partial unit of technology education, or no unit of technology education.During the time that the groups were not participating in technology education, they participated in a physical education course.After these experiences, all groups again were tested on a post-test of achievement in the same five subject areas.

Instrumentation
The pretest and posttest were the TerraNova Performance Assessment published by CTB/McGraw-Hill.The TerraNova had the advantage of longstanding and thorough data on validity and reliability.The TerraNova has general content validity on par with the best achievement test batteries.The test was developed through numerous revisions with a specific focus on relevance to actual curricular practice (Monsaas, 2001;Nitko 2001).As Nitko noted: The developers [of the TerraNova test] began with a thorough analysis of curriculum guides from around the country, of statements of national and state goals and standards, and of textbook series.Efforts were made to align the test content with the NAEP (National Assessment of Educational Progress) and NCTM (National Council of Teachers of Mathematics) frameworks.Teachers, curriculum experts, and other educators reviewed the test specifications and test materials for appropriateness, fairness, and accuracy.
Reliability of the TerraNova test is also high.The individual subtests, as well as composite scores, have reliability coefficients "consistently in the .80sand .90s." (Monsaas, 2001).

Collection and Analysis of Data
Data for this study consisted of one independent variable -group membership -and ten dependent variables, which are the pre-and posttest scores in reading, language arts, mathematics, science, and social studies.Preand post-test scores in reading, language arts, mathematics, science, and social studies achievement were taken directly from the scaled scores reported on the TerraNova test.Scaling of scores is a technique that maps raw scores (number of items correct) onto a predefined scale with equal intervals.Scaled scores can be added or subtracted to calculate meaningful gains.
Statistical analysis of the data consisted of descriptive statistics and an analysis of covariance (ANCOVA) for each academic subject (reading, language arts, mathematics, science, and social studies).Posttest scores were used as the dependent variable, with pretest scores as the covariate and group membership as a fixed factor.

Population and Sample
The population for this study consisted of seventh and eighth grade students at a public junior high school in the Midwestern portion of the United States.The population was 201 seventh grade and 188 eighth grade students.This particular junior high school was selected because it had a modular technology education program that was well suited to the purposes of this study and because of the willingness of the faculty to participate in this study.The study was limited to seventh and eighth grade students to provide uniformity in age and school experience.Two treatment groups and one comparison group were formed from this population.All seventh and eighth grade students in the school were randomly placed in one of the three groups.All three groups took a pretest in September 2000 before the treatment began.The post-test was scheduled in the middle of the school year, in January 2002, so that each of the three groups experienced a full unit, a half unit, or no participation in a unit of technology education.

Treatment and Monitoring
The technology course used for the treatment in this study was a "modular" technology education laboratory consisting of twelve separate learning stations or "modules."Pairs of students were assigned to each module, spending approximately twelve days at one module and then rotating to another.During each class period, students were expected to follow written instructions and complete computer-based and hands-on activities.The classroom teacher conducted assessment using a mix of electronic, paper, and physical products.
Students rotated among the modules during the twelve-week trimester.Twelve weeks of instruction is 60 days, meaning that students could potentially complete five modules.Because of scheduling, special events, and class management activities, students completed four modules during the twelveweek trimester.
The modules used in the course were commercial packages supplied by a major commercial vendor.The vendor had data available that identified core skills in technology addressed by each of the modules.Table 1 shows some of the core skills published by the manufacturer.The manufacturer did not identify how the core skills were identified, or whether the core skills were taken from a reference source.The vendor provided data to correlate each core skill to the specific modules in which it was addressed.According to the vendor, all modules addressed the core skills in writing and reasoning, most modules addressed mathematics and three modules claimed to address core skills in reading.

Analysis of the Data
The analysis of data compared gain scores in the five subject areas and in the three treatment groups using an analysis of covariance (ANCOVA).The ANCOVA allowed the researchers to adjust for initial differences among groups and enabled the researchers to more precisely determine whether outcomes are due to the treatment effect or due to initial difference.Specifically, the pre-test scores were used as the covariate, and the post-test scores as the dependent variable.Group membership was then taken as the fixed factor to determine treatment differences.Since research in this area is still in the exploratory stages, the alpha criterion for significance was set at α = 0.10, corresponding to a 90% confidence level.

Results
The study used pre-test and post-test data from 308 students.At the time of the study, the school had a total 7 th and 8 th grade enrollment of 389 students.Data were not available for the pre-test, posttest, or both for the remaining 81 students.Scores were sorted by treatment group, and mean pre-test and post-test scores for each group were calculated using the Statistical Package for the Social Sciences (SPSS).Rather than give the groups arbitrary names or numbers, the groups are described by "weeks of treatment."Twelve weeks corresponds to the group that experienced the full treatment period of one trimester.Six weeks corresponds to the group that experienced one-half treatment period.The control group is described as "0 weeks" of treatment.Pretest and post-test means and standard deviations for each group are given in Table 2 for seventh grade students and Table 3 for eighth grade students.Pre-test and post-test scores were close to the normed means of the TerraNova, meaning that all groups performed at approximate grade level.In addition, the groups had generally similar pre-test and post-test means.The groups participating in a full unit of technology ("12 weeks" in the tables) have consistently higher pre-test scores than the other two groups.This is true for seven of the ten subject area scores, with seventh grade reading and eighth grade mathematics and science being the only exceptions.The difference was not echoed in the post-test means, which showed no visible trend among groups.
These pre-existing differences on the pre-test were taken into account in the ANCOVA analysis, which adjusted post-test means using the pre-test scores as covariates.Gain scores for each subject in the treatment and control groups were calculated for each grade.Mean gain scores are shown in Table 4.As expected, mean gain scores for each subtest were all positive, indicating an increase in achievement from pretest to posttest.The mean gain varied from a low of 11 points in social studies for the eighth-grade 0-week (control) group to a high of 29 in mathematics for the seventh grade 6-week (half unit of technology) group.The gains are at the level that can be expected from a pre-test and post-test separated by one school year.
A visual examination of the gains by group showed differences between groups, but no clear trend could suggest significant differences among the groups.When mean gains were widely separated, it was the 0-week (control) group or the 6-week (half unit of technology) group reporting the higher gain.These gains were not consistent among the subscores, nor were they consistent between seventh and eighth grade.This lack of visual trend was an indication that the gains for the treatment group were not substantially different than those for the control group.However, these gain scores were used for visual analysis only.The statistical analysis relied on ANCOVA methods, which used pretest and posttest scores directly to determine differences between groups.
ANCOVA results are reported in Table 5 for seventh grade students and Table 6 for eighth grade students.The Between Groups F value for each subject area indicates the difference in corrected means between the three groups.This value was used to determine the statistical significance of the difference among the treatment and control groups.The F values ranged from a low of .085for eighth grade mathematics [ F (2, 130) = .085,p = .919]to a high of 2.01 for seventh grade reading [ F (2, 155) = 2.01, p = .137].F values at this level suggest essentially random differences among group means, a finding that -17- In contrast to the between groups comparisons, the p values for the pretest as the covariate were at the p < .001level for all tests, indicating a significant correlation between the pre-test and post-test.This is a favorable result, but not unexpected.The pre-test and post-test were sequential versions of the TerraNova test.The TerraNova has excellent reliability (Monsaas, 2001), and pretest to posttest correlation is therefore quite high.Discussion ANCOVA data indicated that none of the ten potential differences between groups is statistically significant at an alpha level of .10.Although it is impossible to achieve the precision of an ANCOVA by visual comparison alone, a visual analysis of the gain scores for each of the three groups also fails to show any clear trends or any real trend at all.The data gathered do not show statistical or practical differences among the treatment and control groups.The results of analysis indicated that no significant difference existed between the achievement gains shown by each of the three groups in any of the five subject areas.In addition, no visual trends could be discerned among the three groups, as pre-test to post-test gains fluctuated randomly among the groups.
The pretest had a significance of p < .001when used as the covariate in the analysis of covariance, indicating a high pre-test to post-test reliability.Effect sizes were generally low due to the relatively large sample size of approximately 50 students per group.However, the statistical precision offered by these favorable circumstances failed to expose any realistic achievement differences between those students who had participated in the technology course and those who had not.

Conclusions and Recommendations
Based on analysis of the data collected in this study, it can be concluded that there is no significant difference in reading, language arts, mathematics, science, and/or social studies achievement gain between those students who have participated in a unit of modular technology education and those students who have not.The results of this study did not support the claim that participation in a modular technology course can increase students' achievement in other academic subjects.Although it is not the intent of this study to make implications for the entire profession, the finding of no significant difference, combined with similar findings in the literature, leads to the logical conclusion that the body of research knowledge cannot currently support claims of increased achievement in other academic subjects due to participation in modular technology education.
The conclusion of no significant difference in achievement gain between the groups of this study should encourage others to conduct additional research beyond the limits of this study.Certainly, longer exposure to a technology education curriculum may produce measurable differences where this study did not.More importantly, it may be very productive to explore the effect of technology education on higher-order thinking skills, which may not have been measured with the general achievement test used in this study.In particular, a test employing constructed-response items may provide greater discernment of students' higher order thinking skills and may yield different results than those found in this study.
Additionally, the scope of this research should be extended to all types of technology courses.This study examined a modular technology course with content provided by one commercial vendor.One could reasonably expect differing results when testing technology education's impact on achievement when other content delivery methods (standards-based, traditional, or courses delivered with other commercial products) were utilized.Further research could identify types of technology education that are more effective at raising achievement in certain areas.For example, modular technology education courses may have vastly different effects than exploratory or design-based courses on achievement in mathematics and science.
Proponents of modular technology education should exercise caution when making claims of achievement gains in other subjects as a result of such courses.The researchers do not presume to deny such claims on the basis of this limited study.The researchers also recognize that many students in modular technology courses have unique experiences that may show such effects.However, proponents of technology education may have difficulty in finding published research findings that support claims of achievement gains in other subjects.The researchers encourage persons who are evaluating modular technology courses to seek objective evidence to support any perceived benefits of the courses.
Finally, technology educators should strive to develop a rationale for their courses that does not depend on benefits to other subjects.It is unlikely that any course or field of study will ever thrive on the basis of its potential benefit to other fields of study.For example, it is unlikely that professionals in mathematics would attempt to garner support for the field of mathematics with claims that, by taking courses in mathematics, students improve their performance in history classes.Technology education offers unique content and skills that cannot be duplicated in other fields of study.While continuing to pursue the benefits of interdisciplinary studies, technology educators should develop a strong rationale for the program that rests upon the powerful and unique contributions of technology education and not on the potential impact it might have on other disciplines.

Table 1
Selected Core Skills Addressed in the Technology Modules

Table 2
Pre-test and Post-test Scores for Seventh Grade Students

Table 3
Pre-test and Post-test Scores for Eighth Grade Students

Table 4
Mean Gain from Pre-test to Post-test

Table 5
Analysis of Covariance for Seventh Grade Achievement Scores

Table 6
Analysis of Covariance for Eighth Grade Achievement Scores and Weeks of