Identifying Perceptions that Contribute to the Development of Successful Project Lead the Way Pre-engineering Programs in Utah

Identifying Perceptions That Contribute to the Development of Successful Project Lead the Way Pre-Engineering Programs in Utah


INTRODUCTION Background for the Study
An Educational crisis has been reported from many scholarly perches for the last quarter century. In Rising above the Gathering Storm, the National Academy of Sciences, National Academy of Engineering, and the Institute of Medicine (2005) concluded: We owe our current prosperity, security, and good health to the investments of past generations, and we are obligated to renew those commitments in education, research and innovation policies to ensure that the American people continue to benefit from the remarkable opportunities provided by the rapid development of the global economy and its not inconsiderable underpinning in science and technology. (p. 10) This report and others suggested that America is losing its global competitive edge in the field of engineering, science, and technology because America's educational systems cannot in their present state take on the challenge of educating our children to the standards of the future. In the five years that have passed since Rising Above the Gathering Storm was issued, much has changed in our nation and world. Despite the many positive responses to the initial report, including congressional hearings and legislative proposals, America's competitive position in the world now faces even greater challenges, exacerbated by the economic turmoil of the last few years and by the rapid and persistent worldwide advance of education, knowledge, innovation, investment, and industrial infrastructure. Indeed the governments of many other countries in Europe and Asia have themselves acknowledged and aggressively pursued many of the key recommendations of Rising Above the Gathering Storm, often more vigorously than has the U.S. We also sense that in the face of so many other daunting near-term challenges, U.S. government and industry are letting the crucial strategic issues of U.S. competitiveness slip below the surface. (p. x) It is apparent that America is challenged with restructuring its educational efforts to push more students into engineering and technology fields requiring a thorough understanding of math and science.

In the report The Knowledge Economy: Is the United States Losing Its
Competitive Edge assembled by the Task Force on The Future of American Innovation (2005), they advocated: Federal support of science and engineering research in universities and national laboratories has been key to America's prosperity for more than half a century. A robust educational system to support and train the best U.S. scientists and engineers and to attract outstanding students from other nations is essential for producing a world-class workforce and enabling the R & D enterprise it underpins. But in recent years federal investments in the physical sciences, math and engineering have not kept pace with the demands of a knowledge economy, declining sharply as a percentage of the gross domestic product. This has placed future innovation and our economic competitiveness at risk. (p. 1) Educational reform is paramount in defining our goals for the future and in reaching those goals both in secondary education institutions and our nation.
The educational crisis addressed in this research is characterized by K-12 public education not producing students who have the necessary skills or inclination to be successful in college and university engineering programs across the nation. The Program for International Student Assessment (PISA) ranks the United States 25 th out of 34 in developed democracies in math and 17 th in science (U.S. Department of Education, 2009). A problem exists with a shortage of engineers in the nation (Johnston, 2001). By 2006, the United States ranked 17 th in the percentage of university science and engineering graduates, down from third place 30 years before (Brown, 2009). Statistics show a consistent drop since 1988 in engineering graduates from universities across the nation with a 1.2% increase in 2008 (Gibbons, 2009).
The prevailing theory for the deficient numbers of engineers is the culmination of a pyramid effect, with the top of the pyramid being university engineering graduates and the bottom of the pyramid being pre-engineering programs and other engineering prerequisite courses in K-12 public schools. Not enough students are graduating from college engineering programs across the nation to fill demands because not enough secondary students are entering the "pyramid" at the bottom. The pipeline of students from high school to universities has been severely hindered. Therefore, either not enough high school students are engaging engineering tracks while in high school or they fall out of those tracks for some reason.
To help build this pyramid at the bottom, many secondary schools in the nation have implemented a pre-engineering program known as Project Lead the Way (PLTW).
PLTW partners with middle schools and high schools to provide a rigorous and relevant science, technology, engineering, and math (STEM) education. Through an engaging, hands-on curriculum, PLTW encourages the development of problem-solving skills, critical thinking, creative and innovative reasoning, and a love of learning.
Nationally, students who want to attend college should graduate from high school with the skills needed to be successful in rigorous academic college programs that lead to academic occupations such as engineering. However, reports previously referred to make it clear that there is a need in the nation for educational improvement in STEM fields.
Being faced with the challenge of increasing student engagement in STEM classes, many secondary school administrators across the nation and in Utah have implemented preengineering programs such as PLTW.
According to PLTW (2011) statistics the program is having an impact on student achievement in engineering and engineering technology across the nation. Initiated in New York in 1986, PLTW is now in all 50 states and in the District of Columbia. Its curriculum is in over 4000 schools being used by over 350,000 students nationwide (PLTW, 2009). These statistics suggest that states are adopting and using pre-engineering curricula with the intent of increasing student achievement in STEM classes.
There are many successful PLTW programs across the nation and in the State of Utah. If PLTW is a viable option to help build the nation's STEM pipeline, it would be helpful to those who make decisions on implementing these programs to know factors associated with successful programs. Knowing these factors can help schools build viable and sustainable programs. A need exists to examine the characteristics associated with successful PLTW pre-engineering programs. Research is needed to identify those "perceived factors" that contribute to successful pre-engineering programs that have the ability to attract students and promote student achievement.

Pre-Engineering Curriculum in Utah
There are over 30 different pre-engineering programs in the nation as listed in Engineering in  Understanding the Status and Improving the Prospects, (National Academy of Engineering and National Research Council, 2009). Of all the programs listed, the most widely implemented is PLTW and this holds true for Utah. The Utah State Office of Education (USOE) offers two pathways in pre-engineering for Utah secondary schools to implement. They are the "Utah Plan" and PLTW. Less than five schools in Utah offer classes in pre-engineering under the "Utah Plan" which takes less semesters to complete than completing the PLTW program and better fits some school needs because of scheduling and lower cost. The costs of implementing PLTW classes range from approximately $8,000 to $10,000 each for a class of 20 students depending on the equipment that a school already has. According to Darrell Andelin, the State Engineering and Technology Education specialist in the USOE, the state has subscribed heavily to the use of the PLTW program in its secondary schools and encourages schools to use PLTW curriculum.
It is evident that our nation needs more qualified people in STEM occupations. It is also evident that educational systems need to change their curricula in order to help facilitate change in secondary student credentials when they leave high school, meaning students achieve a higher degree of competence in STEM subjects. PLTW seems to have been embraced by our nation and has diffused through high schools in every state. This is providing a means to help secondary schools quickly make necessary curriculum changes which could lead to improving student achievement. Utah has adopted the PLTW program in many of its high schools and many of these programs have been "deemed successful" by the administrators and teachers. However, what does it mean to be successful? What does a successful PLTW program look like? What factors contribute to its success? A study is needed to identify those "perceived factors" associated with successful PLTW programs in Utah.
Since the appearance of PLTW in Utah schools, little research has been done to find out what it means to be successful. There is a tendency to define program success by showing that the program had an effect on the student's career choice after they have left high school. However, gathering data about the post-secondary effects of the PLTW program is difficult because it is hard to track students after they graduate from high school and there are too many variables to account for in people's lives. Even if graduates could be found, the responses in interviewing or surveying these people about the program's success would be subjective. For example, if students who had taken PLTW classes in high school and were now on a post-secondary engineering track were asked if PLTW was the sole reason for them becoming engineers, the subjectivity of the responses would make it difficult to credit the PLTW program as being responsible. Perhaps survey questioning and data analysis that involved multiple regression could generate statistical significant findings which would indicate how much impact the PLTW program has had on students successfully becoming engineers, but the time required would again make it prohibitive for this study. The basis for showing the success of a PLTW program in Utah may not be coming from research that shows what students are actually doing after they graduate from high school. In fact, after speaking with the Utah State Engineering and Technology Education Specialist, the USOE seems to have little valid post-high school student tracking data. If program success may not necessarily be defined by tracking students after they leave high school then success may possibly be defined in terms of observation of the interaction between students and the PLTW program while they are in high school.
What factors contribute to successful high school programs? Sometimes, students migrate towards STEM occupational majors because they were just better in those types of courses in high school, not necessarily because of guidance from PLTW. In Understanding What "Success" Means in Assessment, Piket-May, Chang, and Avery (1997) attempted to define success in electrical engineering programs. They explained that students drift towards STEM occupations, especially engineering because they liked STEM courses better than others taught in the school and were more successful in them.
They also elaborated about other people's influences on these students by saying, "In high school, students who have good grades in math and science are encouraged to become engineers by a variety of sources. These include guidance counselors, parents, and standardized interest inventory tests" (p. 2). Therefore PLTW may contribute to students understanding what is expected in STEM occupations, but the decision to become an engineer may have come from other sources such as influence by other people or from the natural abilities and aptitudes of the student.
Defining a successful secondary pre-engineering program from the collegiate point of view could be predicated by examining university goals. In reviewing the goals of several universities for their pre-engineering programs many similarities were found.
In typifying university goals in pre-engineering, the University of Las Vegas (2011) had as their pre-engineering course goals: (a) improving math, problem-solving, and college success skills needed for future engineering courses, (b) clarification and mapping of academic and professional goals, (c) expanded knowledge about engineering science majors and career opportunities, and (d) timely intervention and support services to increase academic success and retention. (p. 1) These goals may contribute to inspiring students to become engineers, but most of these goals point to student achievement. Program success, whether in secondary or postsecondary education, again could be defined in terms of observing the interaction between students and the program, which means the program has the ability to attract students and promote achievement where inspiration to become an engineer may be part of that achievement.

Characteristics of a Successful PLTW Program Relevant to This Study
Identifying PLTW program success may have several points of view. It may be viewed as being successful on local, state, or national level. It may also have many different critics (e.g., PLTW, parents, counselors, administrators, engineering educational organizations, researchers, etc.). Each faction may have their own particular reason for deeming the program successful but in reviewing them, PLTW program success tends to be based on the theory that it is successful because it has not been dropped from the schools' course offerings. Even though there have been studies on successful teaching within the PLTW program, the program itself could fail if there are not enough students enrolled in the program to justify its existence. Also, student achievement may not be satisfactory enough to "carry" the program or the program may not be meeting the goals established when it was implemented.
In a conversation with Darrell Andelin, the Utah State Technology and Engineering Specialist (personal communication, April 21, 2011), he identified successful programs as getting the "right" students into the classes, meaning that students have the aptitude for learning about engineering, and that there is sufficient enrollment in tier-one classes. Tier-one classes consist of the three basic classes in the PLTW program.
Appendix A shows the conversation by e-mail with suggestions that he made. In other informal discussions with Career and Technology Education (CTE) directors, school administrators, and teachers around the state of Utah, it was also noted that courses are kept in a school's curriculum because they exhibit the characteristics of being able attract students and maintain satisfactory enrollments. The courses are able to promote measureable student achievement, and are perceived as meeting the goals of implementation.
PLTW indicates their success on a national level by using statistics showing things like: (a) PLTW alumni are 5 to 10 times more likely to pursue engineering and technology classes than other first-year college students, and (b) 97% of PLTW alumni said they planned to pursue a 4-year degree as opposed to 67% of non-PLTW students (PLTW, 2011). The PLTW organization bases its success in two ways: first by using these statistics to try and show what students do after they participate in the PLTW program (where completing PLTW students seem to outperform students who did not participate in PLTW), and second by showing how PLTW has grown since the program is now in all 50 states servicing over 350,000 students in nearly 4,000 high schools (PLTW, 2011). PLTW bases its success on enrollment and student achievement.
This study is not about identifying the number of students needed for satisfactory enrollment or identifying the means of showing satisfactory achievement, this is left up to the individual school. However, identifying factors that are perceived to contribute to these characteristics which promote program success is one of the major focuses of this study. During the conversations discussed above, it was also brought up that courses are dropped because they do not have sufficient enrollment or do not show adequate achievement according to the schools' guidelines.
This study is about examining perceptions of PLTW programs in Utah that are viewed successful because they have demonstrated the ability to: (a) attract students by having adequate enrollments, (b) promote acceptable student achievement according to the schools' standards, and (c) meet the schools' goals of implementation or current program goals. Validation for PLTW program success in this study will come from interviewing CTE directors in the state. During the interviews probing questions relating to PLTW program success, will help generate a list of possible factors perceived to contribute to PLTW success by increasing or maintaining the necessary enrollment for program justification and promote student achievement at an acceptable level. Teachers, counselors, and school administrators associated with PLTW will then be surveyed to find out if they agree or disagree with these factors signifying program success and to what extent. Comparisons and contrasts will also be made in the findings to see if there are demographical differences of opinions between teachers, counselors, and school administrators.

Significance of the Study
This research is of significance to the field of engineering and technology education because it extends the knowledge base on factors perceived to contribute to the success of PLTW programs in secondary schools in Utah. Although this study will only look at schools in Utah, it is hoped that the findings can be used on a broader scale to look at the success of PLTW programs across the nation.
Predicating on the perception that a successful PLTW program has the ability to attract students and promote student achievement, this study will question CTE directors, school administrators, teachers, and counselors to identify and rate factors which are perceived to contribute to program success. Findings from this study will also aid Utah school and state administrators in implementing new programs, sustaining existing programs, and improving existing programs. Scrutinizing the findings of this study could impact student education and help define the mission of the PLTW programs in Utah.

Need for the Study
School administrators often use various indicators (e.g., data on student enrollments, end-of-level tests, grades, etc.) to measure the successfulness or failure of a program. Other indicators such as meeting the goals and objectives of the program can also be used to confirm success or failure of a program. There are many PLTW programs in Utah. However, no one has defined what a successful PLTW program is or examined the factors associated with successful PLTW programs. A need exists to identify factors which contribute to program success by showing what factors aid the program in attracting adequate numbers of students and promote student achievement.
Because there are many schools in Utah that have PLTW classes and with the likelihood of that number increasing, this study provides findings that may help administrators implement successful PLTW pre-engineering programs. Also, because there is limited data available describing what factors may contribute to program success or the revitalization of an existing program, this study will add to that knowledge base.
PLTW is a nationally renowned program which has the unique place of being a hybrid program containing both career and technical education and general education components. Therefore its successful role in Utah schools needs to be defined.
The mission of the PLTW program may be different between schools in Utah and schools in other states. Utah schools may emphasize different components and different program goal priorities than other states do. This study is needed to define PLTW success in Utah and discover factors that could promote that success. The mission of PLTW may have also changed in Utah. In speaking with the former State Technology and Engineering Specialist Melvin Robinson (personal communication, July 7, 2011), who initially facilitated the implementation of PLTW in Utah schools, he said that PLTW initially had 3 motives: the first was to increase the number of high school students entering the engineering pipeline to universities, the second was to give students more practical experience while steering them back into the engineering track, and the third was to provide districts with a stable platform from which to conduct pre-engineering education. This study will also find out if the reasons for implementing PLTW are the same or if they have changed.
The Carl D. Perkins Career and Technical Education Act provides federal funding for PLTW program concentrators and program completers. Student participation numbers in pre-engineering programs such as PLTW must be reported to the federal government in order for those dollars to flow to the state programs. Each year CTE directors have to submit their goals to complete the application for Perkins monies. There is a need for this study to aid in the process of evaluating existing programs and identifying success in order to secure those Perkins funds by increasing the number of student concentrators and completers. The factors for program success identified in this study create a data base to aid district and school administrators in correcting problems that hinder programs from being as successful as they could be.

Research Questions
This study consisted of two phases. The population during Phase I then would be CTE directors in the state of Utah that have PLTW in their districts (N = 10). The data from this population will be obtained from the main categorical questions and from probing questions about the main questions asked in an interview.
The second phase in this study involved surveying school administrators, counselors, and teachers in schools that use the PLTW curriculum to gather their perceptions about success factors of PLTW. These populations will be surveyed using an electronic Internet-based survey system (i.e., SurveyMonkey) and the questions asked will be generated from the responses obtained during the first phase of this study. The results of these surveys may then be compared and contrasted demographically yielding findings that will indicate the strongest factors in each success category. The research questions associated with Phase II were as follows.

Purpose of the Study
The purpose of this study is to examine PLTW program success by identifying controllable factors which may be considered at the time of PLTW program initiation or program evaluation. Achieving this purpose will include creating a theoretical framework for identifying and implementing successful pre-engineering programs in Utah secondary public schools. Examining these controllable factors may lead to stronger success of the program upon implementation or improvement of existing programs making them successful by factor manipulation.
On a larger scale, the purpose of this study is to suggest ways to maintain successful PLTW programs in Utah secondary schools, which could provide more opportunity for students to embark on educational and occupational pathways. This could advance the goals of PLTW and could strengthen our nation's workforce.

Assumptions and Limitations
The following assumptions were made while conducting this research.
1. Data can be accurately drawn from the group of survey and interview participants.
2. Responses will reflect real-life experiences and those who participate in this study will be truthful and thoughtful in their responses to all questions.
3. The information gathered for this study will be reported accurately, without bias, and all reasonable efforts to maintain validity and reliability will be made.
4. All protocol whether by state, district, or individual schools will be strictly observed.
The study was limited to the following. The new law also included new requirements for "programs of study" that link academic and technical content across secondary and post-secondary education, and strengthened local accountability provisions that will ensure continuous program improvement.
The Perkins Act provides almost $1.3 billion in federal support for career and technical education programs in all 50 States. The law will extend through 2012.
Completers-defined by the USOE as students in Utah secondary schools who complete 4 credits in pre-engineering classes.
Concentrators-defined by the USOE as students in Utah secondary schools who have completed at least 2 credits in pre-engineering classes.
Concurrent enrollment-students enrolled in a PLTW class where they were eligible to also receive university credit along with high school credit.

CTE-Career and technology education.
GPA-Grade point average.
Magnet schools-public schools with specialized courses or curricula. "Magnet" refers to how the schools draw students from across the normal boundaries defined by authorities (usually school boards) as school zones that feed into certain schools.
Membership hours-referred to by the USOE as the total hours that are accumulated by students attending a CTE class.
PLTW end of course exam-the exam provided by PLTW and given at the end of the PLTW course, used to obtain concurrent enrollment, and for state program completion statistics.

Introduction
There are not enough engineers graduating from the nation's universities in the United States of America. There is a greater need for scientists, engineers, and other technically skilled workers than can be supplied (Jackson, 2004). Newspaper articles entitled, "Project Lead the Way-Bemidji School District seeks community partners in pre-engineering program" (Ruckdaschel, 2006) and "Kern Family Foundation Commits $10 million to PLTW" (SharpEdge 3.0, 2009) provide evidence that there is public interest in "raising-the-bar" in STEM education. Many publications in newspapers, magazines and educational journals outline the importance of continuing to raise standards and offer diversified instruction to better facilitate student learning. To help guide this study, a review of literature was conducted to examine the following.
1. Pre-engineering in K-12 schools in America.
2. The "Project Lead the Way" program.

Pre-Engineering in K-12 Schools in America
This section is important in this study because it helps with understanding the growth of K-12 pre-engineering programs in America. It also shows that attempts have been made to rectify the problem of America's lack of integrated STEM education and pre-engineering program application. The first part of this section addresses the national need for more engineering education and then the attempts to integrating engineering curriculum into public education. Described in this section are examples of initial preengineering projects that have contributed to the development of the leading preengineering programs across the nation. There will also be discussion about some of those pre-engineering programs. The PLTW program was chosen for this study because it is the biggest pre-engineering program in America and is the most prevalent in Utah. It is also the only formal pre-engineering program in Utah.
Statistics show a consistent drop since 1988 in engineering graduates from universities across the nation with a 1.2% increase in 2008 (Gibbons, 2009, p. 1). This fact, coupled with the attrition rates of our engineering force through change of profession, retirement, and death, spells an educational crisis with colleges and universities not graduating enough scientific and technical talent to replace them. This 20-year trend is expected to increase the need by another 20% in the next decade (Jackson, 2004).
To compensate for the lack of qualified engineers, American industry has sought assistance from foreign-born employees. In 1980, the foreign engineering force working in the United States was approximately 7%. In 1990 that percentage grew to 14% and by 2000 it had risen to 22% (Gibbons, 2009, p. 1). National crises such as the aftermath of There is a quiet crisis building in the United States-a crisis that could jeopardize the nation's pre-eminence and well-being. The crisis has been mounting gradually, but inexorably, over several decades. If permitted to continue unmitigated, it could reverse the global leadership Americans currently enjoy. The crisis stems from the gap between the nation's growing need for scientists, engineers, and other technically skilled workers, and its production of them (p. 4).
A serious shortfall is represented by the gap in our national scientific and technical capabilities. Ignoring this gap may lead to perilous times in America's future.
To help close this gap in engineering personnel, secondary pre-engineering programs have been implemented in over 4,000 schools in 50 states (National Academy of Engineering and National Research Council, 2009). With this much growth, the perception of pre-engineering program by school officials and the public seems to be that these programs are really meeting the needs of today's youth and should be considered for implementation in secondary public schools whenever possible.
For the last 30 years we have increased educational efforts and have tried inexhaustively to get the latest innovations and policies into place. In the 1960s, a lot of funding went into national curriculum efforts, open-planned schools, and individual instruction, followed in the 1970s by a period of stagnation, regrouping, and recovery (Fullan, 1993). Fullan went on to explain that somewhere along the way it seems it was forgotten that one of the main purposes of education was to prepare young people for the work place. Secondary public schools traditionally have been slow to understand, change, and meet the challenges of the modern day work place (p. 4). Another possible reason for implementing PLTW courses is that they reflect the modern day work place.
Today's job market is not the same as yesterday's. Businesses are looking for people who can do things quickly and spend less time with specific on-the-job training.
There is a shortage of skilled engineers, and technical workers are hard to find in the U.S. (PLTW, 2009). But, making change happen in the public school systems to try and produce more engineering students, especially ones who will complete an engineering program in college, is not easy. Better job market preparation for students could be another reason for implementing pre-engineering programs in American schools and the growth of participation in these programs verifies this fact.
Change has been slow in the making. There have been many attempts to implement a pre-engineering program at the high school level, and some programs have enjoyed some success. For example, in 1995 there was a program put into place from the University of Maryland at Paint Branch High School. A 1-year curriculum was used for a course in engineering design. Although the program was under the eye of the district, it continued through the second semester of the pilot, and has forged a good relationship with the University of Maryland (Schwartz, Regan, & Marshall, 1997).
Another good example of a successful program happened in 1963 in Washington, D.C. A project consisting of three months of work was developed for 30 to 35 students to participate in from three local high schools. The students worked on recovering abandoned lots and renovating vacated buildings. The project was called City Vision.
"Some D.C. kids consider their neighborhood as just the two blocks around their home.
But an outreach program at the District's National Building Museum is getting them to broaden their perspective" (Holmwood, 2001, p. 1). This program had students work on projects located around the city, which were intended to enrich neighborhoods with creative engineering designs. Students were taught the basics in design, such as model building, sketching, and composition. Then they split up into groups, each looking at a different neighborhood that could be renovated or put to better use. The semester culminated in a final presentation at the museum. The program helped many students go on to more advanced engineering achievements and broadened their perspective of the world around them. Students were shown how they could make a difference in the world.
Isolated cases such as these, although may have worked for small populations for short periods of time, are not effective for all. There are many examples such as these, and while we applaud the efforts of the people who make them happen, they represent little change in the overall system. We need to change our way of thinking about how to teach engineering and offer it to the nation.
Today, there are many programs available for public schools to participate in preengineering. As previously mentioned in Chapter 1, Engineering in K-12 Education: Understanding the Status and Improving the Prospects, (National Academy of Engineering and National Research Council, 2009) has cited over 30 programs. Table 2.1 shows a brief list of some of these larger programs along with their participation to provide understanding of the diffusion of pre-engineering programs in America. A world in motion This curriculum is used in all 50 states and in 10 Canadian provinces. More than 65,000 AWIM kits have been shipped to more than 16,000 schools since 1990. The developer (Society of Automotive Engineers) estimates that more than 4 million students in North America have participated in AWIM activities (based on the assumption that the curriculum kits are reused an average of 2.6 times in classes averaging 24 students).

Engineering is elementary
This curriculum is used in about 850 schools in 46 states and the District of Columbia. Based on sales figures and teacher participation in professional development workshops, the developer (Boston Museum of Science) estimates that about 15,000 elementary school teachers are using their materials. Approximately 1 million students have been exposed to the EiE curriculum since its inception.
Note. These data are presented as reported by the curriculum developers.
Of the programs cited in this list PLTW is the largest by far. There are other programs such as Materials World Modules and the Infinity program that do have some momentum, but they are not as big as the PLTW program. PLTW is now in all 50 states and is one of the premier pre-engineering programs in the nation. However, even with its diffusion and growth, PLTW is relatively new in America and nascent research is just now yielding precursory findings on its impact on public education.

Project Lead the Way Program
This section is important because it preludes this research. Without a clear understanding of the PLTW program and the workings of its elements, it would be difficult to conduct research on it. This section describes the PLTW program and its learning tactics to achieve student success by introducing them to engineering concepts and also increasing math and science skills coupled with complex systems. This section is crucial in conducting research on PLTW.
The purpose of PLTW is to provide a complete curriculum with a scope and sequence for students to follow in pre-engineering. The PLTW's pre-engineering program at the secondary school level consists of curricula for three tiers of education.
The first tier includes foundation courses, which are: introduction to engineering design (IED), principles of engineering (POE), and digital electronics (DE). After successful completion of the tier one courses students may then take one or more of the tier two specialization courses that include aerospace engineering (AE), biotechnical engineering (BE), civil engineering and architecture (CEA), and computer integrated manufacturing PLTW also attempts to attract a higher percentage of "middle" grade point average (GPA) students into their classes to introduce them to the field of engineering instead of limiting student participation to the academic top. Their "can-do" philosophy suggests that students who thought they had no aptitude for engineering fields of occupation may find success in the PLTW program and learn that they could possibly pursue an engineering field of occupation.
PLTW involves universities in its quest to strengthen the pipeline connection between secondary schools and universities. At some colleges and universities, PLTW classes are offered for concurrent enrollment. Students are usually required to pass an end-of-course exam before credit is granted. The credit received by students at universities and colleges is usually basic, which could fill the role of elective courses.
Schools planning to offer four or more high school PLTW courses are eligible for PLTW certification and may begin the process for certification at the end of the second year. The purpose of certification is to ensure implementation of a high quality PLTW program and to verify college credit eligibility for select PLTW courses.  work as a contributing member of or lead a team;  use appropriate written and/or visual mediums to communicate with a wide variety of audiences;  participate in public speaking;  listen to the needs and ideas of others;  understand the potential impact their ideas and products may have on society;  use problem solving methods and skills;  manage time, resources, and projects;  participate in researching ideas and concepts including data collection and analysis;  go beyond the classroom for answers;  be better prepared for success in two-and four-year college programs. (p. 1) This philosophy seems to enable students to cope with needs they have when they enter the workforce or the university. PLTW classes also have students thinking "outside-thebox" to engineer solutions for today's problems; meaning that this method may offer engineered solutions which are sometimes more efficient, cheaper, more practical, and possibly have less environmental consequence. With this philosophy PLTW hopes to close the "gap" between education and the workplace. Understanding PLTW's philosophy is important to this research because it helps facilitates the generation of factors that may be tested in research which could increase the program's chances for success.
Activities are a method of instruction that involves directed teaching of a particular process or procedure. Activities engage students in learning skills that are later applied in more complex situations and systems. Activities lead students to higher levels of learning (PLTW, 2009). A popular instructional approach used in PLTW is projectbased learning (PBL). PBL is a comprehensive approach to instruction that presents a project or relevant activity that enables the student to synthesize knowledge and to individually resolve problems in a curricular context (PLTW, 2009)

PLTW as an Effort to Increase Student Learning in Math and Science
This and the next two subsections in this review of literature help in understanding some of the goals that the PLTW program has set forth that will aid students in reaching higher academic skills. They will help guide this research by generating interview questions and identifying factors that may contribute to program success in the way PLTW uses its curriculum.
Reasons for raising math and science standards include: poor student achievement test scores, poor student retention of concepts taught, poor application by the student of the concepts taught, and apathy towards the subject material. For these reasons school districts have implemented some form of standard evaluation. This can be done by requiring more credits in these subjects or by implementing classes such as those offered by PLTW, where the subject objectives are taught in a different, more applicable setting.
"PLTW's curriculum makes math and science relevant for students. By engaging in hands-on, real-world projects, students understand how the skills they are learning in the classroom can be applied in everyday life" (PLTW, 2009, p. 1).
When students use problem-solving skills, they rely on their past experience and memory to find a solution. Sometimes these solutions require methods taught in different classes. Using parts of many different subjects along with their supportive curriculums to solve real life problems is a powerful tool schools can use to help students with internalizing math and science concepts, and is another reason for implementing the PLTW program.

Cognitive Approach to Learning with PLTW
Research has begun to explore new classroom techniques such as using computer modeling software, and model building, which will help students learn complex systems ideas and focus on what the learner is actually doing in the classroom. "A central tenet of constructivist and constructionist learning approaches is that learning is actively constructing new understandings, rather than passively receiving and absorbing facts," suggests Jacobson and Wilensky (2006, p. 23). Using this learning philosophy of emphasis on the learner could link model building to scientific experimentation about complex systems, and how students come to understand modern scientific inquiry grounded on cycles of theorizing, model building, and experimentation, which in turn leads to further theorization and model revisions (Jacobson & Wilensky, 2006). Learning cycles such as these engage the learner and create learning opportunities for the teacher where the learner is primed for the knowledge that the teacher offers. This approach is part of PLTW curriculum.
One model of understanding how humans learn is presented by Gee (2003) where he used an old analogy of learning where researchers compare the mind with a computer saying, "Learning is held to be a matter of grand generalizations, principles, rules, abstractions, and logical computations. This view treats the human mind as if it is pretty much like a digital computer" (p. 73). He went on to say that unlike the computer our minds edit the input information according to our interests, values, goals, and sociocultural memberships. Learning has to be situational in nature. The moment material is presented for learning, it has to be done in such a way that raises the consciousness of how our minds store and retain information. The situation has to be connected to something in our mind to deem it worth remembering. Again, research on PLTW precludes that we understand the approach in the methods of teaching in the PLTW program. Solving design problems such as those presented in PLTW presents a new way of thinking to pre-engineering students. Lawanto (2009) stated, "Design problems are among the most complex and ill-structured kinds of problems that are encountered in engineering practice. Researchers characterized by design problems as ill-structured because they have ambiguous specification of goals, no determined solution path, and require integration of multiple knowledge domains" (p. 2). Seeking out analogies and models on problem-solving methods will guide this study in questions asked about PLTW, identifying what program success is, and factors that make it successful.

PLTW's Approach to Learning Complex Systems
"Students should acquire a repertoire of cognitive and metacognitive skills and strategies that can be used when engaged in technological activity such as problem solving, decision making, and inquiry" (Johnson, 1992, p. 30). PLTW and its application in learning in other content areas such as math and science could certainly be identified as a complex system because of its many symbols, rules, algorithms, and applications. This could require a paradigm shift in terms of how teachers think about learning, problem solving, and curriculum development. New research, as suggested by Jacobson and Wilensky (2006), includes five curriculum design principles which are: "(a) experiencing complex systems phenomena; (b) making the complex systems conceptual framework explicit; (c) encouraging collaboration, discussion, and reflection; (d) constructing theories, models, and experiments; and (e) learning trajectories for deep understandings and explorations" (p. 19). Students must construct learning built on their own life experiences and the knowledge they have about their world. Understanding and retention of mathematical and scientific processes and applications may be amplified if these curriculum design principles are utilized during the planning stages of a particular unit of study. Also, the curriculum needs to allow for differences and unforeseen student needs during the unit of study. Designing curriculum utilizing PLTW concepts may facilitate student learning.
Active involvement is necessary when learning mathematics. Teachers must provide opportunities for students to physically engage in the learning process. While lecture and rote memorization does play a role in learning math and science, that does not give teachers a reason to teach primarily through this method. Activities that lead to exploration and understanding math and science on their own are some of the best ways to learn. Lawanto (2009) found that there were significant metacognitive changes during his research due to the nature of the design activities or working styles that differed across the three engineering fields examined (p. 4). PLTW has this kind of involvement in its curriculum. In speaking of experienced centered progressive education, in an article entitled "The Project Method," Kilpatrick (1918) suggested: As these questions (meaning questions about traditional lecture and other instructional methods) rose more definitely to mind, there came increasingly a belief-corroborated on many sides-that the unifying idea I sought was to be found in the conception of wholehearted purposeful activity proceeding in a social environment, or more briefly, in the unit element of such activity, the hearty purposeful act. (p. 49) Even though Kilpatrick made these observations many years ago, historically they have been verified many times since and still hold true today. PLTW promotes activities which help teach students how to learn as well as student motivation. Identifying factors that make PLTW successful by improving STEM academic achievement and teaching students how to learn may be good reasons to have the PLTW program in public schools.

Students Served by PLTW
Society and culture have a significant role in education. To better facilitate this study on identifying PLTW success and factors that contribute to that success we must review the culture and student needs of today. Today's fast moving technological society demands citizens and workers who are knowledgeable and qualified to work in today's technical occupational fields. Worker credentialing and certification have become increasingly important issues in the delivery of secondary and post-secondary career and technical education (CTE). Since the early 1990s, industry credentials and certification have increased in visibility with a substantial jump in the number of credentials available and the number of people seeking certification (Carter, 2005). America is turning to the schools to find out why many of our high school graduates do not have the skills for most basic jobs, or why they do not have a good enough educational background to be successful in rigorous university programs such as engineering.
Movements in education which are capstoned by documents such as "A Nation at Risk" (National Commission on Excellence in Education, 1983) pointed out that change is necessary to keep up with these times, and that our schools are not changing fast enough. Also, legislation like "No Child Left Behind" (U.S. Department of Education, 2008) is asserting itself to make schools accountable for what they teach and tries to guarantee that students are progressing at an acceptable level.
The pre-engineering concept is that of introducing students to the rigors of engineering through exploration and problem solving. Students find out what it is like to become an engineer in a positive way. In order to acquire and retain students at the university level it is beneficial to educate students while they are in high school about engineering and what to expect when they get into college. College engineering students spend an average of 19 hours a week studying (Helfand, 2011) and should be aware of such a time commitment. Students who have a strong background in math, physics, and science do have an advantage in going into engineering, but they need to understand what an engineer does, how to work with other people, and how to develop problem solving skills that require many different disciplines to solve (PLTW, 2009).
Students planning engineering careers in high school are encouraged to take advanced courses in math, science, physics, chemistry, and drafting. Students do need a strong background in these subjects to be successful in college engineering programs: By taking the highest level of college preparatory mathematics they are capable of successfully handling in all four years of high school, students will develop a solid background in math skills and concepts, will be prepared to take each level of the PLTW program, be prepared to succeed in the entry level mathematics course in college, avoid regression between high school and college by taking math each year of high school, and will have a solid background for engineering/ technology. (PLTW, 2009, p. 1) However, it is not until students arrive at the university that they take course work where they learn what engineering is about, and how to use it. Then they go out into the world to get a great job. This system worked well in the past, but today students need to arrive at the university with a background in the field of engineering itself. One problem with this system is that many students are bored with traditional classes. They arrive at high school from junior high thoroughly bored by science and technology related classes and give no thought to the subject beyond required courses (Whittaker, 1994).
The National Society of Professional Engineers has launched a nationwide campaign to reverse the image of engineering. Stereotyping engineers as being antisocial nerds with protractors has led to a drop in the production of competent engineers from universities. To combat this problem, the engineering profession is getting a makeover.
The hope is to show that engineering is fun and that engineers are problem solvers, community leaders, and people who improve the quality of life for us all (Johnston, 2001). Efforts need to be made at the high school level to entice students into the field of engineering. Students need to know more facts about engineering and not rely on the myths. Engineering requires art, balance, critical thinking skills, a good understanding of social economics, and a blend of "people skills" including group decision making.
Another critical factor in effective recruitment of students into high school preengineering classes depends on making an effort to ensure that the students who take these classes like them.
The concept of pre-engineering as a set of classes in high school is resurfacing.
While still requiring many of the same courses from the past, especially math, PLTW's curriculum approach is to teach engineering as a field of study unto itself and blend the other required courses into it. This evolution of a pre-engineering program seems to be well received, based on the numbers of new students enrolling in pre-engineering classes across the nation (PLTW, 2009).
A common myth about engineering is that only the top academic students in the school can be engineers. The field of engineering does require many math and physics classes. But, it also requires sensitivity and creativity. The message needs to be sent that virtually anyone can be an engineer, if they work at it hard enough. Walker and colleagues (2002) stated, "America's schools are challenged to provide all students with a secondary school education that prepares them for high-skill, high-wage jobs, and further education" (p. 2). He went on to explain how our teaching may have something to do with attitudes about high-skill jobs, "Quality teaching and teacher education are inextricably linked to the challenge" (p. 2). Teachers at all levels need to be retrained in how to teach engineering and make it attractive to students who traditionally do not take these classes. PLTW requires teacher training in each class before they are able to teach that class. The PLTW teacher certification process is 2 weeks long for 8 hours each day.
During this training the teacher actually does all the projects that the students do in the class.
High school students lack career decision making skills. Students make career decisions based on inadequate knowledge of their own characteristics and the demands of the job that they think they want to do. When students find out they lack the pre-requisite skills needed for the field they have chosen, it can quite often lead to failure and dropping out of the program. Many of the current school programs actually discourage career decision-making by students. The school usually does little to encourage students to find out all the facts concerning themselves and their chosen field before making a firm decision in career choice (Evans & McCloskey, 2002, para. 5).
Society today is challenged to acquire scientific and technological literacy: "Americans do not understand enough science and technology to make the political decisions required of them" (Whittaker, 1994, p. 73). We live in a push-button-give it to me now-society, where technology is simply understanding how to operate things, not how things actually work or the underlying principle of their operation (Whittaker, 1994).
As a result, fewer students enroll in engineering programs and, of the ones who do enroll, fewer graduate. In high school there are fewer students engaging in rigorous studies to put them on the engineering track, "As we frequently read, science, mathematics, and technology education is in trouble. The number of students taking these courses beyond the minimum required by the state statutes is declining yearly" (Whittaker, 1994, p. 52). The category of math and science was listed by the largest percentage (42.0%) of respondents as one of the top three areas of greatest deficiency that local K-12 schools have in preparing students for the workplace.
Cultures have also changed. Our children's culture is not the same as ours.
Children are subject to the hidden injuries of social and cultural class. From this comes a resistance theory, where a higher percentage of students are antagonistic, nonconforming, vulgar, obscene, and violent. While these traits will not be tolerated in public school, it does point out the fact that the needs of today's generation of students are different than their parents' generation. They do not conform well to the standards that have been set up for them (Lakes & Burns, 2002). Standards and curriculum should change.
Today's students are referred to as millennials-meaning they are born between the years 1982 and 2002, ranging from 10 to 24 years of age. In describing attributes of this generation Monaco and Martin (2007) explained, "The millennial student is the largest and most diverse generation to ever attend college. They are unlike their predecessors who attended college 10, 15, or 20 years ago" (p. 42). They go on to say that the millennial generation is characterized by: lack of professional boundaries influenced by socialization, a need to have immediate feedback, a sense of entitlement, lack of critical thinking skills, unrealistic expectations, high level of parental involvement, and an expected "how to" guide to succeed in and out of the classroom.
Millennial students have the expectation of using technology for education. Their culture is shaped by the use of portable computers, internet access, social networking, and audio/visual technology (Koch, 2010). Indeed in a master's thesis by Elizabeth A.
Howard (2011) about how millennial engineering and technology students experience learning through traditional teaching methods, it was found that three changes were a key to classroom success. These changes were: (a) using technology during lab sessions to create learning experiences for the students, (b) providing lecture material or outlines for students online, and (c) for teachers and professors to integrate online applications such Blackboard into their classrooms to provide additional learning resources. "Our students have changed radically. Today's students are no longer the people our educational system was designed to teach" (Prensky, 2001, p. 1). Methods of instruction used in PLTW classes are more in touch with the millennial digital-native generation. "Whether students are engaged in discussion, solving a problem, or designing a device, the implication is that students will be interacting with each other and the teacher in order to accomplish the learning task" (Shumway, Saunders, Stewardson, & Reeve, 2001, p. 1). Interaction spoken of here will include the use of technology as its underpinning.
People need to broaden their perspective of what engineering is about. "We have been sending the wrong message. In the past, the importance of math and science has been the lead discussion. That's boring," says Patrick Natale (as cited by Johnston, 2001, p. 1), executive director of the Alexandria, Virginia-based National Society of Professional Engineers. "We're coming out with a different message, saying engineering is fun and exciting, and coming out with examples, making it more user-friendly." This perspective is what we want our high school students to have about pre-engineering. Preengineering SEOP tracks in secondary schools should be something that the majority of students would be interested in doing.
Renovating high school pre-engineering curriculum is difficult and slow.
Technology education, which is the category pre-engineering falls under, "claims technological literacy as a goal of its discipline. This is a noble, but darn-nearimpossible-to-achieve goal" (Schultz, 2002, para. 1). Old curriculums will not lead today's high school student down a track of success and students are unwilling to participate in them. It will take a great movement to change what is happening in the field of engineering. Gilli (2002) made two recommendations: First, is to make education more desirable by increasing its efficiency in reaching students and the staff that presents it, and the second is to develop a "blue-ribbon" curriculum with an elite board of specialists.
PLTW exists because of societal needs and understanding these needs will serve as a guide in this study to identify factors of its success.

Studies about PLTW Programs
The Engineering Education Beliefs and Expectations Instrument (EEBEI) was developed by Nathan, Tran, Atwood, Prevost, and Phelps in 2010 to: (a) develop an instrument to measure teachers' beliefs and expectations about precollege engineering instruction, (b) measure teachers views and identify differences that exist among teachers with different training, and (c) examine teacher differences in advising fictional students.
Research using the EEBEI, and the EEBEI-T for teachers has shown, "High school STEM teachers report their instruction was influenced by students' interest, family background, and prior academic achievement" (Nathan et al., 2010, p. 409). The study also discussed that in a comparison between PLTW and non-PLTW teachers, the latter are of the opinion that engineering students must demonstrate high abilities in math and science while PLTW teachers tend to integrate the math and science skills into the project or activity at hand while they are teaching. While socioeconomic status (SES) was not reported as a factor that influenced their teaching, it did influence situational decision making tasks (Nathan et al., 2010). This research indicates that interest, family background, and prior academic achievement are factors which may be tested in this study to see if CTE directors, school administrators, and teachers in Utah agree or disagree on their merit.
The EEBEI-T was also administered to high school guidance counselors and found that advising was shaped by student performance. Guidance counselors tend not to use students' culture, home or ethnic backgrounds to inform course selection advising, and guidance counselors overwhelmingly advised students from all four vignettes in the study to enroll in pre-engineering courses (Nathan, Atwood, Prevost, & Tran, 2011b).
Counselors play a major role in students enrolling in PLTW classes and for that reason are included as a population to be surveyed to find out what they perceive as factors that contribute to successful PLTW programs.
In a quasi-experimental study using the EEBEI-T to measure how professional development changed high school STEM teachers' beliefs about engineering education (Nathan, Atwood, Prevost, Phelps, & Tran, 2011a), report that with regards to which students should enroll in engineering, expectations for engineering learning, and predicting career success of pre-engineering was generally favorable among students who had a high SES through survey logistics even though SES was not a directly tested factor.
This study also indicated that nascent PLTW teachers were more likely to increase STEM integration over time into their curriculum, which indicates that math and science were incorporated into the curriculum as a need-to-know basis in order to complete the project.
This could also be a factor of their comfort level as they develop mastery over their subject. This research indicates that professional development is a factor that needs to be assessed in this study because teachers need to know how math and science are to be used in their teaching to aid in student's retention of math and science concepts.
A study on PLTW conducted in Indiana found that principals presented obstacles when trying to implement PLTW programs because of their tendency to categorize them as traditional technology education classes (Shields, 2007). Perceptions held by administrators and teachers may be different, creating implementation and maintenance problems with the program, hindering success. Rating factors from the perceptions of program success between administrators and teachers and reasons why PLTW is successful is paramount for testing success factors in this study in the state of Utah.
Secondary public school administrators and teachers from across the nation are realizing that their schools could provide pre-engineering programs that allow students to investigate their strengths and interests in engineering and engineering technology (Thilmany, 2003). According to Dearing and Daugherty (2004), leaders from both secondary technology education and college-level engineering have called for changes in the high school curriculum to address the need to sufficiently prepare high school graduates for post-secondary progress related to engineering and technology. School districts across the nation are implementing pre-engineering courses into their curriculum. As schools infuse these pre-engineering programs, leaders and teachers in technology education are debating the virtues of pre-engineering education (Lewis, 2004). Student interest in engineering and engineering technology could be factors that contribute to program success and should be part of this study.
Other studies in Indiana have indicated that technology education teachers have embraced pre-engineering education as a valuable component of technology education (Rogers, 2006). Rogers went on to say that technology education teachers from Indiana also view the pre-engineering curriculum as favorable in developing technological literacy. Rogers and Rogers (2005) concluded that the forward provided by William A.
Wulf, president of the National Academy of Engineering, in the Standards for Technological Literacy: Content for the Study of Technology (International Technology Education Association, 2000) provided clear evidence that pre-engineering has become a component of the technology education discipline.
Secondary schools have experienced a rise in the engagement of pre-engineering programs (Douglas, Iversen, & Kalyandurg, 2004). There has also been an increase in the development of engineering-focused curriculum for grades 9-12 (Dearing & Daugherty, 2004), which gives reason to evaluate the impact of secondary engineering-focused programs on student learning. Indeed, "as these programs continue to grow, there is a need to build a strong base of rigorous research to provide educated and specific feedback on how to improve existing curricula and build a cohesive research agenda on engineering reasoning development in the K-12 grade spectrum" (Kelly, Brenner, & Pieper, 2010, p. 2).
Research on PLTW is limited and the research that has been conducted makes it clear that more research needs to be done, especially on a state-by-state basis, to discover and evaluate the elements of a successful pre-engineering programs. The research available usually concentrates on the teaching methods that PLTW brings to schools and focuses on the success of student achievement using those methods. The research found about PLTW makes it important to this study because it emphasizes the importance of doing research state-by-state in order to increase the chances of PLTW implementation being successful.
In a longitudinal evaluation of PLTW in the state of Iowa completed by Rethwisch, Laanan, Hayes, and Starobin (2011) it was found that, "students of PLTW programs are more likely to be white, male, and strong in the area of math and sciences.
Whites were overrepresented compared to their peers. Males were also overrepresented in PLTW compared to their peers, but female participation was higher in younger cohorts" (p. 13). These findings are important because statistics used to show improved student outcome in math and science because of participating in PLTW classes may be somewhat skewed, because these students could already show high proficiency before entering the PLTW program.
In a report on the third year of implementation of the True Outcomes Assessment system for PLTW by Walcerz (2007) it was found that, "Within any state, the racial and ethnic student population of PLTW schools is collectively proportionate to the overall population" (p. 7). It was also reported that, "females are represented in PLTW classrooms at approximately the same rate as in engineering and technology programs in college" (p. 7). While the report shows statistics which indicate that PLTW students are performing better on standardized achievement tests, have higher numbers moving on to college programs, and benefit from the program attributes that PLTW has in its curriculum, their registered student population was still predominantly white male and these students may have achieved these numbers with or without PLTW. Interpreting reports such as this must include skepticism and caution to make sure the research findings are indeed valid.
A guidebook for local leadership teams on improving learning and outcomes in technology and pre-engineering education was developed by Phelps and Alder (2007) at the University of Wisconsin-Madison's center on education and work. The progress and performance profile of the guide organizes data from multiple sources to ensure its effective use by teachers, counselors, administrators, local industry partners, and policy makers. The profile is organized to provide longitudinal (multiple year) data on four sets of indicators which are: (a) school and community context, (b) program implementation, (c) student and school outcomes, and (d) post-school outcomes. These four indicators address 11 essential guiding questions concerning technology education programs in middle schools and high schools (Appendix B shows a flow chart for the indicator application and a list of the guiding questions) with an emphasis on PLTW that are frequently posed by parents, educators, school board and community members, and policy makers. This groundbreaking work is important to this study because it matches student outcomes with objectives specific to PLTW and offers recommendations for improvement. This document could easily serve as a guide for other states to evaluate and improve their existing PLTW programs or to be heavily considered when implementing a new PLTW program.
The 20 or more studies and reports concerning PLTW, pre-engineering education, and technology education reviewed have been primarily about pre-engineering curriculum topics and the methods used to instruct students in these pre-engineering topics. A study by Bottoms and Uhn (2007) compared PLTW students with other CTE students and explains that the program "works" because PLTW students tend to have higher scores in math and science on achievement tests. Although in this study there appears to be many variables that were unaccounted for, and one should be cautious when interpreting the findings. Most other studies reviewed leave interpretation of success up to the reader and assume that the programs are meeting goals or discuss reasons why the programs were originally initiated. None of the reviewed studies define success or explain whether PLTW seems to be attracting enough students to make it viable or not. After reviewing these studies it is apparent that there is little research on defining exactly what PLTW program success is and what factors make it that way. Also, most of these studies were done in states like Indiana and Wisconsin that have implemented the highest quota of programs in the nation. Again, little research is available on states like Utah which does not have a high quota of PLTW programs and does not have many PLTW certified schools. There is reason to do research in states that are not PLTW program leaders in the nation, such as Utah, and see if PLTW programs in those states are successful and why.

Studies Related to Program Evaluation
The research on program evaluation has suggested some debate about pre-engineering programs falling under the category of technology education (TE). This debate poses a problem in program evaluation because of knowing which standards to use. There are no educational standards for K-12 engineering education. Many argue that The Standards for Technology Literacy: Content for the Study of Technology (National Science Foundation and The National Air & Space Administration, 2007) shown in Appendix C, do incorporate necessities in the field of pre-engineering and does identify successful approaches to teaching engineering at the high school level . For example, "Engineering design challenges include the application of engineering principles to solve real world problems with an active, hands-on approach. Incorporating engineering design challenges into formal coursework is one method of teaching the engineering process through practical application" (Mentzer & Becker, 2010, p. 22).
Evaluation of program success due to enrollment is not easily found. Tech-Prep is a program in which the government awards grants to schools that show a plan for students to achieve 2 years in a high school educational track coupled with 2 years of post-high school in the same track (U.S. Department of Education, 2011). There is research on Tech-Prep which outlines the importance of sufficient students to make the program successful. Brown (1998) stated, "Issues such as recruitment, retention, and dropout rates represent important concepts that should be addressed by efforts to evaluate Tech-Prep initiatives" (p. 1). The research containing this quote is important because its meaning carries over into the supportive programs in secondary education and does preclude that sufficient enrollment is a method for evaluating program success. It also emphasizes the government role and willingness to participate in students' engaging a high school track and continuing that track to conclusion with some sort of postsecondary finish. PLTW provides opportunity for students based on this same philosophy and could be evaluated on the basis of a Tech-Prep model.
A Technology Education Program Evaluation Report (2008) in the state of Missouri used a state-wide survey to find that "Technology Education is currently in a metamorphosis, from relying totally on the ability for teachers to provide students opportunities to create products from the use of metal, wood and communication tools to teaching students how to problem solve to apply skills to a variety of circumstances" (p. 5). The report also notes that partnerships with different STEM organizations have moved Technology Education toward a pre-engineering curriculum in their state using activities-based learning, project-based learning, and problem-based learning or APPBlearning. The reason this report is significant to this study is that it typifies what is happening in states that are implementing PLTW programs. The report does specifically mention PLTW as one of the programs helping instigate change in technology education curriculum across their state. It also encourages continued research to help the program be successful by having acceptable enrollments and exhibiting characteristics which show student achievement and which can be evaluated using the technological literacy standards.
After reading several studies about what constitutes a successful program or school, there were several strands of commonality which were summarized in a metaanalysis by Dagget (2005). In his paper Successful Schools: From Research to Action Plans, he concluded with 10 findings from seven different studies that successful schools incorporate into their belief system. They are as follows: 1. Create a culture that embraces the belief that all students need a rigorous and relevant curriculum and all children can learn.
2. Use data to provide a clear unwavering focus to curriculum priorities that is both rigorous and relevant by identifying what is essential, nice to know, and not necessary.
3. Provide students real-world applications of the skills and knowledge taught in the academic curriculum.
4. Create a framework to organize curriculum that drives instruction toward both rigor and relevance and leads to a continuum of instruction between grades and between disciplines.
5. Create multiple pathways to rigor and relevance based upon a student's personal interest, learning style, aptitude, and needs.
6. Set high expectations that are monitored and hold both students and adults accountable for students' continuous improvement in the priorities identified in #2 above.
7. Sustained professional development that is focused upon the improvement of instruction.
8. Obtain and leverage parent and community involvement successfully in schools.
9. Establish and maintain safe and orderly schools.

Offer effective leadership development for administrators, teachers, parents, and community. (p. 4)
These studies about educational success will drive the CTE interview questions in this study on why PLTW in Utah is successful and identify testable factors which contribute to that success.

Summary
It has been shown that there is a need for programs like PLTW to be implemented for PLTW to be successful in those states. The research done in this project in the state of Utah will help address some of those problems and answer some of the questions about how successful this program is.
The following points characterize this literature review.
 There is a need for pre-engineering programs in secondary schools to promote acquisition and retention of engineering students at the university level.
 While efforts are being made to close it, there exists a gap in the education students leave high school with and the prerequisites needed to be successful in a university engineering program.
 Because of its teaching methods PLTW may help students internalize math and science concepts and better learn them through application and being shown the importance of knowing those concepts.
 PLTW can help students learn complex systems through problem-solving techniques and working in groups.
 There may be many reasons for PLTW implementation in Utah schools and therefore there may be many different goals.
 While there has been some preliminary research done in other states about the perceptions of PLTW, there has not been a lot of research done in discovering how PLTW is performing in Utah and this knowledge is important for the decision-making processes concerning PLTW.
 Successful program evaluation can be achieved through analysis of student enrollment, student achievement, and attainment of program implementation goals.
 Factors that contribute to PLTW program success need to be identified and used.

Research Design and Research Questions
This research used a mixed method design. Both quantitative and qualitative research methods were utilized to answer the research questions. This research was divided into two phases and both phases employed the aspects of qualitative and quantitative inquiry.

Phase I of the study used an interview process to question Career and Technical
Education (CTE) directors (N = 10) in the state of Utah that have PLTW programs in schools in their districts. The interview questions asked CTE directors to identify goals or reasons for implementing PLTW in their schools, and seek perceptions and information related to how they view successful programs. Appendix D shows the CTE director interview questions. Phase I of this study answered the following research questions. In this study, all information was kept confidential and no names of schools or persons were directly linked to the data generated. The data was only identified by the population from which it came.

Population Descriptions and Data Needed to Complete the Study
Four populations that were involved with the PLTW curriculum from the state of Utah were used in this study. The populations included: (a) CTE district directors, (b) administrators of secondary schools that use curriculum from PLTW and who are the most knowledgeable about PLTW, (c) secondary teachers who teach PLTW courses, and (d) counselors in schools that have implemented the PLTW program. The school administrators and counselors used in this study were identified either by CTE directors or by questioning the administrative staff at the school.

Career and Technical Education Directors Data
The CTE director population used in this study was that of secondary school districts in Utah that used the PLTW program in their schools. These 10 districts included Davis, Duchesne, Emery, Granite, Jordan, Weber, Salt Lake, Ogden, Logan, and Canyons. The data collected from this population were from interviews. In this study all 10 CTE directors who had PLTW in their districts were interviewed (see Appendix E).
The reason for doing interviews was to convey the message that this study was important for implementing, maintaining, and sustaining successful PLTW programs in their districts. Personal interviews of CTE directors also improved participation in the research by school administrators, teachers, and counselors because of CTE director support that was established during the interview process. This population was interviewed before the other populations were surveyed so the data could be used to adjust the survey questions in the surveys that were used in Phase II of the study.

School Administrator Data
The second phase of the study included a population of secondary school administrators that have the PLTW program in their schools. The administrators surveyed in this study could be principals or assistant principals depending on who had the most PLTW program knowledge in their school. These administrators were identified from questioning CTE directors, the state specialist, or the school administration and staff. At present, approximately 29 secondary schools in Utah had a class or classes from the PLTW program in 10 districts. Appendix F shows a list of districts that teach PLTW programs complete with schools and teachers. An attempt was made to survey administrators from all 29 schools using an Internet-based survey system. One of the reasons to utilize CTE directors in Phase I was to encourage administrators to complete the survey, which increased the validity of the findings.
Questions asked in the administrators' survey yielded data about the strength of factors suggested by the CTE directors, the review of literature, PLTW, and conversations with research participants that may contribute to PLTW program success.
The data were needed because it generated findings that may help programs become more successful and helped insure success of programs at start up. Data from surveys were one of the driving forces of this study and helped answer the research questions.
Some schools used in this study utilize a "district learning center" where the PLTW program is not housed in their school, but at a different location where several schools in the district may send students. Those administrators were also included in the study as part of the 29 schools because they do have students who participate in the PLTW program.

Teacher Data
The second phase of the study also included a population of teachers who actually taught the PLTW program in a Utah secondary school. There were approximately 33 teachers in the state of Utah who taught courses in PLTW programs in Utah's secondary schools. These teachers were identified using information from the USOE. An attempt was made to survey all of these teachers using an Internet-based survey system.
Questions in the teacher survey rated the strength of the factors suggested by the CTE directors, the review of literature, PLTW, and conversations with research participants that may contribute to PLTW program success. The teacher surveys were exactly the same as those given to school administrators and counselors so that data were compared and contrasted between these populations. However, some of the demographic information collected at the beginning of the surveys was different between the three populations, so the surveys could be filtered as to responses to certain demographics within the population itself.
The data were needed because it generated findings that could be used to help programs become more successful and improve the chances for success at program at start up. Data from surveys were the driving force of this study and answered the research questions posed in Phase II.
Some of the teachers surveyed may work at a learning center and may represent several high schools. These learning center-based teachers were identified by demographic survey questions for data comparison with teachers who do not teach in the learning centers.

Counselor Data
In Phase II of this study, the last population used was counselors in Utah secondary schools that have the PLTW program. Counselors that have the PLTW training were identified using information from the USOE and through communication with individual schools. Using an Internet-based survey system, an attempt was made to survey a counselor at each of the 29 schools that had the most knowledge about PLTW regardless of whether the counselor had the PLTW training or not. Questions in this survey rated the strength of the factors suggested by the CTE directors, the review of literature, PLTW, and conversations with research participants that may contribute to PLTW program success. The counselor surveys were exactly the same as those given to the teachers and administrator so that comparisons and contrasts could be made between the data generated from teacher, counselor, and administrator surveys. These data were needed because it generated findings that may help programs become more successful and improve chances for success of programs at start up. Data from surveys were the driving force of this study and helped answer the research questions posed in Phase II.

Data Collection Instruments and Collection Method
Two different data collection methods and instruments were used in this study. In Phase I, the instrument used with the CTE directors consisted of an in-person or phone interview which was transcribed and coded for analysis. The determination of using an in-person or phone interview was made based on schedules and travel time. An in-person interview was priority but, if a particular CTE director was located many hours of travel away and the trip could not be combined with interviewing other CTE directors then a phone interview was used. Responses from the interview in Phase I were used to add, remove, or adjust the survey questions used in Phase II.
The data instrument used in Phase II of this study completed by school administrators, teachers, and counselors was a written survey that asked questions on factors which may contribute to PLTW program success. The surveys in this phase were administered using SurveyMonkey an Internet-based survey tool.

Instrument Pilot Tests
After obtaining Institutional Review Board (IRB) approval (see Appendix G), the CTE interview questions and the teacher, administrator/counselor instruments questions were pilot tested. The interview questions were tested for clarity and understandability by performing a mock interview with three CTE directors-two from Colorado and one from Wyoming-who were not participating in the study itself. Two of the directors participating in the pilot interviews held PhDs. Interview participants were specifically asked to comment after each question on whether or not the interview protocol tended to procure rich information about the question and if the probing question categories were understandable and justified. Also, the time it took to do each interview was considered.
Because of the distance between these participants, communication was by telephone and e-mail where participants had the opportunity to voice their opinions and criticisms concerning the instrument questions. It was learned from the CTE pilot tests that some minor word changes would make the questions more understandable and a question was raised concerning collecting data by using interviewee perceptions. It was decided that there should be some word changes to improve question clarity. Also, because no data previously existed on this topic, the use of interviewee perceptions-in this case Utah CTE Directors-might be the only way to find the answers to the research questions.
Pilot participants approved the interview questions and probing question categories with a few minor word changes made to the instrument.
The survey instrument pilots were responded to by two school administrators, four teachers, and three counselors in Colorado and Wyoming not participating in the actual study. These states were chosen because they had a lot of similar demographics as Utah and could give a fair assessment of the instrument. Pilot participants were given the surveys to critique the questions. Specifically, after each question participants were given the opportunity to comment on: (a) the survey items being clear and understandable, (b) the demographic information requested being adequate, and (c) ways that could improve the questions' reliability and validity. Participants could comment in a dialog box provided after each question. Feedback from the pilot-testing was discussed with participants by phone and e-mail. From participant feedback it was learned that opinion questions should contain the choice "not sure." Also, only one of the phrases "do you think" or "In your opinion" was used in each opinion question to reduce redundancy instead of using both phrases in the same question. There were also some other small word changes made to increase question clarity. One participant would have liked a question added specifically to teachers concerning how much the administrative support within their school helped with program success. After consideration it was felt that the survey instrument questions should be identical between all participants for comparison and so this question was not added. The PLTW program questions were the same on the questionnaires responses between teachers, administrators, and counselors allowing them to be compared by triangulating the comments and concerns.
Doing these pilots of the interview and survey instruments strengthens this study by insuring that participants fully understand the questions being asked and that the data can be collected in a timely manner. These pilots also add to the study by increasing the reliability and validity of the instruments used.

CTE Director Instrument and Collection Method
The interviews were partially structured and partially unstructured (Gay & Airasian, 1999, p. 221). The structured part of the interview consisted of open-ended questions from the five main categories followed by probing questions (unstructured) designed to elicit more detail by prompting responses not obtained when the main question was answered. It was also requested from the interviewee to state how much they agreed or disagreed with the probing question. The CTE directors from the 10 previously mentioned districts were interviewed. The CTE directors were asked questions from the following categories.

Reasons or goals for implementing PLTW.
3. Schools meeting the goals for originally implementing PLTW. 4. Meanings of success in PLTW programs.
5. Factors that contribute to PLTW program success.
Interview questions and probing questions were developed and validated through discussions and guidance with CTE directors, doctoral research committee members, school administrators, teachers, and a thorough review of the literature. To collect these data, it was necessary to make an appointment with the CTE director by e-mail or by telephone, travel to a place, and select a time of their convenience to conduct the interview. Interviews were also done over the telephone using a conference call set up depending on the logistics previously discussed. The conference call was between me, the CTE director, and anyone else the CTE director wanted to have present. A copy of the questions was made available to CTE directors a few days before the interview accompanied by a letter of information (see Appendix H) so they could be formulating their answers. A reminder was sent out by e-mail a day before the scheduled interview.
Interviews were recorded with the promise of confidentiality. After recording the interviews, they were transcribed and coded for content analysis where common themes were developed by grouping the data and categorizing it.

Administrator/Teacher/Counselor Instruments and Collection Method
Using the review of literature, PLTW, and conversations with research participants surveys were created which were given to individual school administrators, counselors, and teachers who are involved with PLTW programs in Utah public secondary schools. Data concerning success factors from the CTE director interviews was also used to adjust the surveys to strengthen content validity. Demographic questions yielded information about how many students each survey respondent represents. From this the number of students represented by the respondents was calculated and compared to the total number of PLTW students in Utah. Enough school administrators, teachers, and counselors responded to represent at least 46% of the students served by PLTW. This gives the questionnaires sample and size validity. Two weeks after the initial e-mailing of the survey, a follow-up mailing was sent reminding participants that their responses were necessary to make this study representative of them. Two total reminders were sent out at 1-week intervals, making three points of contact for questionnaire participants.
Part I of each population's survey asked for population demographic information. program dynamics such as project-based education and professional development.
Appendices I, J, and K contain the teacher, school administrator, and counselor surveys.
The surveys were given to the school administrator and teacher populations using SurveyMonkey a professional Internet-based survey system. In this survey system the data was collected and percentages generated which rated the strength of each contributing factor. These data were used to drive statistics that showed findings that ranked contributing factors and correlated these findings with population demographics.
The surveys used closed ended questions with an ordinal scale to ask the opinion of each of the factors presented in the questions. At the end of each question is a comment box so the participant can express reasons why their answer was selected if they choose. Using methods suggested by Nardi (2003) in his book, "Doing Survey Research" for guidance, an example of a question using the possible factor of student environmental might be as follows.
1. Classes in the PLTW program use a "hands-on" technological environment with computers and lab equipment as one of its key teaching elements. In your opinion, how many of the students taking PLTW classes in your school primarily take the class in order to take advantage of this type of learning?
More than 75% Most (between 50% and 75% of the students) Some (between 25% and 50% of the students) Few (Less than 25% of the students) Please feel free to comment on this question When engaging in the conversations and literature review that helped create the interview questions used in Phase I of the study, some contributing factors for success were suggested and integrated into the survey for teachers, administrators and counselors.
These factors could contribute to program success by motivating students to initially take a PLTW class, motivate students to continue taking PLTW classes, or promote student achievement in the class. Examples of suggested factors contributing to PLTW program success that were addressed in Phase II of the study included:  The students' interest in the subject matter  The students' family influences  The students' influence from peers  The teacher's competencies or charisma for making the class appealing  The type of credit received for the PLTW class.
 The classroom setting where students could be attracted by a problem-solving technological environment.
 Guidance received from a counselor, especially if the counselor has had the PLTW training.
 Students not informed about the PLTW courses.
 Concurrent enrollment where students may opt for college credit.
 College preparation where students take advantage of PLTW classes to better understand the rigors of a competitive collegiate environment.
 Improvement of student prerequisites, meaning that students achieve better in STEM classes.
 The credentials of the teacher, which may provide better instruction and possibly give the class a more sophisticated theoretical engineering framework.
 Teacher preparation time is insufficient to provide the quality of instruction needed.
The survey instrument was short enough (approximately 25-30 questions) that administrators, counselors, and teachers had time to ponder each question in order to answer it thoughtfully. At the same time, the survey instrument included enough questions to generate the data needed to complete the study without being redundant (Nardi, 2003, p. 65).

Research Validity and Reliability
Validity is the degree to which an instrument measures what it is supposed to measure (Gay & Airasian, 1999). Two areas were considered to achieve content validity in this study. The two areas were item validity in the instruments used and sampling validity making sure that if the entire population cannot be used then a large enough sample is used. Sampling validity, sample size, and the use of pilot testing has already been explained in this study. The following pertains to item validity in the interviews and surveys.
Interviewing is an act of social interaction that usually falls under the heading of qualitative research. Golafshani (2003) stated, "This means such methods like interviews and observations are dominant in the naturalist (interpretive) paradigm" (p. 600). Validity and reliability may not necessarily be viewed separately and are then achieved through the research using terminology such as credibility, transferability, rigor, trustworthiness and quality in qualitative paradigm. Golafshani went on to explain that "To ensure reliability in qualitative research, examination of trustworthiness is crucial" (p. 601) and that trustworthiness may be verified using the technique of triangulation, where convergence of multiple and different sources of information were used to form themes or categories in the study. Triangulation in this study was achieved through the process of transcribing, coding, and organizing the data into categorized themes through multiple inputs (teachers, administrators, and counselors).

Data Analysis for This Study
Data obtained from the CTE director interviews was analyzed by examining the interview recordings. Answers for each of the five questions along with the probing questions in the interview were transcribed and coded into categories or themes for comparison between the respondents. Response themes were reported in the findings along with the number of CTE directors indicating the response. The first four interview questions were designed to separate the districts demographically by asking for the number of schools in the district, the type of community that the district served, how long PLTW classes had been offered in the district, and how many schools in the district were certified by PLTW. Tendencies of the responses with respect to district demographics were noted and highlighted in the findings. Tendencies in the response theme generated from the interview data were reported using descriptive statistics in which the number of CTE directors who responded similarly to a question were calculated. The CTE director interviews also generated a list of possible factors perceived to contribute to PLTW program success. This list was used to ensure that the perceptions of CTE directors were represented in the surveys given to school administrators, counselors, and teachers.
Data from the school administrator, counselor, and teacher surveys was analyzed using descriptive statistics. Demographical information collected in the first part of the survey aided in determining response differences between the different groups of respondents. SPSS was used to generate descriptive statistics about each question with regard to any of the particular respondent groups.
The opinion questions in the survey offer a text box where participants may comment on the question. If opinions were offered they were coded and categorized according to their theme. They were also considered in the findings of the study.
After descriptive statistics were generated for the data, comparisons were made to see how the populations perceive factors that contribute to successful PLTW programs to be the same or different. The ensuing discussion addresses the findings and possible reasons for the statistics. For example: are there differences in program views between CTE directors, school administrators, teachers, and counselors and what are they?

FINDINGS
Chapter IV presents an analysis of the data collected in the study. Data was collected in two different phases using four different instruments. Findings from these instruments were used to answer the research questions. The instrument used in Phase I contained interview questions and the other three instruments used in Phase II were Internet-based surveys. The findings from these instruments were categorized into four sections in Phase I and four parts in Phase II. Parts were used instead of sections in Phase II to match the questionnaires.
In Within each question demographical differences (i.e., district size, school enrollment size, and community status) in the data were addressed between the populations that the directors represent as discrepancies in responses were identified.
Phase II of this study involved using questionnaires to collect data. These data were obtained from surveying teachers that teach PLTW, school administrators that have PLTW classes in their schools, and their associated counselors. In Utah there were 29 public schools (i.e., 22 high schools and 7 junior high schools), 2 charter schools, and 3 applied technology centers that offer PLTW classes. In this study, only school administrators and counselors from all 29 public schools were invited to participate in the questionnaire. Questionnaires were developed for each of the three groups. The questionnaires were similar in nature. The only differences in the instruments were the demographic questions and a question about the support of the other two groups (i.e., how teachers feel about the support of administrators and counselors and so on).
Invitations to participate in the questionnaire with the associated web address were sent by e-mail to all PLTW teachers (N = 33), a school administrator in each PLTW school (N = 29), and also a counselor from each of those PLTW schools (N = 29). In this study, 23 teachers (70%), 18 school administrators (62%), and 12 counselors (41%) responded to the questionnaire. All 33 PLTW teachers in the state from all the schools were invited to participate. Some teachers move between schools or teach classes that have students in them from more than one high school. This accounts for 33 teachers serving more than 33 schools that have PLTW.
Demographical data collected from the questionnaires for teachers, administrators, and counselors will be examined in Part I under Phase II Results in this chapter. Findings from the questionnaires for each population about why PLTW is successful in their schools will be examined in Part II. Findings from the questionnaire for each population about why students enroll in PLTW classes will be examined in Part III and findings from the questionnaires about the PLTW program's ability to promote students' achievement will be addressed in Part IV.
Statistical analysis of the questionnaire data was accomplished by calculating descriptive statistics (i.e., in particular the mean, and standard deviation) for each question. Within the three main groups (i.e., teachers, administrators, and counselors) filtering the demographic questions generated data which could be separated within the groups into subgroups such as how participants from high schools responded vs. junior high schools, and how participants from large schools over 1,000 responded versus smaller schools under 1,000.

Phase I Results
Findings from Phase I of this research are organized into and examined in the following sections:

Interview Questions 1-4: Director Demographics
In Phase I of this study, 10 directors were identified to be interviewed because they represent the 34 Utah secondary schools, charter schools, and applied technology centers which offer PLTW classes. This represents almost all of the secondary students in Utah which take pre-engineering classes. Of these, 29 schools serve urban communities and five of these schools serve rural communities as specified by CTE directors in the interviews.
In this study only five of the 34 schools were PLTW certified and these schools were all located in urban areas. It also should be noted that some of the larger districts send their students to a central technology school where possibly only one school is certified but represents many high schools in that district. The first four sections of this chapter will report the findings so comparisons can be made between responses of the different demographical groups. Data on director demographics is shown in Table 4.1. 2 Note. Some of the CTE Directors interviewed had students attending applied technology centers and may have been counted by more than one CTE Director as shown in these numbers. There are 29 public secondary schools, 3 applied technology centers, and 2 charter schools which have the PLTW program.

Interview Question 5: Scholastic Ability of Students
Question 5 of the interviews asked directors to comment on what scholastic groups of students they felt were served by PLTW. Six (60%) said that PLTW attracted middle-to-upper-scholastic level students. A good "B" student was mentioned several times in the interviews by four interviewees from larger districts. Only one interviewee, who was from a larger district, suggested that PLTW's main purpose was to prepare students specifically for engineering programs at major universities and mentioned that they were looking for the top 15% high-end students that were on an engineering track.
Three participants said that PLTW was not an elitist program but provided opportunities for students to engage in either engineering or engineering technology programs in making career decisions. One respondent said they would take all students regardless of their academic performance history because success may be found in PLTW classes through the types of curricular motivators used.

Section Two: Implementation and Goals
Question 6 asked the directors to comment on the reasons why PLTW was implemented into their schools. Question 7 asked if PLTW implementation goals were presently being met. After analyzing the interview data, the top two reasons for implementing PLTW as given by seven of the directors were: (a) they liked what the PLTW plan brought to the district with high quality curriculum and its professional development for teachers and counselors better than other plans, and (b) they felt like PLTW gave students an "outlet" (career path) in CTE for furthering their technical education into many different post-secondary schools and careers. Four more reasons for implementing PLTW were given by at least five directors which were: (a) the district wanted to follow economic trends and was acting on national career data, (b) they wanted to take math and science skills and apply them in a mechanical environment utilizing design concepts, (c) they wanted a curriculum which was continually updated and current, and (d) they wanted to provide a feeder to technical education centers. This last finding does coincide with providing students an outlet as mentioned in the first two findings; however it is mentioned here because three of the directors also specifically mentioned the pathway into their technical centers.
Comments made during the director interviews were mixed when asked if the present PLTW programs in their schools are meeting the goals set at the time of implementation. Three of the interviewees said, "No" because of the difficulty in finding the right instructor or poor alignment between universities and their PLTW program.
Finding the "right kids" for the program was also mentioned as being a problem.
Directors indicated that "right kids" meant students that wanted to learn the concepts taught in PLTW classes and use those concepts to further their life either in their career choice or post-secondary education. Seven directors responded saying, "Yes" that their programs are meeting the goals set at the time of implementation.
One change noted by at least six directors concerning PLTW certification suggested that PLTW seems to be more likely to work with schools that only offer one or two PLTW classes and did not necessarily intend on becoming certified. In the past, directors felt that PLTW exerted pressure on schools to become certified. PLTW seems to have realized that many Utah schools, particularly smaller ones, do not have the resources to provide all the PLTW classes necessary for the school to become certified. This change in directives gives more opportunity for smaller high schools to offer some PLTW classes in their schools.

Questions 6 and 7: Probing Responses
When asked probing questions following Question 6 about goals or reasons why PLTW was implemented into their districts, over half of the directors interviewed "agreed" or "strongly agreed" that PLTW provided : (a) a way to promote professional development among CTE teachers, (b) a practical opportunity to introduce "preengineering" into the school's curriculum, (c) a high quality pre-engineering program, (d) a way to strengthen the school's STEM curriculum, (e) a way to partnership school, industry, and community to increase educational opportunity for students, (f) student pathways to university programs, and (g) a way to give students an opportunity to get concurrent college credit while in high school. While presenting the probing questions to the directors during the interview, the directors were specifically asked if they strongly agreed, agreed, neither agreed nor disagreed, disagreed, or strongly disagreed with the probing question and prompted to comment further on their response. For rank order purposes the mean was calculated for the probing questions in all tables shown in this research by assigning the following numbers to the responses: 5-strongly agree (SA), 4agree (A), 3-neither agree nor disagree (N), 2-sisagree (D), and 1-strongly disagree (SD).
The rank order of the probing responses for interview question 6 are shown in Table 4.2.
Director responses to the probing questions following question 7 about PLTW presently meeting program goals were varied. At least seven directors agreed or strongly agreed that: (a) they felt their program was successful within the confines of PLTW or meeting national requirements, and (b) enrollment in PLTW courses has been satisfactory to justify the course offerings. Most of the directors did not feel that program goals or opinions about those goals had changed since implementation. Also, personnel changes have not tended to change the goals for implementing the programs in their schools. One director declined to comment on goals set at the time of implementation because their newly formed district inherited the program and they were deciding exactly what those goals were. The results of the probing questions about PLTW presently meeting implementation goals are shown in Table 4.3.

Section Three: Attributes of Successful Programs
In responding to the interview question 8, "What do you feel are the attributes of a successful PLTW program?" Nine directors said that the right instructor was the key. The instructor needed to have all the attributes of knowing how to engage the students in their class with good teaching skills, but also needed content skills in the application of math and science relative to the projects that the students were working on. One director used the phrase, "Their element (meaning teachers) makes the program fly or die." Five interviewees said that good "buy-in" from the district was very important to program success meaning that the district had to provide the necessary program administrative support and funding to ensure a viable existence.
The probing questions asked about question 8 indicated that all the directors either agreed or strongly agreed that programs are successful if: (a) they have the ability to attract students and maintain adequate enrollment, (b) they have the ability to promote student achievement, (c) they are perceived to have met the goals of implementation, (d) they are meeting present program goals, (e) the program produces desirable student outcomes, (f) the program creates good public relations, and (g) the program platform brings to the school a way to develop partnerships between school, community, and industry. The rank order response results for these probing questions are displayed in Table 4.4.

Section Four: Factors That Contribute to Program Success
Interview question 9 asked directors to identify contributing factors of PLTW  In the first probing question, interviewees were asked if student interest in the subject matter was a factor for success. All 10 directors agreed (i.e., six agree and four strongly agree) that to promote program success efforts should be made to insure that students who are interest in PLTW classes should be sought after and enrolled.
Comments made during the interview indicated that while this sounds like a simple thing, advertising the class and making sure students have class information can be very difficult to do. One Director indicated that even though they advertise pre-engineering classes by posting them on bulletin boards and including them in announcements, there seem to still be a lot of students in their school that do not know they offer preengineering classes. Getting information to students is important.
Informing family members and peers about PLTW classes was found to be a contributor to program success. All 10 directors agreed (i.e., eight agree and two strongly agree) that family members and student peers are major factors that influence the success of PLTW because more students enroll to be with their friends or because of what their friends may have said concerning the class. Also, a sibling or parent can influence enrollments because of what they tell perspective students.
All 10 directors agreed (i.e., one agrees and nine strongly agree) that providing counselor PLTW training is a factor of program success. Counselor training is a large contributor because it helps students to be better informed about the classes they take and how those classes can help them in their life choices.
All 10 directors agreed (i.e., two agree and eight strongly agree) that if taking PLTW classes would generate more required math or science credit for students, then enrollments would increase. Because of the application of math and science concepts in PLTW classes, the underlying feeling in the interviews was that more PLTW classes than just principles of engineering should carry a required math or science credit. However, one of the interviewees did say that if there were students in PLTW classes that needed math credits, then perhaps they should not be in the PLTW class because of the amount of math the class requires and that their performance in the class may be hindered because of lack of skill.
All 10 directors agreed (i.e., three agree and seven strongly agree) that the "hightech" labs that PLTW offered along with project-based instruction were appealing to a lot of students and that the labs were a factor in the success of the program. It was mentioned that many students more effectively learn by doing. Using the "hands-on" approach helps students to achieve with a method rarely used in mainstream academic classes.
Nine directors agreed (i.e., seven agree and two strongly agree) that providing a way for students to obtain university credit for taking PLTW classes is a feature of the program that makes it more successful. There was a comment made that students attending a local university from their high school thoroughly understood the material in classes they took for the first year. One of the directors did disagree with this because his district was not taking advantage of college credit. It was also pointed out by the same individual that the elective credit given for PLTW courses had little overall value even if the student decided to go to that particular university because of the rigorous classes required for a degree.
One theme that was common through all the directors during the interview was how important the teacher was to the program. All 10 directors strongly agreed that providing a teacher who is knowledgeable about the application of PLTW class content and had good teaching skills was crucial to the success of the program. It was mentioned many times that for a PLTW program to be successful, you should find the right teachers.
The perception of PLTW classes improving STEM education in general was predominant among interviewees. All 10 directors agreed (i.e., six agree and four strongly agree) that part of the success of their PLTW programs was in improving education in other STEM subjects. The PLTW classes gave relevance to learning; particularly math and science.
When asked if they thought the type of credentials the teacher had made a difference in the success of PLTW responses were mixed. Three of the directors strongly agreed that credentials made a difference, and five agree. One response was neither agrees nor disagrees, and one response was disagree. The underlying theme in the importance of teacher credentials as a factor of program success was that the personal traits of the teacher usually manifested itself in the teaching more than knowledge of the content area. It was mentioned that math and science teachers sometimes struggled with the application of concepts. Skilled and technology science (applied technology) teachers sometimes lacked depth in math and science content. The teacher credential that was the most recommended was a credential in technology and engineering science education (applied physics and pre-engineering, USOE, 2009).
All 10 directors agreed (i.e., three agree and seven strongly agree) that informing students about PLTW classes was a factor in making the PLTW program successful. The theme from this probing question was; it cannot be assumed just because a school offers a class that students know it is a registration choice, and what the class offers. Directors mentioned it takes a lot of advertising and counselor training to get the word out about what an offering entails.
The last two probing categories had all directors agreeing (i.e., five STRONGLY AGREE AND FIVE AGREE) that success also depended on the districts willingness to properly fund the PLTW program in their schools and that success may be hindered by students not having enough room in their schedules for five PLTW classes to complete the program. Directors also said that program success does depend on students being able to complete the program. Response rates on factors that contribute to successful PLTW programs are tabulated in Table 4.5.

Phase II Results
In Phase II of the study questionnaires were sent to teachers, administrators, and counselors in Utah public schools which have PLTW programs. This also includes teachers from technical schools which serve many high schools at a central location. One of the goals of this study, which adds strength to the research, was to identify any response differences between the demographically different populations of respondents. The populations consisted of teachers, school administrators, and counselors. In addition to these three populations the questionnaire data also filtered responses between participants that were from high schools and junior high schools and between schools with student populations over a thousand and student populations under a thousand. Statistical differences between these group means were analyzed and reported whenever they were statistically significant.
One of the questions on the questionnaire asked teachers the number of PLTW

Part II: Examining Program Success
In Part II of the questionnaires all participants were asked seven questions pertaining to the success of PLTW in their schools. Each of the questions in this section consisted of a statement in which the participant was asked to select whether they strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree.

Calculating Group Mean Differences
In Part II of the findings, in order to establish a mean for each groups responses on the questionnaire, each participant response was assigned a number. "strongly agree" was assigned a 5, "agree" was assigned a 4, "neither agree not disagree" was assigned a 3, "disagree" was assigned a 2, and "strongly disagree" was assigned a 1. The mean for each question was calculated, which was between 1 and 5. For example, question 7 on the teacher questionnaire showed that 15 teachers chose "strongly agree," seven teachers chose "agree" and one teacher chose "neither agree nor disagree." The mean for this groups response would be (5*15 + 4*7 + 3*1) / 23 = 4.61.
In order to interpret the group response means   x for these questions, the following scale was created; "strongly agree" if the mean score was greater than 4.5, "agree" if the mean score was greater than 3.5 and less than or equal to 4.5, "neither agree nor disagree" if the mean score was greater than 2.5 and less than or equal to 3.5, "disagree" if the mean score was greater than 1.5 and less than or equal to 2.5, and "strongly disagree" if the mean was 1.5 or less. Therefore in the above example a mean of 4.61 would represent "strongly agree" on the questionnaire.
Many of the questions had comments made by the participants. These comments are listed in Appendix M. If the comment made a viable difference in the results of the question, it is mentioned on a question-by-question basis.

Part II Questionnaire Results
The following sections contain the findings for each of the questionnaire questions. There were a different number of the demographic questions asked to each population. Therefore the question numbers for identical questions in Part II of the questionnaires were different between the three populations. Identification of matching questions from the questionnaires is accomplished by using "T" to signify teachers, "A" to signify administrators, and "C" to signify counselors followed by the question number.
Therefore, a question having (T5, A5, C7) for an identifier would be question number 5 from the teacher questionnaire, question number 5 from the administrator questionnaire, and question 7 from the counselor questionnaire, which would be the identical question on each.
The first questions (T7, C9) in Part II of the questionnaires was given only to teachers and counselors. Teachers (N = 23) and counselors (N = 12) were asked to consider the statement: "A supportive school administration is very important to the success of PLTW in my school." The results of the responses are tabulated in Table 4.7.
The results yielded a teacher mean of 4.61, a counselor mean of 4.92 and an overall participant mean of 4.71. These data show that the respondent group as a whole strongly Neither agree nor disagree were asked to respond to the statement, "A dynamic teacher is very important to the success of PLTW in my school." Table 4.9 displays the response rates to the statement.
The results yielded an administrator mean of 4.67, a counselor mean of 4.83, and an   There were 25 (48.1%) of the participants that responded "strongly agree" and 20 (38.5%) that responded "agree" for the statement. Table 4.10 displays the response rates to the statement.
The next questions (T10, A10, C11) in Part II on the questionnaires asked    Give students university options  Table 4.12. One administrator did not respond.
The overall participant mean is 3.98. These data show that the respondent group as a whole agreed that the PLTW program was meeting their goals of implementing the program. There were 16 (30.8%) of the participants chose the response strongly agree and 24 (46.2%) chose agree for the statement. The mean for participants from schools with less than 1,000 students is higher than the mean from participants from schools with 1,000 or more, showing that participants from schools with less than 1,000 students are more likely to strongly agree with the statement. The results for calculating the mean for this question, along with the standard deviation for each group is shown in Table 4.13.
The final questions (T12, A12, C13) in Part II of the questionnaires asked teachers (N = 23), administrators (N = 18), and counselors (N = 12) to respond to following statement: "Utah's PLTW affiliate university has been able to adequately meet our program needs." A teacher mean of 3.04, an administrator mean of 3.56, a counselor mean of 3.67, and an overall participant mean of 3.36 shows that the mean participant response was just a little above "neither agreed nor disagreed" that the PLTW affiliate university in Utah is meeting their program needs. It should be noted however that 45.3% of the participants did agree with the statement, and only 18 of the 53 respondents chose the response "neither agree nor disagree." Fifteen percent disagreed or strongly disagreed, thus bringing down the average. The response rates are shown in Table 4.14. scholastic status, two types of questions are asked in this part of the questionnaire.
The first type of question is similar to the questions asked in Part II of the questionnaire, which asked participants to respond to a statement as to whether they agree, disagree, or neither and how strong their opinion is. These questions were interpreted the same way they were in Part II.
The second type of questions in Part III asks participants to give their opinions as to how many students in their PLTW classes, as a percentage, they feel meet the requirements of the questionnaire question. The response choices in these questions were: (a) more than 75%, (b) most (between 50% and 75% of the class), (c) some (between 25% and 50% of the class), (d) few (less than 25% of the class), and (d) not sure. To facilitate better understanding of how each group responded it was necessary to calculate a mean response for each. In order to calculate the mean for the responses, numbers were assigned to the responses as follows: (a) "more than 75%" was assigned a 5, (b) "most" was assigned a 4, (c) "some" was assigned a 3, (d) "few" was assigned a 2, and (d) "not sure" was assigned a 1. For example, on question 14 of the teacher questionnaire, eight teachers selected "more than 75%," seven teachers selected "most" (between 50% and 75%) , four teachers selected "some" (between 25% and 50%), two teachers selected "few" (less than 25%), and one teacher selected "not sure" the mean for this group would be (5*8 + 4*7 + 3*4 + 2*2 + 1*1) / 22 = 3.86.
In order to interpret the group response means   x for these questions, the following scale was created; "more than 75%" if the mean was greater than 4.5, "most" if the mean was greater than 3.5 but less than or equal to 4.5, "some" if the mean was greater than 2.5 but less than or equal to 3.5, "few" if the mean was greater than 1.5 but less than or equal to 2.5, and "not sure" if the mean was less than or equal to 1.5.
Therefore, in the above example a mean of 3.68 would represent "most" (between 50% and 75%) on the questionnaire. Results from this question are shown in Table 4.15 Participants overall most frequently choose "A" students to be the most likely to be attracted to and benefit from taking PLTW classes with a response rate of 43 (81.1%). The response of "B" students also had a high response rate of 40 (75.5%) showing that all groups feel "A" and "B" students are most likely to be attracted to and benefit from PLTW classes. All groups had response rates lower than 60% in the "C" category with less than 17% for the "D" category. Two teachers and two administrators choose the category of, "Scholastic indicators do not matter." The next questions (T14, A14, C15) in Part III asked teachers (N = 22), administrators (N = 17), and counselors (N = 12) the question, "In your opinion, how many students in your PLTW classes primarily took the class because they were genuinely interested in the subject?" One teacher and one school administrator did not respond to this question. The results of the responses are tabulated in Table 4.16. The results yielded a teacher mean of 3.86, an administrator mean of 3.94, and a counselor mean of 4.08. The overall participant mean was 3.94. These data show that the mean response fell at just below "most (between 50% and 75% of the class)" of the students in PLTW classes took the class because they were genuinely interested in the subject. There were however, 21 (41.2%) of the respondents that choose the response of "more than 75%." The next questions (T15, A15, C16) in Part III asked teachers (N = 22), administrators (N = 16), and counselors (N = 12) the question, "In your opinion, how many students in your PLTW classes primarily took the class because of the influence from family members?" One teacher and two administrators failed to respond to this question. The results of the responses are tabulated in Table 4.17. The results yielded a teacher mean of 2.50, an administrator mean of 2.94, and a counselor mean of 3.00. The overall participant mean was 2.76. These data indicate that the majority of the participants chose the response "some (between 25% and 50% of the class)" of the students in PLTW classes took the class because of the influence of family members.
There were 31 (62.0%) of the respondents that choose this response. The next questions (T16, A16, C17) in Part III asked teachers (N=22), administrators (N = 17), and counselors (N = 12) the question, "In your opinion, how many students in your PLTW classes primarily took the class because of the influence from their peers?" One teacher and one administrator did not respond to this question.
The results of the responses are shown in Table 4.18.
The results yielded a teacher mean of 2.64, an administrator mean of 2.65, and a counselor mean of 2.67. The overall participant mean was 2.65. These data indicate that the majority of the participants chose the response "some (between 25% and 50% of the class)" of the students in PLTW classes took the class because of the influence of peers.
There were 30 (58.8%) of the respondents that choose this response.
The next questions (T17, A17, C18) in Part III asked teachers (N = 22), administrators (N = 17), and counselors (N = 12) the question, "In your opinion, how many students in your PLTW classes primarily took the class because they like the  The next questions (T19, A19, C20) in Part III asked teachers (N = 22), administrators (N = 17), and counselors (N = 12) the question, "In your opinion, how many students in your PLTW classes primarily took the class to take advantage of the "hands-on" technological learning environment?" One teacher and one administrator did not respond to this question. The results of the responses are tabulated in Table 4.21. An overall participant mean of 3.60 indicates that on average participants thought that "most (between 50% and 75% of the class)" of the students in PLTW classes took the class to take advantage of the "hands-on" learning environment. There were 17 (33.3%) of the respondents that chose "more than 75%" for their answer. The mean for participants from junior high schools is higher than the mean from participants from high schools.
Participants from junior high schools more frequently choose "more than 75%" for their answer. The results for calculating the mean for this question, along with the standard deviation for each group is shown in Table 4.22.
The next questions (T20, A20) in Part III asked teachers (N = 22), and administrators (N = 17) to respond to the question, "In your opinion, how many students in your PLTW classes primarily took the initial class because of counselor guidance?" One teacher and one administrator did not respond to this question. The results of the responses are tabulated in Table 4.23. The data shows that the respondent group as a whole most frequently chose "some (between 25% and 50% of the class)" of the students in PLTW classes took the class initially because of a guidance counselor. There were 19 (48.7%) of the respondents who chose this response. The results yielded a teacher mean of 2.68 and an administrator mean of 2.88. The overall participant mean was 2.77.    These data indicate that collectively participants tended to think that "some (between 25% and 50% of the class)" of the students in PLTW classes took the class college credit.
There were 15 (30%) of the respondents that choose this response on the questionnaire.
The next questions (T23, A23, C23) in Part III of the questionnaires asked teachers (N = 21), administrators (N = 17), and counselors (N = 12) to respond to; "One component of the PLTW program is to better prepare secondary students for the rigors of college engineering classes or training for a highly technical career. In your opinion, how many of the students in your PLTW classes primarily took the class because they plan on using the knowledge and experience gained for college and career preparation?" Two teachers and one administrator did not respond to this question. The results of the responses are tabulated in Table 4.26. The results yielded a teacher mean of 3.10, an administrator mean of 3.59, and a counselor mean of 3.58. The overall participant mean was 3.38. These data show that the teacher and counselor participants most frequently thought that some (between 25% and 50% of the class) of the students in PLTW classes took the class for college and career preparation. There were 18 (36%) of the total respondents that choose this response on the questionnaire. However, 17 (34%) respondents marked "most (between 50% and 75%). The administrator mean of 3.59 and the counselor mean of 3.58 were both higher than the teacher mean of 3.10. These data indicate that administrators more frequently thought that most (between 50% and 75%) of the students in PLTW classes took the class for college and career preparation than teachers. The results for calculating the mean for this question, along with the standard deviation for each group is shown in Table 4.27.
The next questions (T24, A24, C24) in Part III asked teachers (N = 22), administrators (N = 17), and counselors (N = 12) to respond to the question, "In your opinion, how many of the students in your PLTW classes primarily took the class to improve their achievement in math and science?" One teacher and one administrator did not respond to this question. The results of the responses are tabulated in Table 4.28. The results yielded an overall participant mean of 2.47. These data show that the participants most frequently marked that few (less than 25% of the class) of the students in PLTW classes took the class to improve performance in their math and science classes. There were 25 (49.0%) of the respondents that choose this response on the questionnaire.
However, the counselor mean of 3.08 was higher than the teachers' mean of 2.23 As seen in Table 4.28 counselors were more likely to mark that "some (between 25% and 50%)" of the class took the class to improve performance in their math and science classes. The results for each subgroup in calculating the mean for this question, along with the standard deviation for each subgroup is shown in Table 4.29.
The next question (T25, A25, C25) in Part III asked teachers (N = 21), The next questions (T26, A26) in Part III asked teachers (N = 23) and administrators (N = 17) to consider the statement: "Counselors in our school play a major role in convincing students to take other PLTW classes after taking the initial class." One teacher and two administrators did not respond to this question. The results of the responses are tabulated in Table 4.31. The results yielded a teacher mean of 3.38, and an  administrator mean of 4.07. The overall participant mean was 3.67. These data show that teachers and school administrators agreed that counselors were perceived to play a major role in convincing students to take another PLTW class after taking the initial class.
There were 24 (63.3%) of the respondents that choose this response on the questionnaire.
The final questions (T27, A27, C26) in Part III of the questionnaires asked teachers (N = 21), administrators (N = 14), and counselors (N = 11) to respond to the following question: "In your opinion, how many of the students in your PLTW classes will complete the PLTW program by taking at least three PLTW classes?" Seven respondents came from schools that did not offer more than two PLTW classes. The results of the responses are tabulated in Table 4.32. The results yielded a teacher mean of 3.14, an administrator mean of 2.79, and a counselor mean of 2.91. The overall participant mean was 2.98. These data show that the respondent group as a whole indicated that "some (between 25% and 50% of the class)" of the students in PLTW will complete the program in their school by completing at least three PLTW classes. There were 18 (39.1%) of the respondents who choose this response on the questionnaire and 13 (28.3%) who chose "few (less than 25%)."

Part IV: Examining Student Achievement Factors
Student achievement was identified in Phase I of this study as a key factor in PLTW success. The questions from this section were about identifying factors that are perceived to promote student achievement in PLTW classes. The questions in Part IV were written to elicit a response to a statement with agreement and disagreement. An exception would be the first question, which asks participants to select teaching credentials that they feel would promote student success the most. The same strategies will be used to present the findings from the questions as used in Parts II and III.
The first questions (T28, A29, C28) in Part IV of the questionnaires asked teachers (N = 23), administrators (N = 17), and counselors (N = 12) to respond to the statement: "Generally speaking, which of the following teaching credentials do you think would be most likely to enhance student achievement in PLTW classes (check all that apply)?" One administrator did not respond to the question. The results of the responses are tabulated in rank order in Table 4.33.
As can be seen from  The  Strongly disagree 0 0 0 0 0 0 0 0 well on the end of course exams provided by PLTW" One administrator did not respond to the question. The results of the responses are tabulated in Table 4.37. The results yielded a teacher mean of 3.13, an administrator mean of 3.18, and a counselor mean of 3.08. The overall participant mean was 3.13. These data indicate that opinions seem to be split between agreeing with the statement and disagreeing with it. Of the group as a whole 22 (42.3%) did "agree" with the statement and 17 (32.7%) did disagree with the statement while only 12 (23%) chose the overall participant mean response of "neither agree nor disagree." The next questions (T32, A33, C32) in Part IV asked teachers (N = 23), administrators (N = 17), and counselors (N = 12) to consider the statement, "Student achievement is greatly enhanced because of the local partnerships PLTW promotes which link the school to the community as an additional resource and opens pathways for student careers." One administrator did not respond to the question. The results of the responses are tabulated in Table 4.38. The results yielded a teacher mean of 3.43, an  administrator mean of 4.12, and a counselor mean of 4.33. The overall participant mean was 3.87. These data show that the respondent group as a whole most frequently agreed that student achievement was greatly enhanced because of the local partnerships PLTW promotes, which link the school to the community as an additional resource and opens pathways for student careers. There were 25 (48.1%) of the respondents that selected the response "agree" and another 14 (26.9%) that "strongly agree." All together about 75% of the total participants agreed or strongly agreed with the statement.
The final questions (T33, A34, C33) in Part IV of the questionnaires asked teachers (N = 23), administrators (N = 17), and counselors (N = 12) to consider the statement, "Student achievement is greatly enhanced because of the commitment PLTW exhibits towards counselor training to promote equitable learning." One administrator did not respond to this question. The results of the responses are tabulated in Table 4.39. An overall participant mean of 3.67 indicates that survey participants tend to agree that student achievement is greatly enhanced because of the commitment PLTW exhibits towards counselor training to promote equitable learning. There were 27 (51.9%) of the participants that chose agree for this statement. However, the counselors' mean was higher than the teachers. Also, the administrator's mean was higher than the teachers.
Both counselors and administrators tend to "agree" with the statement, while a teacher mean of 3.17 indicates that teachers tend to be more neutral. The results for calculating the mean for this question, along with the standard deviation for each group is shown in Table 4.40.

Summary
This chapter presented the findings gathered by the interview and questionnaire instruments. Each question was presented along with the findings for that question.
Demographic questions allowed the data to be segregated by subgroups and tabulated according to the size of the school and type of school. The data were also presented showing differences between teachers, school administrators, and counselors. Any mean differences between these populations were reported in the findings. Interpretation of the findings and discussion about them are shared in Chapter 5.

DISCUSSION, RECOMMENDATIONS, AND SUMMARY
The purpose of this study was to examine PLTW program success by identifying controllable factors which may be considered at the time of PLTW program initiation or program evaluation. Conclusions drawn from the findings from this research will be presented in this chapter, which contains the following four sections: 1. Overview of the Study-This section will provide a brief description of the study and how it was accomplished.
2. Discussion-This section will present the research questions and discuss them in relation to the findings.
3. Recommendations-This section will highlight the main findings of the study and present recommendations for those considering, implementing, or improving PLTW classes or programs. Also, recommendations for further research will be presented in this section.

Overview of the Study
PLTW was implemented in Utah in 1990 as an effort to improve education in science, engineering, technology, and math courses. Since 1990, its programs have been implemented into 29 public secondary education schools. The purpose of this study was to discover what characteristics are associated with successful PLTW programs.
Specifically this study identified reasons for implementing PLTW into Utah schools, described the aspects associated with successful PLTW programs, and identified perceptions of the factors which lead to successful PLTW programs. The populations of the study included CTE directors in the state of Utah with PLTW programs in their districts, teachers in Utah who teach PLTW classes, and school administrators and counselors that have PLTW programs in their schools.
The study was conducted in two phases. Phase I of the study involved collecting data from the CTE directors in the state of Utah who had PLTW classes or programs in their districts. An interview instrument was developed, pilot tested, and then given to all the CTE directors in the State. The instrument collected data to find out why PLTW was implemented in their districts, to identify what they believed were the characteristics of a successful PLTW program, and to find out what factors they believe contribute to that success.
In Phase II of the study, information collected from the CTE director interviews, the literature review, and conversations with other people (e.g., the Utah State Technology and Engineering Education Specialist, industry leaders, and other STEM teachers) involved with PLTW programs guided the development of questionnaires that were administered to teachers, administrators, and counselors to find out their perceptions about what factors contribute to promoting successful PLTW courses and programs. The questionnaires were administered using SurveyMonkey an Internet-based survey instrument.
In this study, all PLTW teachers that teach PLTW classes in Utah were invited to participate in the questionnaire. One school administrator and one counselor from each school that had PLTW were also invited to participate in the questionnaire. There were 23 (70%) teachers, 18 (62%) school administrators, and 12 (41%) counselors who responded to the questionnaires.
Originally in the development stages of this study, thoughts were given to determine PLTW program success by finding out how many students actually became engineers or pursued careers in STEM. However, pursing these avenues of research was not practical because either it was too difficult to find data on students who had gone through the PLTW program or the data did not exist. Even if the students could be found, information gathered from the students might be too subjective as to whether or not PLTW was the sole reason for them being where they were, either in a university engineering program or an engineering-based career. Therefore it was determined that the only way to find out if PLTW programs were successful in Utah was to ask the people who were directly involved with planning and administering the program. These people included CTE directors, school administrators, teachers, and counselors. Through their contacts with the PLTW community by collaborating with parents, students, each other, and PLTW, they had the best perceptions about what factors contribute to program success.

Discussion
In Phase I of this study, it was found through interviewing CTE directors that two overarching themes emerged from the data through coding and categorization, which were considered to be necessary for PLTW program success. The first theme was related to how many students were enrolling in the classes. The classes in the program need to have enough student enrollments to justify the cost of offering it. The budget formula in Utah is such that the number of students in a class determines the amount of funding the school receives to pay for salaries, facilities, and management. Therefore, if not enough students are in the classes, they are offered at a loss and the budget will have to be made up in a different way.
In some cases classes may be offered at a loss if the class is determined to be sufficiently valuable to those few students who take it. For example a PLTW program may have satisfactory enrollments overall, but in the capstone advanced class enrollment may be less than what is necessary to justify the class. However, the school program needs the advanced class and therefore the advanced class will be offered to students and conducted with insufficient enrollment because it is of enough value to students for program completion.
The second theme that emerged was related to perceptions about students achieving academically. The CTE directors felt that students must take away something from the class that is valuable for them in life, either occupationally or domestically.
Academic achievement can be determined in schools through testing and observation.
Also, academic achievement monitoring can be mandated from a state level through core standards and state mandatory tests. Academic achievement can also be determined through concurrent enrollment and end of class testing such as those used by PLTW.
In the following sections each research question will be presented. The findings from Chapter IV including the two overarching themes mentioned above will be used to discuss and answer each question. The interviews from Phase I will be used to answer research questions one through four and the questionnaires from Phase II will be used to answer research questions five through seven.

Research Question #1
The first research question was asked of the CTE directors: What do CTE directors in Utah perceive as the goals or reasons that the PLTW program was originally implemented into their districts? The findings seem to reveal that CTE directors believe that the PLTW programs in their schools were established to introduce a high quality secondary pre-engineering program which included professional development to help teachers with state-of-the-art techniques in teaching engineering concepts for students which had an aptitude for achieving academically. They also wanted a program which gave students an outlet in engineering and technology education where students could participate in a pathway that could lead to a career in engineering or engineering technology by forming partnerships between schools, industry, and community.
Implementers wanted a program that coincided with the national and economic trends that were affecting education and which was compatible with math and science where it could possibly help boost core test scores.
In this study, the CTE directors believed PLTW was implemented for many reasons. It is interesting to note that the highest reason was to "improve teacher training by providing professional development." It appears that this reason may have been selected first because the directors value quality teaching. Also, this is in keeping with recent efforts in Utah aimed at improving teaching by providing professional development to implement the Utah State Common Core Curriculum in STEM subjects.
In the CTE director's interviews it was mentioned by several directors that new programs implemented by schools in their district should provide extensive training for teachers.
Another reason for training teachers could be that CTE directors feel that the methods of instruction need to change. Traditional "stand-and-deliver" may need to be replaced with more discovery-project-based educational methods of instruction. The findings also showed three other strong reasons for PLTW program introduction that included the following: introduce pre-engineering into their schools' curriculum, gaining a perceived high quality pre-engineering program, and strengthening the schools' STEM curriculum.
The mean value range between these three factors was 0.4. This seems to show that all three reasons are valuable and important for implementation. Perhaps CTE directors want high quality pre-engineering programs with trained professional teachers in their schools where the classes integrate well with other STEM courses. This may also be in keeping with President Obama's push to increase STEM education. The findings from Chapter IV presented in Figure 5.1 shows a bar graph of the reasons why PLTW was implemented into Utah secondary schools by rank order.
Forming partnerships between schools, industry, and the community also ranked high with an approval mean of 4.0. This seems to show that CTE directors believe that schools should not be isolated islands of institution but should be collaborating with all the educational players. The reason for this could be that CTE directors recognize that opportunity for students increases when a partnership with collaboration exists between public secondary schools, industrial organizations, and the local community. CTE directors could also believe that PLTW is a good fit with Professional Learning Communities where one of the key elements is collaboration between all the members to discuss the needs of students.
While believing these are still positive reasons for implementing PLTW programs, CTE directors did not seem to think that sending more students to university engineering programs and having a way for students to get university concurrent enrollment ranked quite as high as the afore mentioned reasons. The reason for this could be that CTE directors are very concerned with the education that students are receiving in their schools and this is more important than contributing to the university engineering student pipeline. Another reason for the ranking of these two reasons could be that while receiving university credit and informing students of university engineering programs is one of the reasons for implementation, it may be tended to be thought of as an autonomous part of any high quality program.
It was also noted among the reasons given in the interview's probing questions that community pressure, prestige, and bringing additional funding into the school were not reasons for implementing PLTW. The reason for this could be that CTE directors want the focus of building quality programs and these reasons do not directly relate to that.

Research Question #2
The second research question asked of CTE directors was: What do CTE directors in Utah, that have the PLTW program in their districts perceive about how their PLTW programs are presently meeting implementation goals in serving public education? The findings revealed that the overall majority (7 out of 10) of the directors felt like PLTW was doing a good job in meeting the goals set at the time of implementation. There were some mixed director responses, especially among three of them, when asked if they were meeting the goals set when the program was implemented. They pointed out that this was because of the difficulty districts had in finding the right instructor or problems with getting the information about PLTW classes out to students so they could make good registration decisions. Poor alignment between their schools and universities was also cited as a reason for not reaching original goals of implementation.
The reasons that were given in research question 1 for adopting PLTW into their school districts were being realized. In the districts that said they were meeting implementation goals, it was apparent from comments that PLTW was flourishing; that enrollment had increased dramatically and opportunity for students in their schools was increasing with the partnerships that had been formed between schools, industry, and the community.
One interesting finding was about the PLTW organization itself. Originally the PLTW organization wanted schools to become certified and pressured schools to offer enough PLTW classes to meet this expectation. But, in the director interviews it was noted that PLTW seems to have backed off this position. Perhaps PLTW realized that smaller schools may not be able to sustain all the classes and therefore offered more support to schools which offer just one or two classes to students without the intention of becoming certified.
The goal in serving public education in Utah seems to be met by PLTW programs because the programs give students direction in their education. By blending STEM subjects into classes that show how each is relative in finding solutions to problems, PLTW programs give students avenues of use for their pre-engineering education. These avenues include a spectrum of engineering occupations that range from engineering in its purest form to engineering technologists. Today's modern classroom must combine the efforts of school, society, and industry to guide the students into occupations in this competitive world economy.

Research Question #3
The third research question asked of CTE directors was: "How do CTE directors in Utah that have the PLTW program in their districts define what success means in their PLTW programs?" The findings generated a categorical list of reasons why a PLTW program or its' classes would be successful in a school. As presented in Chapter 4, the list of reasons why PLTW programs or classes were successful included: 1. The ability to attract students and maintain adequate enrollment.
2. The ability to promote student achievement.
3. The perception of having met the goals of implementation.
4. The program has met the present educational goals.
5. The program produces desirable student outcomes.
6. The program creates good public relations.
7. The program platform brings to the school a way to develop partnerships between school, community, and industry.
This list is important to this research because in searching for factors that make PLTW successful in Utah schools, it must first be determined what it means for a PLTW program to be successful. In the following paragraphs, there will be a short discussion on each of these perceptions CTE directors believe contribute to PLTW program success.
For a PLTW program to be successful, directors noted that, classes in the PLTW program must have the ability to attract students and maintain adequate enrollment.
There are few classes, especially elective classes that can exist in a secondary public school environment if they do not have enough students in the class to justify their existence. Justification comes through funding teachers, facility and so on. However, if not enough students take the class it also indicates that in schools serving public needs, the interest for the class is not there. A successful class draws students to it. There may be many reasons for this to happen; the only point here is that successful classes do it. This coincides with the fact that a class is successful because the school does not drop it from their registration.
For a PLTW program to be successful directors noted that, classes in a PLTW program must promote sufficient student achievement. Evidence of student achievement can be found through end of course exams, grades, student observations, projects, and students' pursuits after taking the course. Student achievement must be worthwhile meaning that it promoted student knowledge in academic areas as well as increased the students' understanding of program concepts and how they may be used in life -either domestically or professionally.
In order to be successful, CTE directors noted that the PLTW program must have met the original goals or reasons for implementing it. As discussed in earlier research questions, directors felt that the PLTW program had met the original goals for implementation. These goals may have changed slightly in some demographic areas, but most of the original goals are in place and PLTW programs are striving to meet these goals.
For a PLTW program to be successful, directors noted that a successful PLTW program meets the present program goals. While this reason was very close to PLTW meeting the original goals for success, in some cases the reasons for having the program have changed. Some districts have split and the reasons changed slightly according to one of the directors interviewed. There were also indications in a shift of emphasis between the original reasons. For example, the biggest reason for implementation may have been to send more students to university engineering programs, but after having the program in their schools for a year or two the biggest reason may now be to increase core scores in math and science. While both of these may have been among the original reasons for implementation, the emphasis on importance may have changed. The directors also noted that "program emphasis can also change when new school administrators are hired." For a PLTW program to be successful, directors noted that classes in PLTW programs must promote high quality student outcomes. During the interviews directors indicated that to promote high quality student outcomes the PLTW program gives A successful PLTW program develops a platform incorporating school, community, and industry. During director interviews it was mentioned that industry was heavily involved with many PLTW programs in the form of consultation and student employment either on a part time basis or in some cases full time after graduating from high school. Parents were pleased with the PLTW programs and in some cases even sought out schools that had this program for their children to participate in. From the interviews it was apparent that a solid network had been forged between school, community, and industry and is part of the main scaffolds for program success.

Research Question #4
The fourth research question asked of CTE directors was: What do CTE directors in Utah that have the PLTW program in their districts perceive the factors are that contribute to their PLTW program success? As shown in Figure 5.2, there are 12 different factors listed from the findings presented in Chapter 4 and all these factors appear to be required for program success. From this list it can be seen that having quality people facilitate the program ranks in the highest two places on the list. CTE directors seem to feel that providing quality teachers and knowledgeable counselors are paramount in making the program successful. They are the people who are in the "trenches" interacting with the students. The reason for this may be that if students do not have positive interactions between teachers and counselors enrollments may drop. The reputation of the class may be such that students do not take a PLTW class initially or they do not sign up for more than one class in the program. Also, if there is not harmony between teachers, counselors, and students then achievement in the class may not be as high making the class or program less successful. Directors want to provide a teacher who is personable with students and has the right credentials with a great deal of knowledge about the subject.  these classroom settings was also mentioned as a factor for PLTW program success.
In reviewing these factors, all the directors noted that, "one strong factor in program success was to sufficiently inform students about the program and what its classes offer so good choices can be made according to the needs of the students." In order to do this, a concerted effort must be made to get information about the program out to family members, students' peers, counselors, teachers, and the students themselves.
The directors also considered the counselor training provided by PLTW a credible factor for program success in guiding students into the program. This was important to make sure the right kids signed up for the program and that students had enough room in their schedules to take the PLTW classes. Counselors can also aid in screening students to make sure students entering the program appear to have a high interest in the subject matter, which ranked eighth in the success factor list.
The findings indicate that in general for a PLTW program to be successful the student must be provided with information about the PLTW and pathways that it might take them. Also, the learning has to be done in such a way that all students who take PLTW classes can be successful in them if they try. These considerations for program success seem to be different than most other secondary classes because the main focus of program success is centered on the student. All of these factors are ways to help students achieve and the lens of success is from the students' point of view. This seems to be a fundamental change in education. PLTW classroom curriculum presentations are different to accommodate learning differentials and different student learning styles.
Another factor for success mentioned in the list was providing university credit for taking PLTW classes. It was interesting how many CTE directors thought that providing university credit for taking PLTW classes was important to the program success even though they did not feel their state affiliate university or local universities had done exemplary jobs of facilitating connections with local high schools in the way of concurrent enrollment. Some of the technology centers in the state however are doing things to promote concurrent enrollment in engineering technology career pathways and providing concurrent enrollment with some apprenticeship opportunities for students.
The last factor that will be discussed on the list is that of acknowledging improvement of STEM education in the school overall by offering PLTW classes. One of the educational movements across the nation is to improve STEM education. The "E" in stem is for engineering. Again, President Obama has also acknowledged the need for improving STEM education in our nation. One of the factors of PLTW success is indeed acknowledging the fact that pre-engineering programs do contribute to student achievement in STEM subjects. In a speech at the Decatur Community Recreation Center President Obama (2013) said, "So from the time our kids start grade school, we need to equip them with the skills they need to compete in a high-tech economy. That's why we're working to recruit and train 100,000 new teachers in the fields of the future-in science and technology, and engineering and math where we are most likely to fall behind" (para. 20).

Research Questions 5, 6, and 7
Research questions 5, 6, and 7 were the same in nature. In each research question, the language was changed and directed so that teachers, administrators, and counselors were asked to consider the factors that contribute to developing, implementing and sustaining successful PLTW programs. The fifth research question directed at PLTW

Questionnaire Part I Discussion
The demographic parts of the three questionnaires (i.e., teacher, administrator, and counselor) asked respondents information about themselves or their school. The data from these questions was used for informational purposes about the study, and also to see if there were significant differences between the subgroups in their responses. It was found that there were no cases where respondents disagreed between the demographic groups on any of the survey questions. There was some discrepancy about the strength of agreement between the subgroups, but it did not make any difference in the outcome of the question (e.g., strongly agree as opposed to agree). Data from the question to teachers about the number of classes they taught and how many students were in those classes was used to show validity of the study in terms Have you received the PLTW counselor training? C6 Note. The question letters and numbers refer to the questionnaire (i.e., T = teacher, A = administrator, and C = counselor) and the actual number on the questionnaire. of the percentage of students that were represented by the teachers that responded to the questionnaire. It was interesting that 16 (69.6%) of the teachers that teach PLTW classes also teach non-PLTW classes. However, when filtering the data there was no significant difference between the responses of teachers who taught only PLTW classes and those who also taught some non-PLTW classes.
The findings showed that 17 (73.9%) of the teachers had their endorsements in technology and engineering science. The success of a PLTW program could mean finding a teacher who has this endorsement. The literature generally suggests that well trained teachers are instrumental to program success. A technology and engineering science endorsement is an indicator on the background needed for program success.
One factor that could have a bearing on PLTW program success is adequate preparation time allotted to teachers. One of the demographic questions asked teachers, administrators, and counselors how much time was needed each week to prepare for PLTW classes. Teachers responded with an average time of 3.5 hours per week: administrators responded with 3.0 hours and counselors with 3.2 hours. All three groups agree within a half hour per week of the time needed to prepare. This is an interesting finding because even though administrator and counselor responses may not be as credible as teachers because teachers are the ones doing it, all three groups are within a half hour per week of each other in estimating the time needed to prepare. Also, 81% of the teachers thought that this was sufficient time. Preparation time needed by teachers for PLTW classes does not seem to be an issue in determining PLTW program success. Also, there were no comments about preparation time in the questionnaires. If teachers would have indicated that the preparation time was a lot more than administrators and counselors indicated then adequate preparation time may have been a factor in PLTW program success.
The question for school administrators and counselors about PLTW program certification revealed that eight (28.6%) of the 28 respondents' schools did not plan to certify with PLTW at this time. However, when reading the comment responses to this question, there were 10 comments that said their schools send their students to a technology school which was PLTW certificated and their students did have the benefits of attending a PLTW certified school. There were 11(39.3%) of the respondents who said their schools were PLTW certified with the remaining nine (32.1%) schools planning to become PLTW certified within the next 5 years. There were comments made, that certification would be difficult in smaller schools because of the lack of enrollment that is needed to fund five separate PLTW classes.
The last questions in part I revealed that administrators who responded to the questionnaire had been in their present position for an average of 4.5 years and counselors for 6.7 years. Also, the average length of time that PLTW has been in schools is 5.5 years. While these numbers are a bit ambiguous, it does show a trend that PLTW programs are relatively new in the Utah educational system and are being administered by younger administrators and counselors whose vision of the educational system may be quite different than their counter parts who may have been in the system for a longer period of time. The last finding from the demographic questions was that eight (80%) of the counselors who responded to the question have taken the PLTW counselor training (two counselors did not respond to this question). The data presented in this paragraph seems to indicate a high degree of commitment to implementing the PLTW program and creating a framework for sustaining it.

Questionnaire Part II Discussion
Part II of the questionnaires asked participants their opinions about why PLTW is successful. Figure 5.3 shows that three of the strongest factors necessary for a successful PLTW program are supportive school administrators, supportive counselors, and a dynamic teacher. In fact, this questionnaire finding supports the finding from Phase I of this study where all the CTE directors interviewed indicated that the right teacher was instrumental to the programs' success.
This figure also shows that PLTW is perceived as being successful because of the high quality of the curriculum and that programs are meeting the implementation expectations and goals. Teachers had a mean response near 3.0 which is neutral when they were asked if programs were successful because of their association with the state affiliate university. However, there was a difference between the teachers' mean, and the administrators and counselors mean to this question. It seems that teachers think that the affiliate university has been less of a contributing program success factor than administrators or counselors. Perhaps this is because teachers are more closely involved with students' outcomes, and are better apt at measuring teacher professional development impact on students. The response rates to the questions in Part II of the questionnaire are displayed in Figure 5.3.

Group mean
Strongly agree Strongly disagree One other question in Part II of the questionnaire not shown above asked, "What did teachers, school administrators, and counselors think were the goals or reasons for implementing PLTW into their districts?" The number one answer from all respondents with 42 (82.3%) answering this way was to provide a career pathway for students. The next highest response with 36 (70.6%) was to provide students with more opportunity in engineering related education. From these answers it appears that respondents are in agreement that PLTW gives students pathways in engineering education that are important for their futures. In this question the response options about program quality, strengthening STEM education, and giving secondary students university related options such as prerequisite credit were also highly responded to as being a viable reason for implementing PLTW programs. This list correlates with the list given by CTE directors in the interviews. Therefore there is consistency in reasons why PLTW was introduced into schools between CTE directors, teachers, school administrators, and counselors.
These findings indicate that our secondary school leaders (i.e., CTE directors, school administrators, teachers, and counselors) are recognizing the need for a collaborative team approach in facilitating secondary educational programs such as PLTW. These programs need to have a high quality curriculum which in addition to content provides the students with pathways that lead to careers. There also needs to be improvement in the collaborative process between universities and public secondary schools, so students can be better informed of their options and can begin working on the post-secondary education needed for their chosen occupation before graduating from high school.

Questionnaire Part III Discussion
Sufficient student enrollment in PLTW classes has been perceived by the participants to be an indicator of program success. Part III of the questionnaires asked respondents opinion questions about why students enroll in PLTW classes. To facilitate discussion concerning the findings in this part of the questionnaire, the questions were  In Figure 5. 4  what counselors do, but PLTW formally trains counselors on the aspects of the PLTW program so that they can pass the information on to students. The training is required and is perceived to be of help with enrollments in PLTW classes. One interesting note is that the teacher mean was closer to 3.0, which is "neither agree or disagree" and the administrator mean was above 4.0, which is "agreeing" that counselors play a role in students taking multiple PLTW classes. The difference of opinion may be because teachers do not see how counselors interact with students as much as administrators do.
Also, administrators may understand the counseling role better than teachers.
It can be seen in Figure 5.5 that respondents believed students were taking PLTW classes because they were genuinely interested in the subject and that they wanted to take advantage of the "hands-on" learning technological environment, where students learn by doing and collaborating with others. These were the two top reasons in this group of questions that participants thought that students enrolled for in PLTW classes. The means 1. T29, A30, C29-Student achievement is enhanced because of pre-existing student knowledge 2. T30, A31, C30-Student achievement is enhanced because of PLTW teacher training 3. T31, A32, C31-Student achievement is enhanced because students are motivated to do well on the end of course exams 4. T32, A33, C32-Student achievement is enhanced because of partnerships formed between the school, industry, and the community 5. T33, A34, C33-Student achievement is enhanced because of counselor training between teachers, administrators, and counselors suggests that generally the response of "most" (between 50% and 75%) of the students took PLTW classes for these reasons.
Teachers however did tend to select the response of "some" (between 25% and 50%) of students enrolled in PLTW classes because of the learning environment than counselors and administrators. Perhaps in teaching those classes teachers feel that the PLTW environment and method of teaching is not as strong a reason for students to enroll in the class as administrators and counselors may think.
The next discussion will include that of students enrolled in PLTW classes because of influence from family and friends, they liked the teacher, guidance they received from a counselor, and/or possibly for college prep and college credit. The teacher mean was lower in the "family influence" question than administrators and counselors which suggest that teachers may generally feel that less students were in their classes for this reason. While there was some fluctuation between a mean of 2.5 and 3.5 in the above mentioned categories, participants tended to select the choice of "some" (between 25% and 50%) of the students were taking PLTW classes for these reasons.
While these may be important factors to consider when implementing or improving a PLTW program they do not appear to be as individually important as other factors.
Another interesting note is that in general the participants chose "some" (between 25% and 50%) of the students taking PLTW classes would complete the programs in their schools by completing all the required PLTW classes. This suggests confidence in the quality of the program that all three groups would believe that this many students would indeed complete all the classes necessary to be deemed a program completer. Teachers had the highest mean of the three groups, which again exhibits confidence in the program and possibly in their teaching.
Compared to the other reasons for students to enroll in PLTW classes, the teacher and administrator means suggests that fewer students enroll to increase their proficiency in math and science than any of the other reasons. The counselor mean for this question on the other hand suggests that improvement in math and science is a stronger reason for students to take PLTW classes.
These findings about why students enroll in PLTW classes are very important to this research because the reality of keeping any elective class in the school offerings includes the fact that there must be a high enough enrollment to justify the offering. In some schools students who take the course Principles of Engineering may receive a science credit, but the rest of the PLTW classes in the program are elective. These findings seem to indicate that in order for students to want to sign up for a PLTW class they have to fully understand the program and what the classes will teach them. Students may be informed through many different ways as shown in the findings. These different ways must be utilized by program facilitators to attract students into the programs.
Elective classes have the difficult task of making the class enjoyable for students while still maintaining standards for the grades that are given. A successful PLTW program does depend on facilitators understanding how students receive information concerning PLTW classes and that the information they receive is accurate about what these classes can do for them.

Questionnaire Part IV Discussion
The last section of the questionnaires had questions that asked respondent's their opinions about factors that enhance student achievement in PLTW classes. Figure 5.6 shows the responses for the five questions asked teachers, administrators, and counselors.
The mean for the first questions responses shows that teachers and administrators "agree" and that counselors "strongly agree" that student achievement is enhanced if students have pre-existing knowledge in math, science, and technology when they begin a PLTW class. Because of the nature of a pre-engineering class, it makes sense that the more academic skills in math and science that a student possesses, the more success they will have in the class. The respondent's means also indicate that they "agree" that students' achievement is enhanced because of the teacher training provided by PLTW. Teacher and counselor training helps insure that students understand what membership in PLTW classes entails and that they will receive instruction the way it was intended to be presented. As mentioned before, a qualified teacher is considered critical in PLTW program success. It is reasonable that a good teacher training program will help teachers become better at their craft.
Both the administrator's and counselor's means indicate that they "agree" that the partnerships PLTW forms between school, industry, and community also aid in enhancing student achievement and that student achievement is enhanced because of counselor training. However, in for both of these questions the teacher's mean suggests that they are more neutral choosing "neither agree nor disagree" with the statement. This could be because administrators and counselors better understand that student participation in the PLTW program could lead to gainful employment or placement in an educational pathway that could lead to a college degree in engineering, where teachers do not fully understand how these two factors will help their students to be more successful in life. With collaboration between these entities student understanding of how the program fits in their life could be more evident.

Recommendations for Implementation or Restructuring
This research is useful as it provides information to help facilitate the implementation of successful PLTW programs or improve existing programs. The following recommendations should be considered when implementing or improving a PLTW program.

Utilize a dynamic teacher-It was mentioned multiple times in this study by
CTE directors and school administrators how important hiring the "right" teacher is.
They indicated that the right PLTW teacher is willing to go the extra mile to make sure the program satisfies the needs of the program and the students in it: perhaps in public relations, industrial relations, or curriculum preparation. It also meant that the teacher is "genuine" to the students and produces an environment conducive to learning. Directors and school administrators were also supportive of the PLTW teacher training that requires teachers to participate in professional development which gives them state-ofthe-art instructional curriculum and shows them the correct instructional methods.
Teacher professional development was thought to enhance student achievement. This research has revealed that teacher training is well thought of and is a valuable part of the PLTW program. Enrollment and achievement have been perceived by the participants in this research to increase because of a dynamic teacher that students like. A successful PLTW program depends on finding the right teacher.
2. Capitalize on student interest-One of the findings from this research was that CTE directors, teachers, school administrators, and counselors agree that students genuinely seem interested in the subject and are thought of as wanting to take advantage of the unique learning environment that PLTW offers. It was generally shown in this research that the people who interact with students such as family members, peers, and counselors do aid in helping students to become interested in the class. Realizing this, all the "players" involved in producing the program should do everything they can to capture the interest of the students by providing information about the PLTW program, the instructional methods used, and what the knowledge learned in the class and the credit generated can do for them. interviews that the PLTW program is expensive and before implementing the program everyone involved needs to understand where the funding is coming from, and also that there is a suitable facility to operate the classes in.

Recommendations for Further Research
Future research about PLTW programs could go two different directions. The first would be to find measurements of success of the PLTW program without the data being so dependent on opinion. The research needs to be carried longitudinally to university affiliates of PLTW to find out how many of their students are there because of PLTW.
The second is to use this research as a model to ascertain the worth of any elective program. Success for elective classes is defined by student enrollment and the progress of the students who take the classes.

Summary
In an effort to improve STEM education, the PLTW pre-engineering program for secondary schools was introduced in 1986 in New York. Since then it has spread to all 50 states and is extensively used. It was introduced in Utah in 1990 and has diffused into 10 different school districts and involves 33 secondary schools.
PLTW offers students a different avenue of education based on a project method which involves holistic educational concepts that are needed to problem solve. It teaches students to analyze problems, to collaborate with others about the problem, and introduces to students methods to solve problems. The PLTW method utilizes and reinforces concepts learned in math and science classes, which increases student achievement in them. 9. PLTW has changed their program requirements significantly to accommodate smaller population schools. A school may now commit to teach as few a one PLTW course. Digital Electronics (DE) is no longer required as a foundation course and is now listed as an elective. Affiliate universities across the country have increased their standards to earn concurrent credit. These policies allow local flexibility while still maintaining university rigor.
10. The teacher must be deeply committed to teach PLTW. If the teacher is not committed you will not have a successful program.
11. Successful PLTW schools usually have at least two teachers delivering the program. It is very difficult to sustain a full teaching contract with just engineering courses. Only magnet schools and certain charter schools maintain full engineering assignments. A typical school might have one instructor teaching IED and other CTE course. Another instructor might teach POE and other science courses. Additional personalities and curriculum background seem to benefit the overall engineering program.
12. Technology Education teachers are successful as PLTW teachers only if they have a strong math and science background. Science teachers succeed as PLTW teachers if they have a strongly orientation towards hands-on learning and can demonstrate the application of science and math concepts. Engineers make great PLTW teachers if they can relate to young people and develop good teaching skills. A quality teacher is the heart of a successful PLTW program.
13. PLTW teachers must constantly upgrade their skills and be willing to apply new curriculum. PLTW is a dynamic program with regular updates, new applications, new software, new equipment, etc. This is not a curriculum for a teacher to learn once and then coast for the next 10 years.
14. The teacher must be enthusiastic and positive about engineering. The teacher's personality and positive attitude will drive the enrollment in the program.
15. The teacher must be willing to work with students and remediate those that are struggling. Not all students will be at the same academic level.
16. The teacher must be willing to allow students to solve their own engineering challenges and do their own research. This is not a program where students sit quietly in their seats while being entertained by the teacher's vast store of knowledge. Students are challenged to solve problems, work in teams, do research, gather and analyze date, document their work, do homework, and present oral reports. Students will grow from their successes and failures.
17. Teachers should be using the year-end tests provided by PLTW and monitoring program improvement.
18. Good PLTW teachers take advantage of web-based learning resources provided by PLTW. Teachers have access to a Virtual Academy to learn new curriculum and update their skills.
19. Successful PLTW programs have an active advisory committee of industry partners. These partners provide curriculum support, guest speakers, industry tours, student mentoring, job shadowing, and even internship opportunities.
20. And finally, quality PLTW programs go through the certification process, strive for continual improvement and become a Certified PLTW School.

Listing of STL Content Standards
The Nature of Technology Standard 1. Students will develop an understanding of the characteristics and scope of technology. Standard 2. Students will develop an understanding of the core concepts of technology. Standard 3. Students will develop an understanding of the relationships among technologies and the connections between technology and other fields of study.

Technology and Society
Standard 4. Students will develop an understanding of the cultural, social, economic, and political effects of technology. Standard 5. Students will develop an understanding of the effects of technology on the environment. Standard 6. Students will develop an understanding of the role of society in the development and use of technology. Standard 7. Students will develop an understanding of the influence of technology on history.

Design
Standard 8. Students will develop an understanding of the attributes of design. Standard 9. Students will develop an understanding of engineering design. Standard 10. Students will develop an understanding of the role of troubleshooting, research and development, invention and innovation, and experimentation in problem solving.

Abilities for a Technological World
Standard 11. Students will develop abilities to apply the design process. Standard 12. Students will develop abilities to use and maintain technological products and systems. Standard 13. Students will develop abilities to assess the impact of products and systems.

The Designed World
Standard 14. Students will develop an understanding of and be able to select and use medical technologies. Standard 15. Students will develop an understanding of and be able to select and use agricultural and related biotechnologies. Standard 16. Students will develop an understanding of and be able to select and use energy and power technologies. Standard 17. Students will develop an understanding of and be able to select and use information and communication technologies. Standard 18. Students will develop an understanding of and be able to select and use transportation technologies. Standard 19. Students will develop an understanding of and be able to select and use manufacturing technologies. Standard 20. Students will develop an understanding of and be able to select and use construction technologies.
The purpose of this CTE Director Interview is to ascertain your perceptions about the Project Lead the Way (PLTW) program in your district. Participation in this research is voluntary. The data generated by this interview is anonymous and will not be connected to you or your district in any way. This interview will be recorded for later analysis. 1. How many high schools are in your district? (demographic question) a) 8 high schools plus an alternative high school b) 4 high schools c) 9 high schools d) 4 high schools soon to be 5 and a technical school e) 5 high schools and one technical center f) 1 high school g) 1-we're a charter school h) 2 i) 4-but only 2 offer PLTW classes j) 2 2. Do the secondary schools in your district predominantly represent rural or metropolitan communities? (demographic question) a) Metropolitan-Urban b) Rural, more urban, mostly suburban c) Urban communities d) Urban communities e) Urban communities f) Small town but still considered urban g) A mix of rural and urban h) Rural i) Rural j) Urban-inner city 3. How many years have your district offered Project Lead the Way (PLTW) courses? a) 7 th year, we started PLTW in 2005 (demographic question) b) 8 years c) 12 years, since 1990 d) 3 years, Canyons School District has only been here for 3 years e) 5 years f) 6 th year-I believe g) 6 years h) 3 years i) 3 years j) 9-I think 4. How many high schools in your district are "certified" PLTW schools? If some of your schools are not, why? (demographic question) a) None-working on it b) 1-This one (Two Rivers) others are feeders-high cost and eliminate duplication c) 1 certified school-more are not because of high cost d) 1 certified school e) None-but just completed self-evaluation to get the tech center certifiedduplication f) We offer 4 classes in a hybrid setting, but are not certified. g) We are not a certified school-we don't have the funding for EDD-no PLTW pressure h) None i) None j) 2-both are certified 5. What scholastic group of students does PLTW serve?-What percentage do you perceive as being AP? (#24) (new question #2) a) Higher end student on engineering track (15%) b) All ages, all levels, all genders-not an elitist program c) Median to upper bound college students-career goal in engineering or engineering tech. d) Middle to upper groups e) Medium to high students-good "B" student f) Totally varied-some were placed in by counselors g) A and B students-everybody here has to take some PLTW classes h) Achieving students, I would say C through A i) Good "B" students j) "B" students (Research question) #1. Do CTE directors perceive that PLTW programs in the state of Utah are achieving the goals which were set when the PLTW program was implemented into their district? Comments: a) Ours has gone south, we have had some struggles there, not a large draw, b) yes, but it might be in the knowledge they can achieve in college not necessarily in the form of college credit, c) which came with time after collaboration, d) we are studying student outcomes at this time, e) we have agreements with SLCC, USU, and UVU, f) we are not pursuing that at all o The opportunity to get teachers trained (professional development) on "up to date" STEM-based curriculum. (#23) (New Question #8) a), c), d), g), j) Strongly Agree b), e), f), h), i) Agree Comments: c) this gave teachers a robust understanding of implementing math and science, d) are teachers are really excited when they get back; this carries over in their enthusiasm for the class and to kids o The "prestige" and recognition as being identified as a school that offers "engineering" type programs. (not a reason) h), i) Agree j) Neither a), b), c), d), e), f), g) Disagree Comments: b) not really, it was mostly students' needs driven. e) I don't think that was the intent o Community pressure-families wanting challenging STEM courses for their students. d), h), i), j) Agree a), b), c), e), f), g) Disagree (not a reason) Comments: a) not a large push, b) more industry pressure, e) there was no community pressure o The partnership and support offered by PLTW (#25) b), c), d) Strongly Agree a), e), h), i), j) Agree f) Neither g) Disagree Comments: c) it required counselors to be part of the team, and forced the school to get involved with these programs and their community needs, d) we are still in the process of forging those relationships and it is working so far, f) I believe there could be an incredible support system but we don't take advantage of it o The opportunities to receive grants or other funding by offering PLTW programs. e) Agree d), j) Neither a), b), c), f), g), h), i) Disagree (not a reason) Comments: a) In fact, it was very expensive, c) but the payback is tenfold when students break through that educational wall in post-secondary education, d) the program is expensive and we are continually looking for sources of funding

(Research question) #2. What do CTE directors in the State of Utah perceive about how PLTW is presently meeting implementation goals or serving public education?
7. (related interview question) Do you think the current PLTW programs are meeting the goals set at the time of implementation? Why or Why not? (New Question #6) a), f), g) No, because it is difficult to find the right instructor, the instructor makes the program, our problem is internal b), c), e), g), h), i) Yes, but more opportunity is given to smaller schools who offer only one or two classes. The goal is to get more kids on campus and successful in post high school training c) Yes, but it was designed around a 4-year high school and we are a 3-year high school so kids have to cram their schedules to get it all in, especially with raising graduation class requirements c), j) No, because we are not aligning with post-secondary institutions as well as we had hoped, it is difficult to meet local requirements with a national program like PLTW d) It is hard to say, we are only 3 years old and we inherited this program. We are still in the throes of implementation ("d" did not comment on probing categories). j) The goal is to get more kids to finish and we need to work on that j) The "right" kids are sometimes not being put into PLTW classes Comments: a) numbers come easier with multiple high schools that attend the tech center, c) we strongly feel the program has drawn enough kids to justify its existence, in fact it's grown too much, e) our enrollment has been great, f) I don't think it's the fault of PLTW, the teacher was struggling with that before and I still think that's the case o Successful because of the student outcomes it produces. (#16, #20) a), b), c), d), g), h), i), j) Strongly Agree e), f) Agree Comments: b) We have 130 seniors going out to businesses, government entities, law offices, and hospitals doing full blown internships. The model has expanded beyond PLTW. It teaches our kids what is expected in the workplace. d) Today's education demands that we take a close look at our product which is the students we produce and what their abilities truly are.
o Successful because of the public relations it creates. (#22) b), d), j) Strongly Agree a), c), e), f), g), h), i) Agree Comments: c) from a student's point of view-yes. d) we have an advisory committee and the people who sit on that board have been awesome. We have shadow days and the next step is internship. e) yes, I think that's an attribute that we initially didn't realize, but it has been a good source of PR.
o Successful because of the program platform it brings to the school (i.e., professional development, partnerships, complete "canned" curriculum, Public perception, etc.) b), d), h), i), j) Strongly Agree c), f), g) Agree a) Disagree (#25) (Research question) #4. What do CTE directors perceive as the factors that contribute to PLTW program success?
9. (related interview question) What factors do you think contribute to PLTW program success in your schools? a), b), d) a good advisory board, good community support (industry) (#25) b), d), f), g), j) a good teacher (#10) b) good relationships with post-secondary institutions (#15, #16, #24) d) up to date equipment-cutting edge lab so students feel like we are up to industry standards.
(#12) e), g) good administration support (New Question #5) e), g), h), i), j) good counselor support-having good understanding of the program (#26) e) obviously high student interest in engineering. We start a new program hopefully based on student interest and they have the desire to do it. We go through all the players, but student interest and support are key factors (New Question #7) h), i) Schools and students informed about what the program is Probing Question categories: o Student interest in subject matter. (#7) b), c), e), j) Strongly Agree a), d), f), g), h), i) Agree Comments: c) the interest has to be in applying it to life, it hurts me to no end because we go to school for what?-to get a job and be productive in society. The interest has to be in things like problems solving, which is something we do every day of our lives. d) if the kid doesn't care then how can they progress or why did they take the class?
o Other people's influence such as family members or peers. (#8, #9) e), f) Strongly Agree a), b), c), d), h), i), j) Agree g) Disagree Comments: b) it's the local commerce and industry that drives our program c) a lot of times we hear students say they took the class because their brother or sister took it. d) if we can get parents behind it-that makes a big difference. J) it's a factor as to getting students to sign up for the class, but becomes less of a factor of their success once they are in the class.
o Counselor guidance. (#13, #19, #26) b), c), d), e), f), g), h), i), j) Strongly Agree a) Agree Comments: a) We hope so-we push it with our counselors a lot and try to educate them as to the critical needs of engineers. b) we train them in-house, we make them do tours. We also make them recruit in the lower grades. d) counselors make a bid difference o Granting math or science credit. (#11) a), b), c), d), e), f), h), i), Strongly Agree g), j) Agree Comments: a) A huge factor -and should with the rigors that are involved it should be. b) they do get a science credit for Principles of Engineering and Digital Electronics, c) The more we offer in math and science credit the more students we are going to get. We get whatever is left over so the student who wants to gear toward engineering has to do it as an elective and that is not right. d) Absolutely-that is a battle we always fight because graduation requirements keep getting higher and higher so it squeezes out CTE to some degree. e) the science credit in our district from POE has been a great draw for students. Comments: c) I think more Technology Education, but it depends on the individual characteristics and traits. d) I think a balance is best. You should get an academic person, but often time these people don't understand the hands on application of how things work in the real world. e) it totally depends on the guy on one hand you could see if teachers have an engineering or math back ground they could certainly bring those things, but I would say that if you have a dedicated teacher who understands the fundamentals of teaching, and you give them help in areas they struggle. Then they could be a powerful teacher. f) I think it comes down to the personality of the teacher or the ambition of the teacher. (#7) It would be great if they were math and science but they get into what I call "flat instruction" vs. the rich instruction of PLTW. I really think it comes down to the individual teacher's personality and not necessarily if they have certain certifications.
o Students not informed about the class or program. (#14) a), b), c), d), g), h), i) Strongly Agree e), f), j) Agree Comments: a) that's a constant battle to help students understand what we are trying to prepare for them. b) we do extensive promotion in the Jr. Highs c) even with all the advertising we do with screen savers, bulletin boards, videos, etc. it's still amazing to me that kids still say, "we have engineering in this school". So the word is hard to get out there, the more we inform students of their opportunities the stronger our enrollments are. d) I've been working with a STEM group to come up with an enrichment course in the 7 th grade. I'm excited about that because getting the curriculum people behind that is huge because they don't know the role of CTE very well yet. j) we are offering Intro. To Eng. to our 9 th graders next year.
o Program cost-willingness to spend the money. (#25) b), c), d), e), g) Strongly Agree a), f), h), i), j) Agree Comments: b) all education is an investment, when we find something that works as well as PLTW, we tell the players that it's worth it c) you have to be committed and have all the partners agree that this is the right thing collectively to do-then it works. d) all partners have to have a common goal to do what's right for kids to secure the necessary funding to make the program work o Students can't fit it into their schedule. (#18) c), d), e), g), j) Agree a), b), f), g), i) Neither Comments: a) yes, and no, I mean students buy what they value, and they have the elective room in their schedules.-however the legislature is trying to shring those options, but they are still there. c) a lot of room is taken with required classes.
Survey Changes 1. Create a survey just for counselors and reduce confusion of trying to have one survey for both school administrators and counselors.
2. Ask about the scholastic group of students served by PLTW in surveys.
In your opinion what scholastic group of students do PLTW classes attract and serve? (check all that you think apply) "A" students "B" students "C" students "D" students 3. Ask counselors and teachers about the importance of a supportive school administrator. 4. Ask teachers and school administrators about the importance of a supportive counselor.

Ask school administrators and counselors about the importance of a dynamic teacher.
A supportive school administration (supportive counselor, dynamic teacher) is very important to the success of the PLW program in your school?
Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree 6. Ask if in their opinion: Is the PLTW program meeting the implementation goals set when the program was initiated in their school.
The current PLTW program in our school is successfully meeting all the program goals set at the time of implementation.
Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree 7. Ask if PLTW is successful because of the overall quality of the program. One of the most important reasons that the PLTW program in our school is successful is the overall quality of the program's curriculum.