Articles Assessment of Project-Based Learning in a MECHATRONICS Context

Introduction Project-based learning (PBL) that has authenticity in the pupils’ world enables the teaching of science and technology to pupils from a variety of backgrounds. PBL has the potential to enable pupils to research, plan, design, and reflect on the creation of technological projects (Doppelt, 2000). Imparting creative thinking within the design process of pupils’ projects not only requires changing the teaching methods and learning environment, but also adopting new assessment methods such as student portfolios. Engineering education, which is common in Israel, has a unique structure in that it combines practical and theoretical knowledge, synthesizes vertical and lateral thinking, and creates a rich and flexible learning environment. The CTT (Creative Thinking in Technology) program (Barak & Doppelt, 1999) integrates Co.R.T. Thinking tools (De Bono, 1986) into the technology curriculum using the LEGO-Logo learning environment for creating authentic projects. The program began in 1994. Pupils study lateral thinking tools in order to deal with different alternatives, to consider multiple factors, and to refrain from premature judgments on ideas. They use vertical thinking tools in order to document their design process and to calculate and to structure programming for the control of their projects. Earlier field research by Barak, Waks, & Doppelt (2000) showed that pupils prefer a learning environment that emphasizes planning and building activities and team projects. Pupils have stated that these aspects of a learning environment contribute to creating challenges, curiosity, imagination, and success in studying technological subjects (Doppelt & Barak, 2002). As the CTT program evolved, a Creative Thinking Scale (CTS) was


Introduction
Project-based learning (PBL) that has authenticity in the pupils' world enables the teaching of science and technology to pupils from a variety of backgrounds.PBL has the potential to enable pupils to research, plan, design, and reflect on the creation of technological projects (Doppelt, 2000).Imparting creative thinking within the design process of pupils' projects not only requires changing the teaching methods and learning environment, but also adopting new assessment methods such as student portfolios.Engineering education, which is common in Israel, has a unique structure in that it combines practical and theoretical knowledge, synthesizes vertical and lateral thinking, and creates a rich and flexible learning environment.
The CTT (Creative Thinking in Technology) program (Barak & Doppelt, 1999) integrates Co.R.T. Thinking tools (De Bono, 1986) into the technology curriculum using the LEGO-Logo learning environment for creating authentic projects.The program began in 1994.Pupils study lateral thinking tools in order to deal with different alternatives, to consider multiple factors, and to refrain from premature judgments on ideas.They use vertical thinking tools in order to document their design process and to calculate and to structure programming for the control of their projects.Earlier field research by Barak, Waks, & Doppelt (2000) showed that pupils prefer a learning environment that emphasizes planning and building activities and team projects.Pupils have stated that these aspects of a learning environment contribute to creating challenges, curiosity, imagination, and success in studying technological subjects (Doppelt & Barak, 2002).As the CTT program evolved, a Creative Thinking Scale (CTS) was ____________________ Yaron Doppelt (yaron@pitt.edu)was the Academic Coordinator of the Science and Technology Youth Center, Technion -Israel Institute of Technology, Haifa, Israel.At the time this was published he was serving as a Research Associate at the Learning, Research, and Development Center (LRDC) at the University of Pittsburgh in Pennsylvania.
developed in order to assess pupils' portfolios (Barak & Doppelt, 2000).The study reported herein extends research conducted on the CTT program and proposes a Creative Design Process (CDP) that can be assessed using the CTS.

Theoretical Background
The theoretical background of this study cuts across three areas of inquiry: engineering education, the infusion of creative thinking into the design process, and the assessment of project-based learning.

Engineering Education: A MECHATRONICS Context
De Vries (1996) claimed that we should help pupils integrate knowledge from science and other disciplines into the design processes.It is evident that there is a role for science education and that science education remains a crucial part of general education, even where technology education has gone beyond the "technology is applied science" paradigm.Technology education is an equally valuable subject to science education, and both subjects should be taught (Gardner, 1997).
Engineering education in Israel is part of the comprehensive high school curriculum.At the end of junior high school, pupils have to choose one or more areas as a "major," such as the Sciences, Humanities or Technology.Within Technology is Engineering.The Engineering curriculum for high school in Israel contains several core subjects that are related to physics and mathematics, such as civil engineering, computers and electronics, and mechanics and control systems.MECHATRONICS is a new sub-major in the mechanics and control area.Pupils study MECHATRONICS for three years, 10th -12th grades.They study physics, system control, mechanics, and programming for five hours per week, with a total time devoted to all classes of 20 hours per week.This syllabus is about half of their weekly schedule.
The MECHATRONICS syllabus has been implemented in nine schools since 2000/2001.In 2003 there were fifteen schools that chose to advise their pupils to take this syllabus.The MECHATRONICS curriculum suggests that educators create rich learning environments filled with real world applications.The assessment processes of learning outcomes in a rich learning environment have had an important impact on the learning process (Doppelt & Barak, 2002).In addition, the perspective of pupils on the most influential characteristics of the learning environment is important for the teaching-learning process and for designing the learning environment (Doppelt, 2004).This study suggests a way of infusing creative thinking into the design process during project-based learning (PBL).

Infusing Creative Thinking into the Design Process
The design process is similar to the process of problem solving.Many general structures for design processes appear in the literature.A generalized approach to structuring the design process might include six stages: defining the problem and identifying the need, collecting information, introducing alternative solutions, choosing the optimal solution, designing and constructing a prototype, and evaluating and correcting the process.The design process might also include four thinking levels: 1. Understanding existing systems 2. Systematic and functional understanding of engineering systems 3. Applying set procedures for analysis and synthesis 4. A controlled design process that assures that the above steps are applied by the students.(Mioduser, 1998) Teaching technology using a generalized design process has been criticized by researchers who have claimed that it is difficult for pupils, and even for teachers, to learn how to use it (McCormick & Murphy, 1994).This is not surprising given that different technology disciplines go about designing in different ways (Hill & Anning, 2001).In order to avoid the pitfalls of a generalized design process, teachers should assist pupils to integrate the knowledge of the various disciplines and learn the applicable standards and rules, as well as the underlying scientific principles and economic concepts (De Vries, 1996).This article will describe a design process that applies these suggestions and is sensitive to the authenticity of the pupils' projects.This design process was based initially on the PISCO framework (De Bono, 1986).PISCO stands for: Purpose, Input, Solutions, Choice and Operations.Infusing the teaching of thinking skills into a specific disciplinary course may provide a rich learning environment that will contribute not only to the development of thinking skills but also to a better understanding of the discipline under study (Glaser, 1993;Ennis, 1989;Zohar & Tamir, 1993).De Bono (1986) differentiates between two types of thinking: lateral thinking, which refers to discovering new directions of thinking in the quest for a wealth of ideas, and vertical thinking, which deals with the development of ideas and checking them against objective criteria.Lateral thinking and vertical thinking are quite different processes.It is not a matter of one process being more effective than the other, as both are necessary.Rather, in order to be able to use both effectively, one must appreciate their differences.Lateral thinking is a central, but not singular, component of creative thinking.Waks (1997) pointed out that during work on a technological project, lateral thinking initiates the learning process while pupils seek alternatives and examine different solutions.Vertical thinking is essential at the stage of choosing a solution and developing it.Vertical thinking and lateral thinking complement each other, and both are the essential elements of creative thinking (De Bono, 1986).

Assessing Creative Thinking in PBL
Technological systems that are controlled by a computer provide a rich learning environment and expose the learner to a variety of representations and configurations, such as realistic models, simulations, mathematical models, algorithms, graphics, and animations.One of the better-known examples of such a rich computer-based learning environment is the LEGO-Logo system (Doppelt & Armon, 1999;Jarvinen, 1998).Resnick and Ocko (1991) stated that this learning environment puts children in control because they formulate their own designs and experiments and work on projects that they care about personally instead of reproducing someone else's experiment.
Project based learning in technology encourages pupils to work in teams (Barak & Maymon, 1998;Denton, 1994).In this way, pupils combine "handson" activities with what Papert (1980) has termed "heads-in" activities.Projectbased learning (PBL) could be used as a tool to develop pupils' competencies by working on integrated projects (Barlex, 2002).Project-based learning through authentic issues taken from the pupils' world enables pupils from various backgrounds to study science and technology in a way that makes sense to them (Seiler, Tobin & Sokolic, 2001).An authentic project deals with real life situations and, by definition, is integrative in nature.This approach has been implemented throughout the world and past researchers have shown interesting findings regarding the opportunities afforded by PBL (Barak & Doppelt, 1999, 2000;Barak, Eisenberg & Harel, 1995;Barlex, 1994;Doppelt, 2003;Doppelt & Barak, 2002;Hill, 1998;Resnick & Ocko, 1991).One of the key elements for success in PBL is engaging the pupils in the assessment process.
Imparting creative thinking within science and technology education not only requires changing the teaching methods and learning environment, but also adopting new assessment methods, such as portfolio assessment, which is based on records of pupils' activities.The portfolio might consist of such items as written material, computer files, audio and video media, sketches, drawings, models, and pictures.The portfolio reflects what pupils have learned, how they question, analyze, synthesize, solve problems, and create new ideas, and how then design and build useful products or systems.The portfolio also shows how pupils interact intellectually, emotionally, and socially with others (Collins, 1991;Wolf, 1989).Barak and Doppelt (1999) have shown that pupils were able to cope with complex problems and they developed solutions that depended on creativity, supporting the notion that they applied lateral and vertical thinking.Pupils created portfolios containing documentation of their creative thinking and the learning processes in which they were engaged.Over a period of several years, each class developed criteria for assessing their portfolios.On the basis of these experiences, a Creative Thinking Scale (CTS) was developed and was formally applied to the assessment of pupils' portfolios (Barak and Doppelt, 2000).
The suggested assessment scale of creative thinking can help educators strive for a gradual development of higher-order thinking skills in two main areas.The first is the choice of a project topic that that pupils make.This includes dimensions of complexity, originality, and creativity on the one hand and the extent of mathematical, logical, and scientific thinking on the other.The second area considers the thinking and learning processes applied as the pupils developed their project and includes problem solving, teamwork, and reflective thinking.Thus, learning through the designing and implementation of technology projects based on portfolio assessment and directed towards a systematic development of vertical and lateral thinking may promote teaching and learning that assist the successful integration of the pupil into the dynamic and changing world outside of the school once they graduate.

Intervention Program
The intervention program included in-service training of teachers for a duration of 112 hours.This consisted of twenty-eight workshops of four hours per meeting during the school year.The program was aimed at introducing three main topics: the tutoring process, the Creative Design Process (CDP), and assessment using the Creative Thinking Scale (CTS) criteria.Eighteen teachers from nine schools who had taught MECHATRONICS since 2000 were involved in the project.

Six Stages of the Creative Design Process (CDP)
The creative thinking framework was originally developed by De Bono (1986).He based this framework on five thinking steps that he named "PISCO" -Purpose, Input, Solutions, Choice and Operations.The Creative Design Process (CDP), which is presented in this article, adopts this creative thinking framework and extends it to project-based learning (PBL) in technology education.
The creative thinking tools that are suggested in the CDP are part of the Co.R.T thinking program (De Bono, 1986).They include P.M.I (Plus, Minus, Interesting), C.F.A (Consider All Factor), Rules, C&S (Consequence and Sequel), O.P.V (Other People's View) and F.I.P (First Important Priorities).
First stage: Design Purpose.The first step in the design process is defining the design problem.The pupils need to set the design goals.These goals must fit the definitions of the problem.The achievement of these goals will be under the restrictions that the designer has set forth and include budget, availability, equipment and tools, schedule, and so on.Three steps are recommended to the pupils for the documentation of this first stage: 1.The problem and the need -Pupils describe the reasons that motivated them to choose their project.They also define the problem and define the needs that their solution will address.2. The target clientele and restrictions -Pupils describe the target clientele and define the restrictions that they must take into consideration.The Rules thinking tool can help you consider rules, standards, and other restrictions.3. The design goals -The pupils define the necessary demands they expect to be met by the system.
Second stage: Field of Inquiry.The second step in the design process is to define the field of inquiry in which their problem resides.This is founded on the problem definitions and goals from the first step.Pupils must research and analyze existing systems that are similar to the one they are developing.Pupils need to organize the documentation of their inquiry.This includes the following main areas: 1. Information Sources -Books, professional magazines, manufacturers' catalogs, and Internet sites.2. Identification of Engineering, Scientific, and Societal Aspects -Engineering concepts, scientific concepts, societal and environmental aspects, cultural values, and potential issues and dilemmas.3. Organization of the Information and its Assessment -Arrangement of the information according to the goals and restrictions of the problem.Pupils need to summarize the information gathered so that the design problem and approach are informed by it.Pupils should express their opinions regarding the match of the information they have gathered to their design problem.They must also provide a rationale for why their evolving design is a better alternative to systems that already exist.
Third stage: Solution Alternatives.The third step in the Creative Design Process involves the consideration of alternative solutions to the design problem.This is a lateral thinking stage and includes three components: Ideas, Factors, and Opinions (of other people).Pupils need to be educated so that they feel freedom in their thinking and are not discouraged by the judgments that other pupils and their friends might make.This strategy will increase the likelihood that numerous design possibilities will be considered, with the intent of inspiring creativity and arriving at an idea that no one else has developed.There are no bad ideas at this stage.The pupils are presented the following suggestions and guidelines: 1. Ideas Documentation -The use of the P.M.I (Plus, Minus, Interesting) thinking tool is recommended to the pupils so that they consider as many ideas as possible and formally evaluate them, considering all aspects.The aspects of the ideas are rated as positive, negative, or interesting, meaning that they have promise but more investigation is needed.2. Consider All Factors (C.A.F) -Pupils are asked to write down all the factors related to the system they are designing.They must consider the perspectives of the consumer, the designer, the manufacturer and the marketer.The use of the F.I.P (First Important Priority) thinking tool is recommended to pupils in order to help them set priorities.This thinking tool could assist in choosing the optimal solution.
Fifth stage: Operation Steps.The fifth step in the design process is planning the operational ways to implement the chosen solution.Planning must be consider the following points: • Sometimes the chosen solution is a complex system.Dividing it into subsystems may assist in defining the steps that are needed to develop the solution.
• In many cases choosing the ideal materials, parts, and mechanisms are a central part of the design process.• The sketches and drawings (computer-generated as well as traditional) are important to the presentation of a design.• Choosing machines, tools, and manufacturing processes are necessary steps to creating a prototype.• Planning how to make the prototype is critical to success.Planning activities include developing a timeline of tasks and making sure the necessary materials, parts, machines, human resources, and so forth are available and will be available when needed.
Six stage: Evaluation.The last step is to evaluate the overall process and product.This is a summative evaluation and relates back to the formative evaluation steps that were done as the idea was developing.In this final stage the pupils need to document: • What difficulties were encountered and what methods were used to overcome them?• Does the system actually provide a viable solution to the problem?• Does the prototype fulfill all its performance requirements?
• In what way could the prototype be improved?
• What are the implications for further development?

The Creative Thinking Scale
In this study the Creative Thinking Scale (CTS) was used to assess the suggested Creative Design Process (CDP).The CTS was introduced to the teachers during the workshops mentioned earlier.It consists of the four thinking levels that De Bono (1996) defined:

Awareness of Thinking
The first level deals with developing an awareness that thinking is a skill that can be developed.Pupils are taught how to prepare to engage in thinking about something, how to conduct inquiry, and how to listen to and evaluate the opinions of others.

Observation of Thinking
The second level deals with observing the consequences of action and choice, considering other people's views, and comparing alternatives.

Thinking Strategy
The third level deals with the directed use of some thinking tools, organizing thinking as a sequence of steps, and using thinking to define goals.

Reflection upon Thinking
The fourth level deals with a systematic use of thinking tools, clear awareness of the need for reflective thinking, self-evaluation of thinking, designing thinking tasks, and methods to implement these tasks.
The CTS evaluates the pupils' portfolios across two dimensions.The first considers the design, construction, and evaluation of the product or system.Evidence of lateral thinking, including originality, authenticity, usefulness, and unique design is sought.Likewise, evidence of vertical thinking is also sought and includes functionality, reliability, accuracy, geometric structure, and the application of scientific principles.
The second dimension considers the processes of learning and includes thinking, problem solving, and teamwork.Evidence is sought of individual and group efforts in problem solving, collaborative decision-making, and leadership.Table 2 presents the CTS, the development of which was detailed elsewhere (Barak & Doppelt, 2000).

Level 3: Strategy
The directed use of some thinking tools; organizing the thinking as a sequence of steps; using thinking to define goals.
Original system functional block diagrams, structural tree or flow chart.Description of a number of iterations in the planning and construction of the model.Comparison among possible models and choosing from them.
Examples of the contribution of individuals and teamwork to solving complex problems.Evidence of the planned use of the thinking tools, open-mindedness, and postponing decision making (lateral thinking); setting priorities, goals and criteria (vertical thinking).

Level 4: Reflection
A systematic use of thinking tools; clear awareness of reflective thinking; self-evaluation of thinking; designing thinking tasks and methods to implement these tasks.
Examination of the final product's features, compared to the set goals.Conclusions on successes or difficulties during the development process.Suggestions for improvement in the planning and construction process.
Conclusions drawn from the influence of the team's collaboration on the completion of the project.Pupils' view on the influence of the team's functioning on thinking and learning processes.Assessment of the selected solution compared to the goals.

Method
This study aimed first to investigate the way pupils design their projects.The second aim was to describe the assessment of Project-Based Learning (PBL).The third aim was to explore the ways teachers set goals for their pupils according to the CDP and the CTS.A qualitative approach was used in order to foster collaboration with teachers and to gain close interaction with pupils, teachers, principals, and supervisors from the Israeli Ministry of Education.
In 2003 there were nine schools that offered MECHATRONICS as a major during the 10 th -12 th grades (16-18 years old).Eighteen teachers were involved in delivering the MECHATRONICS courses.Approximately 180 pupils have graduated from high schools with MECHATRONICS as their major.Pupils' projects are examined through a matriculation examination at the end of the 12th grade.In these examinations a supervisor arrives at each school and the pupils are required to present a portfolio of their design process and the final product or system that resulted.Team projects are assessed the same as single projects.The examination supervisor asks questions of each of the pupils in the team.
A national contest was organized for all the Israeli pupils in the MECHATRONICS programs.All the pupils knew at the outset of the contest that they would be required to design, construct, and to program a system that would assist humans.Criteria for assessing the projects were developed with the teachers.The teachers were instructed to introduce the criteria to their pupils six months prior to the contest.The assessment process during the contest itself focused on evaluating the design process and resultant products based upon a presentation made by the pupils.
Agreement was reached among the teachers on the criteria for assessing the pupils' work during a meeting six months before the contest.These criteria were validated in advance of this meeting through the agreement of five senior teachers from schools that were not participating in the contest and five researchers from the Technion -Israel Institute of Technology.These ten individuals were invited at the contest to serve voluntarily as members of the assessment committee.

Participants
In all nine schools the teachers assessed the projects using the developed criteria.Each school could send three teams to the contest.Fifty-four pupils were chosen by their teachers to participate in an Israeli national MECHATRONICS contest, which was held at the Technion.The pupils were in the second semester of their 12th grade and they had been learning MECHATRONICS according to the syllabus that was described earlier.The teachers of these pupils, eighteen from the nine participating schools, applied PBL in their classes during the 12th grade.

Data Collection and Analysis
As mentioned earlier, the CDP and the CTS had been introduced to the teachers during a workshop.The author actively participated in the workshop.During the workshop, criteria for assessing PBL were agreed upon among the teachers.These criteria were used to assess the pupils' projects in the national contest.Six month prior to the contest, the teachers and pupils became familiar with the assessment criteria.The pupils were required to present their projects during the contest.By way of these presentations, the researcher was able to investigate the implementation of the CDP and the impact of the CTS upon pupils' projects.

Findings
The findings are presented in three categories.First, projects are briefly described, showing the variety and authenticity of the pupils' work.Second, a representative example of the assessment process is presented.Finally, the judges' assessment of the MECHATRONICS projects in the national competition is presented.

The Projects
Fifty-four pupils were eligible to enter the national MECHATRONICS contest, representing 18 projects.Of these, three teams were unable to bring their projects to the final level needed to actually compete in the contest.In the end, forty five pupils representing 16 projects were in the competition.Only one project was done by a student working alone.

The Judging Process
The pupils presented their projects to an assessment committee consisting of teachers from the participating school, senior teachers from schools that were not participating in the contest, and researchers from the Technion -Israel Institute of Technology.Procedures were used to assure that a given project was not judged by a teacher of the pupils who produced it.
There were two stages to the assessment process.In the first stage, the projects were presented to teams of three judges.Approximately three projects were evaluated by each team of judges.From this procedure, six projects moved on to the second, or final, stage of the evaluation process.The descriptions in Table 3 show that the projects were authentic and varied.
Table 4 presents a representative project, which demonstrates the assessment process.This example is project No. 7 from the final assessment presented in Table 3.These findings show a high-level of agreement among the independent assessments of each criterion as scored by the judges.The scale for the scoring was 10 percent for the first and the last criteria, and 20 percentages for each of the other criteria.Judge 1 was a senior teacher who had not participated in the intervention but was familiar with the criteria.The correlation between judges 1 & 2 was 0.957; between judges 1 & 3 it was 0.990 and between judges 2 & 3 it was 0.944.Similarly, high correlations were found among all of the other assessments, which were scored by the other judging teams.The assessments at the same time and there was no interaction between one team and the other teams.This shows that the judges have a shared perception of the criteria.

The Final Assessment of the Projects
Table 5 presents the findings from the final assessment process.In the final stage the pupils presented the six projects, shown in bold in the table, to the committee in a large auditorium.In the auditorium 160 guests watched the final contest and included families, teachers, school principals, supervisors from the Ministry of Education, and researchers from the Technion.The assessment committee watched each presentation and assessed the six projects according to the developed criteria, as mentioned previously.The six projects were an automated controlled system for lifting a hoister weight, an automated multiplayer basketball game, a mini football game, a jogging simulator, an automated system for changing and playing compact discs, a computerized scanner, and a system for coaching ping-pong players.

Discussion
Seven years of experience in implementing the CTS in order to assess pupils' projects demonstrates how this methodological assessment can help educators develop and evaluate learning assignments aimed at fostering creative thinking in technology (Doppelt & Barak, 2002).Through the CDP and systematic reflection on it, pupils can develop awareness of their internal thinking processes and document them.The purpose is not to educate pupils to design according to some generalized procedure, external to them, for constructing their ideas, solutions, and products (De Vries, 1996).This is counter to creativity.Rather, it is an educational goal to teach them to document their thinking properly and thereby enable them to reflect on their creations and how they developed them.Research was recently conducted in this area by Doppelt, Mehalik, and Schunn (2005).
Pupils are expected to internalize their adaptation of the design process, to use it in their own way, to apply it to new situations, and to demonstrate general patterns of lateral and vertical thinking in their technology projects.No less important is fostering pupils' meta-cognition, or 'thinking of thinking'.The way pupils commence, progress, and complete their project demonstrates that creative thinking in technology is a combination of vertical and lateral thinking (Waks, 1997;Barak & Doppelt, 2000).
In addition, the projects reported herein show that pupils in high school can create, design, implement, control, and document authentic, real-life projects instead of solving well-defined problems prescribed by the teacher.In fact, the criticism of current engineering education is that there is an overemphasis on solving well-defined, closed-ended problems (NSPE, 1992).Furthermore, pupils have proven through their projects that they are capable of dealing with the "large definition of DESIGN" -that the DESIGN activity does, in fact, encompass the entire process of planning, designing, constructing, and managing the development of a product (De Vries, 1993;Hill, 1998).
The CTS has enabled teachers and researchers to set goals for the pupils (and for the teachers) during the PBL.The consistency of the judges' scores and the successful application of the criteria developed by the teachers strengthened their validity.The findings of the assessment process indicate that the CDP and the CTS are useful and can be implemented by teachers who have participated in a suitable in-service training.The assessment of technology education can serve as a highly integrative element in technology education that allows pupils to combine and integrate various knowledge and skills (De Vries, 1997).

Conclusions
The in-service teacher workshop assisted the teachers in discussing tutoring issues, design stages, and assessment criteria.The independent judging showed a high-level of agreement among the judges.These assessments were similar to the researchers' and senior teachers' assessment in the final stage of the contest.The researchers and senior teachers had not taken an active part in the tutoring process during the school year.They were familiar, however, with the criteria and had agreed upon them.This model can be adopted in other relevant issues regarding collaboration between field practice and academic research.
This article introduced the Creative Design Process (CDP) and the creative thinking scale (CTS).The CDP is aimed at assisting pupils in documenting the design process.The CTS could be used as a guideline for teachers during their tutoring and for pupils during the development of creative solutions to problems.The findings showed that pupils learned to document their design process according to the CDP.Finally, teachers turned into better tutors after they became familiar with the CTS.
The implementation of the CTS concerning the outcomes of the CDP has important consequences for the professional development of teachers and for the development of pupils' skills.Teachers can use the CTS as the goal of their teaching.If the CTS is introduced together with the CDP to pupils, they can develop their competencies according to various learning styles.The methodological assessment used in this study during the intervention program with the teacher and during the contest with teachers, senior teachers and researchers can be used in other science and technology domains.The contest was also found to be a useful instrument to enhance collaboration between researchers and teachers and among schools.This research could add a relevant body of knowledge to the assessment of technology education.

3 .
Consequence and Sequel (C&S) -Pupils must consider and document all of the consequences of each of their ideas on such elements as the environment, society, and the individual.Both short term and long term consequences must be considered.4.Other People's View (O.P.V):-Pupils must seek the opinions of others about their ideas and they must document what they find.Fourth stage: Choosing the Preferred Solution.-The fourth step in the design process is choosing the preferred solution.The choice is made from the various ideas that were documented in the third stage.The solution chosen will possibly fulfill the following criteria: 1. Have the largest amount of plus and/or interesting points and the least minus points 2. Considers as many factors and viewpoints as possible.3. Is adequate in both the short and long term.4. Appears to be a good solution in the minds of others.5. Meets all the requirements set forth in the definition of the problem.

Table 1
is designed to assist pupils in choosing the best solution.An evaluation scheme for evaluation pupil ideas

Table 2
The Creative Thinking Scale (CTS)

Table 5
Final scoring of each participating project