M-21 CSI : A Validated 21 st Century Skills Instrument for Secondary Science Students

The tremendous change in the 21 century is the emerging nature of a new set of economic and social indicators which eventually capture the changes in structural transformation, technological advance, and competition in the job market. This paper aims to discuss the development and validation process of the Malaysian 21 century skills instrument (M-21CSI) to be used within the teaching and learning of science processes. The M-21CSI consists of five distinctive elements which are: i) Digital age literacy; ii) Inventive thinking; iii) Effective communication; iv) High productivity; and iv) Spiritual value. Through an extensive review of the literature and group discussion, constructs that represent the Malaysian 21 Century Skills were identified and further refined by Two Round Delphi Studies. Subsequent to that, items representing each construct were then drawn up. In order to justify the validity of this instrument, the questionnaire built has been tested its validity by face validity, language and content validity. This then was followed by factor analysis. The reliability of each construct was justified in terms of its internal consistency by means of Cronbach Alpha. Findings of this study confirmed the validity and reliability of the M-21CSI and thus indicated that it is a useful instrument to evaluate the mastery of Malaysian students towards 21 century skills.


Introduction
In the 21 st century, Malaysia faces new challenges due to globalization, liberalization, internationalization and the development of information and communication technology (ICT).It is important for the educational system to make parallel changes in order to fulfil its mission to the society, namely the preparation of students for the world beyond the classroom.Therefore, the educational system must understand and embrace the 21 st century skills within the context of rigorous academic standards.To overcome these challenges, efforts have been made in the education system through curriculum content and learning activities that lead to the development of the physical, emotional, spiritual and intellectual relevance to current needs and future.
Taking into account the many changes and challenges in the 21 st century, some improvements and strengthening the education system have been done in the National Education Blueprint 2006-2010(PIPP 2006-2010).Among them are such as to formulate policy in Teaching and Learning Science and Mathematics in English (PPSMI) in 2003, the establishment of Central Schools, the International School of Policy changes towards making an international school in Malaysia competitive in Asia, etc. (Ministry of Education, 2006).
In order to succeed in this digital economy, students will need digital age proficiencies.In the Malaysian context, the Ministry of Education (MOE) has taken several initiatives to foster students' skills in using ICT, This effort involved the Smart School Pilot Project conducted in 1999 and 2002 in 88 schools in which a Smart School Flagship Corridor Multimedia emphasized the use of ICT as an enabler of pedagogical and school management in order to keep pace with the rapid regional information and communications technology.The above initiative is to ensure Malaysia's Vision 2020 can be realized; which is to "establish a scientific and progressive society, a high-power changes and look ahead, which is not only a consumer of technology but also a contributor to scientific and technological civilization" (Ministry of Education, 2006).
There has been numerous reports on the global, knowledge-based economy and the flat world that tomorrow's workers must be prepared to shift jobs and careers more frequently, to be flexible and adaptable in acquiring job skills, to integrate and focus a changing mix of job-derived and education-based knowledge on business processes and problems (Friedman, 2005).Moreover, OECD ( 2004) confirmed the application of information technology to the very core of business operations has caused a profound change in the needed skills and talents of New Economy workers.Markets in the New Economy are rewarding those who have high educational achievement and technical skills (Task Force on the Future of American Innovation, 2005).
The globalization of the 21 st century has also brought challenges of language literacy and communication to a developing country such as Malaysia.To realize human development as envisaged in the society of the 21 st century, science education in Malaysia is expected to attempt the students with the physical, emotional, spiritual and as well as intellectual.As citizens of the 21 st century, we are obliged to participate in the mainstream of development; we must try very hard to train ourselves with skills that enable us to accept the challenges of the 21 st century.Therefore, the aim of this research is to develop and validate the instrument based on the Malaysian 21 st century skills (M-21CSI) to capture students' mastery in the skills identified in this study.

The 21 st Century Skills
Education is the cornerstone of all efforts to produce skilled manpower.The education sector in our country is changing dynamically, and is expected to undergo many changes in the 21 st century.The objective of making Malaysia a center of excellence in education in Asia and also at the international level demanded the country to improve the quality of education to world class education.The 21 st century promises a sophisticated era as a result of developments in science and technology (S&T).Therefore, the strategic paradigm must be made so that Malaysia could face a more challenging global environment.Educational paradigm should be changed from receiving to creating, rather than adhering to the commitments, from individual to partnership, from routine to innovative and creative.The shift paradigm is not denying the needs of the existing things, but to shift the boundaries of thought to something new (Ibrahim 2001).The 21 st century skills were emphasized because what the world is facing at present as well as the future is very challenging.From the literature review conducted, there are many models related to the 21 st century skills, including enGauge 21 st Century Skills (2003) and The Partnership of 21 st Century Skills (2002).
The elements of the 21 st century learning outcomes are the skills, knowledge and expertise students should master to succeed in work and life.According to the enGauge (NCREL 2003), 21 st century skills consist of digital age literacy, inventive thinking, effective communication and high productivity.Digital age literacy refers to the ability to use digital technology, communications equipment and / or networks to access, manage, consolidate, evaluate and create information in the knowledge society.There are components under the digital age literacy namely, basic literacy, scientific literacy, economic literacy, multicultural and global awareness.Basic literacy is the skills of reading, writing, listening and speaking in their mother tongue and the international language of English as well as the ability to give meaning and express ideas through a variety of media such as the use of ICT.Scientific literacy is a basic science subject to make connection with S&T to make something interesting.Multicultural literacy is a society that is harmonious, united which thrives in a peaceful and prosperous environment.It is closely related to the global awareness of literacy skills because the definition of global awareness itself is to satisfy the capacity to understand and identify the relationships among world organizations, countries, communities and economic and socio-cultural groups and individuals across a global community (NCREL & Metiri Group, 2003).
Inventive thinking skills consist of adaptability and managing complexity, self direction, curiosity, creativity, risk taking, higher order thinking and sound reasoning.Adaptability and managing complexity refer to the ability to handle multiple goals, tasks, and inputs, while understanding and adhering to the constraints of time, resources and systems.Whereas, self direction refers to students' ability to set goals related to learning, plan for the achievement of those goals, independently manage time and effort, and independently assess the quality of learning and any products that results from the learning experience.Curiosity refers to the students' desire to learn more about something and is an essential component of lifelong learning.Next, creativity is the acts of bringing something into existence that is genuinely new and original, whether personally or culturally.The student's capacity to think about a problem or challenge, to share that thinking with others and to listen to feedback is known as risk taking.Higher order thinking and sound reasoning refer to the cognitive processes of analysis, comparison, inference and interpretation, evaluation and synthesis which applied to a range of academic domains and problems solving contexts.The students are able to compare analysis, make inference and interpretation, evaluation and solve problem solving in the tasks given to them and in their everyday life.
Effective communication comprises of teamwork and collaboration, interpersonal skills, personal responsibility, social and civic responsibility and interactive communication.Teamwork and collaboration refers to the skills of students in making an assignment or working with their group members effectively.Whereas, interpersonal skills refer to the ability of understand the feelings, motivations, habits and aspirations of others.This person who has interpersonal skills can interact easily and work with others to produce a practically useful and can provide motivation to others according to the theory of multiple intelligence (Gardner, 1983).Interactive communication is one way of visual communication using a combination of text and / or voice for the communication to interact with each other.In the context of science education, initiatives to integrate the use of ICT and multimedia in teaching and learning are necessary because of the increasing competition in order to create IT-literate society.This would result in a more interesting and meaningful learning (Heather, Michael & Sylvia, 2002).enGauge (2003) stated that high productivity refers to prioritizing, planning, managing for results, effective use of real world tools and ability to produce relevant and high quality products.The first concept of prioritizing, planning, and managing for results is the outcome of using and managing the time and resources efficiently, solving problems effectively and also possessing strong leadership skills.The second concept is the use of real-world tools to effectively control the students in mastering and using the newest and latest technology.The concept of high productivity is the ability to produce relevant and high quality products which are informative and original.
The Partnership of 21 st Century Skills (2002) clearly states the importance of the 21 st century skills as essential asset to create a scientific, progressive, creative and insightful society.The society is expected to not only take advantage of new technologies but to also contribute to the creation of scientific and technological civilization of the future.According to the Partnership definition, 21 st century learner must learn the following subjects and complementary skills; i) Core academic subjects include English, world languages, arts, Maths, Economic, Science, Geography, History, government and civics; ii) Interdisciplinary themes to be embedded into each subject such as global awareness, financial, economic, business and entrepreneurial literacy, and health literacy; iii) Learning and innovation skills infused into each subject which include creativity, innovation, critical thinking, problem solving, communication and collaboration; iv) Information, media and technology skills required of today's demands including information literacy, media literacy, communications and technology literacy; and v) Life and career skills that are the survival skills needed to navigate in today's fast paced, high technology world.
Analysis of the literature reveals that teachers play a very big and important role in generating the 21 st century skills students, for instance in the inventive skill that demands a high level of thinking.However, due to time constraints and curriculum practices higher levels of thinking are hard to be encouraged by teachers, except on their own initiative (Zohar & Schwartzer, 2005).Based on these constraints, it is believed that more effective teaching techniques can help the integration of 21 st century skills using student-centered methods such as problem-based learning, project based and other techniques that help students develop the skills to face the challenging era.Developed countries often use this approach in the classroom and are believed to produce students who can face the world of the 21 st century.It is hoped that this study will contribute to knowledge of science students' performance in 21 st century skills.In addition, this study will also be able to provide the relevant authorities such as the MOE of Malaysia with information regarding the students' achievements and their level of 21 st century skills in science subject.

Research Design
This study employed cross sectional survey method with the purpose of identifying the Malaysian 21 st century skills elements (M-21CSI) for science students.The research task was best explored through qualitative methods.Through an extensive review of literature and focus group discussion, constructs that represent the Malaysian 21 st Century Skills were identified and further refined by two round Delphi studies.A two round-Delphi process was used to develop a consensus among 40 panels to identify the elements of the 21 st century skills (Osman & Marimuthu, 2010).The panelists for Delphi study were composed of 40 members from five different categories, which mainly included science and technology university lecturers, industrialists, science expert teachers, school administrators and science students (Osman & Marimuthu, 2010).

Sample of the Study
In this study, 760 responses were collected from form four students from 14 secondary schools throughout the state of Selangor, Malaysia.The researcher employed stratified sampling in this study to ensure that each member of the population was equally likely to be chosen as part of the sample.

Instrument of the Study
The development of the Malaysian 21 st Century Skills Instrument (M-21CSI) used the three-stage approach.Stage 1 began with the identification of salient scales.Stage 2 involved the writing of individual items within the scales and Stage 3 involved the field testing items followed by an item analysis and validation procedures.Below are descriptions of the steps involved in each stage.

Stage 1 -Identification and Development of Salient Scales
Stage 1 consisted of three steps that led to the identification and development of salient scales.The first step included reviewing the literature related to the 21 st century skills instrument.The major sources of the elements in the skills mainly were adapted from the enGauge 21 st century skills ( 2003) and the Partnership of 21 st century skills (2002).This crucial step sought to identify key components that researchers, educators and practitioners would consider as the skills needed in the challenging era.
The second step was conducting a face-to-face interview and focused group discussion.A group of experienced science teachers was approached to get their opinions regarding the 21 st century skills issues and also concerning the teaching and learning scenario.
The third step was to classify and rearrange the newly developed scales in the M-21CSI.In order to get the final domain of M-21CSI, the two round Delphi techniques were held (Osman & Marimuthu, 2010).The Delphi study allows panelists to reach consensus on idea and to contribute new ideas in identifying the elements of 21 st century skills that should be integrated in science subjects.The panelists for Delphi study were composed of 40 members from five different categories, which mainly included science and technology university lecturers, industrialists, science expert teachers, school administrators and science students (Osman & Marimuthu, 2010).After the first round of Delphi conducted, the results revealed some additional suggestions provided by members of the panel to be included in the questionnaire that focused on spiritual values.Thus, after taking into consideration these factors, 21 st century skills in this study cover five major components of the digital age literacy, inventive thinking, effective communication, high productivity as well as spiritual values.
Figure 1.below shows the conceptual framework of this study which consist of five newly developed scale in the M-21SCI which are digital age literacy, inventive thinking, effective communication, high productivity as well as spiritual values.In the process of teaching and learning, the teachers need to apply an integrated approach as the backbone in order to create the desired INSAN (human capital) with critical and creative thinking skills.This was illustrated by five scales which directed towards to form a balanced INSAN (human capital) with critical and creative thinking skills in a pentagon.Each corner of the pentagon means that all five scales of 21 st century skills were inculcated during the process of teaching and learning.Meanwhile, the arrows from each of the corner shows the direction, goals and objectives to be achieved.

Stage 2 -Writing Individual Items
Based on the Malaysian 21st century skills instrument (M-21CSI), a set of questionnaire was drawn up.Each skill component consisted of sub-constructs which were the guide to develop a set of questionnaire in recognizing students' performances.Table 1 presents the number of items developed based on the domain of M-21CSI.Besides that, the entire set of items was shown to a panel of experts to ensure the face and content validation of the instruments.

Sample of questionnaires
Digital Age Literacy 24 I can understand the science concepts taught in English well.
I think the issue of environmental pollution must be addressed.
Create a discussion with friends using technologies such as chat room to solve the problems.
Inventive Thinking 42 I always think a variety of methods and perspectives in solving problem.
I set a goal of learning.
I always make a hypothesis of my observations on the environment.
Effective Communication 15 I listen to the opinion of other members with respect.
I am able to overcome the conflicts that arise between members of the group.
High Productivity 18 I sort tasks by priority.
I use appropriate technology to projects that I do.

Spiritual Values 7
I feel grateful to be able to learn science.
I realize that S&T enables human to use their God-given talents wisely.

Stage 3 -Validity and Reliability Analysis
The development of the Malaysian 21 st century skills instrument (M-21CSI) utilized the intuitive-rational strategy in which only items with high internal consistency remained in the final instrument.This section describes the methods in which the M-21CSI questionnaire was refined and its validity and reliability were determined.

Validity of the Instruments
According to Jackson (2006), validity is a measure of the extent to which the instrument can measure what should be measured in the study.In order to justify the validity of this instrument, the questionnaire developed has been tested in terms of its validity by face, language and content validity.A science teacher with 12 years teaching experience conducted the face validity, whereas the language was checked by a teacher with 20 years teaching experience.For the content validity, two experts in the field of science education from Universiti Teknologi Malaysia and Universiti Kebangsaan Malaysia were involved.
Next, construct validity was investigated as described below using principal component factor analysis with varimax rotation and Kaiser Normalization.The aim of the factor analysis was to ascertain the fundamental structure of a comparatively large set of variables (Garson, 2001).In the confirmatory factor analysis, the first step involved extraction of factors via principal component analysis.By doing so, certain Eigenvalue represented by a certain percentage of variance was generated.The Eigenvalue represented a measure that attached to the factors and indicated the amount of variance in the pool of the original variables that the factors explained.Each construct (factor) was retained only if its Eigenvalue was more than 1.
The second step involved an additional procedure called factor rotation.Varimax rotation method was used due to its advantage in producing factor (construct) that was free and independent upon one another.By doing so, the subsequent factor interpretation was relatively easy (Blakenship & Moore, 1977;Bryman & Cramer, 1998).The determination of whether the items contributing to factors were determined by factor loading based on the cut-off values.As recommended by Hair et al. (2006), the 0.4 cut-off values were considered acceptable.Using the procedures mentioned above, five factors were successfully extracted, which as a whole contributed to 51.81 percent of the overall variance and were considered acceptable.In the development of the M-21CSI, each item was assessed for its internal consistency.This was a measure of the extent to which items within a scale measure the same construct as the other items in the same scale.Those items that were not highly correlated with their respective scales were removed and data were re-analyzed until all of the items with the lowest item-scale correlations were removed and the alpha coefficient was maximized.Table 3 describes the scale reliability using the Cronbach alpha coefficient for a set of questionnaire based on the M-21CSI.Using a generally applied standard, this range is considered "acceptable" to "good" (Goerge & Mallery, 2001), since the closer the alpha is to 1, the greater the internal consistencies of the items are.

Discussion
Most of the relevant studies have been conducted in the US or Australia, which are not representative of the Malaysian environment.Therefore, the Malaysian 21 st Century Skills Instrument (M-21CSI) was developed to capture the skills toward Malaysian students in 21 st century.The main difference between other 21 st century skills with M-21CSI is the inclusion of the element of spiritual values which reflects Malaysian identity.In this demanding environment, student's achievement must be broadened to include the 21 st century skills that will be required for students to thrive in the future.Information and communication technologies are raising the bar on the competencies needed to succeed in the 21 st century.These skills may ensure that the students are prepared for a better future.
As society changes over time, more skills need to be acquired in order to prepare for a better future.Academic excellence in itself alone may not be sufficient to ensure success (Osman & Marimuthu, 2010).The definition of student's achievements must be broadened to include the 21 st century skills that will be required for students to thrive in the future.Students ought to have the ability to apply the knowledge that they have learned to face the challenges of life beyond school.Basic skills are necessary but not sufficient in itself for job security.Thus, the 21 st century skills as identified in this study comprised some important skills that are needed in the digital age.It is a current trend in education where students are able to solve multifaceted problems by thinking creatively and generating original ideas from multiple sources.
Based on the analysis conducted, the questionnaire developed based on the Malaysian 21 st century skills instrument (M-21CSI) was found to have a high reliability and also good construct validity of which can be used in the study to identify the 21 st century skills among the students in the context of the teaching and learning process.The instrument can be used to identify whether the 21 st century skills were developed during the teaching and learning processes in the classroom or while doing outdoor activities.This instrument also has its own uniqueness as it can be used to measure the Malaysian 21 st century skills identified as digital age literacy, inventive thinking, effective communication skills, high productivity and spiritual values.Each component in the 21 st century skills also has its own skill set, which offers guidance to recognize students' performances.The components of the Malaysian 21 st century skills instrument were regarded as highly important and relevant.These skills may ensure that the students are prepared for a better future.It is hoped that this study will be able to provide the relevant authorities such as the MOE of Malaysia with information regarding the students' achievements and their level of 21 st century skills in the science subject.

Conclusion
This paper has reported the development and validation of the Malaysian 21 st century skills instrument (M-21CSI) designed to assess the 21 st century skills mastered by the science students in secondary schools.The development of the M-21CSI is as an alternative to identify the necessary skills needed to succeed in the 21 st century.In this era, improving students' outcomes is crucial to develop a more competitive workforce as Malaysia pushes towards being a developed nation by 2020.In order to be successful in inculcating these skills, it is argued that three significant mechanisms are in demand.First, the public at large must acknowledge the 21 st century skills as essential to the education of today's learner.Second, schools must embrace new designs for learning based on emerging research on how people learn information processing, effective uses of technology and the 21 st century skills in the context of rigorous academic content.Lastly, policy makers must base school accountability on assessments that measure both academic achievement and the 21 st century skills.

Figure 1 .
Figure 1.The conceptual framework of this study

Table 1 .
Domain of the Malaysian 21st century skills instruments (M-21CSI) and the sample of questionnaire

Table 2 .
Factors that have been extracted and percentage of variance

Table 3 .
Scale reliability using Cronbach's alpha coefficient for the questionnaire of M-21CSI