Educators ’ selection and evaluation of natural sciences textbooks

Due to the central role of textbooks in the teaching and learning of science, it is imperative that textbooks provide correct content and instructional support. We investigated how 16 South African Grade 7 natural science educators selected their textbooks and how they evaluated these textbooks. The results were analysed according to the constructivist paradigm, as implemented in the Revised National Curriculum Statement, as well as selection criteria found in the literature. The selection criteria that the educators listed in interviews were isolated ideas that were not embedded in a constructivist framework. The educators also lacked the necessary scientific and pedagogical content-knowledge to effectively evaluate the textbooks. Recommendations are made for improvements in educator training, departmental guidance, and textbook writing. Experience in textbook evaluation during their training courses could boost educators’ competence in selecting the books and detecting and compensating for their deficiencies. This would contribute to filling South African educators’ need of hands-on examples that can be applied immediately in their classrooms.


Introduction
Science textbooks are primary sources used by science educators throughout the world to guide them in teaching the content and skills prescribed in the curricula (Chiappetta et al., 1991;Hubisz, 2003;Leite, 1999;Stoffels, 2005).Educators as well as learners spend a great deal of their preparation, class and homework time working with textbook materials (Nicol & Crespo, 2006: 331).Textbooks are expected to provide a framework for what is taught, how it may be taught and in what sequence it can be taught.Of the many factors, which promote or impair science learning, the textbook is one of the most critical (Leite, 1999;Hubisz, 2003).
Many middle-school educators in the United States of America (USA) do not have sufficient training in physical science (Hubisz, 2003).The situation is similar in South Africa (Stoffels, 2005).Textbooks can be especially helpful to the inexperienced educator (Dreckmeyr et al., 1994).A good textbook can help these educators to keep ahead of most of their learners and learn as they go (Hubisz, 2003).However, educators should be flexible enough to break away from the textbook presentation to answer questions from enthusiastic learners or give more examples and explanations to help those that struggle.South African educators are expected to design and develop learning materials according to the needs of their learners (Department of Education, 1998).Rogan (2004) found that very few educators could do this.Consequently they opted for curriculum-aligned support material that was commercially prepared (Stoffels, 2005).
Choosing the appropriate textbook for use in the science classroom is not an easy task (NSTA, 2005).A textbook is an interpretation of the curriculum, guided by the authors' world-views, values and presuppositions (Leite, 1999).Therefore, the textbook message is neither neutral nor a faithful specification of any curriculum.Textbooks are also under the influence of constraints, such as economics, and precedents set by states (Hubisz, 2003).As the users of the textbooks and assumed knowledge experts, the responsibility to choose the textbooks for their learners is often given to the science educators themselves.The first research objective in this study was to investigate the criteria used by Grade 7 science educators to select their textbooks.Sixteen educators (one each of 16 schools) from the Potchefstroom (North-West Province) region contributed to the study through structured interviews.
The Project 2061 of the American Association for the Advancement of Science (AAAS, 2006) executed in the USA found that not one of the middlegrade science textbooks was satisfactory; i.e. had a high potential for helping students learn ideas essential for science literacy (Roseman et al., 2001).The textbooks covered too many topics without developing any of them well.According to Hubisz (2003), most middle-grade textbooks in the USA are pedagogically ineffective and scientifically inaccurate.The question arises whether South African textbooks come up to expectations.The second research objective examined the 16 Grade 7 educators' evaluation of their science textbooks.
One of the key ideas generally taught in Grade 7 in South African schools relates to the topic of heat.In the evaluation of textbooks, we focused on the concept of heat due to its importance for the study of thermal, chemical, and physical processes.In the Revised National Curriculum Statement (Department of Education, 2003), abbreviated to RNCS, the concept of heat occurs under both the themes of Energy and Change and Matter and Materials.An additional reason for the selection of this key idea was that concepts related to heat (e.g.heat transfer processes and energy degradation) lend themselves to the teaching of inquiry skills, problem-solving strategies and contextual applications, as is inscribed in the RNCS.

Theoretical framework
Constructivism is the most generally accepted paradigm used in education today.A key aspect of this theory is that knowledge about the outside world is viewed as human construction (Duit & Treagust, 2003).Learning is consequently not viewed as a transfer of knowledge, but that learners are actively constructing or creating their knowledge, based on their experiences and existing knowledge.Learning may occur through the addition of knowledge to current concepts, creation of new concepts, or major modification of current concepts (Bybee, 2002).This process of constructing meaning is embedded in a particular social setting of which the learner is part (Ensor et al., 2002;Duit & Treagust, 2003).
Through the process of meaningful learning (Ausubel, 1963), instruction should build on learners' prior knowledge (i.e.their existing concepts and preconceptions) seeking to integrate new knowledge into their cognitive structures (Novak, 2004).Therefore, the prior knowledge and especially the alternative conceptions that learners bring to the classroom should be taken into account before new concepts are introduced.In science education alternative conceptions are perceived as ideas that are inconsistent with the current scientific meaning (Thijs & Van den Berg, 1995) and are consequently regarded as non-scientific.Several alternative conceptions of the concept of heat have been reported in literature (Erickson & Tiberghien, 1989;Driver et al., 1994;Leite, 1999), e.g.heat is a fluid, a substance, energy possessed by an object, or the same as temperature.
The constructivist paradigm indicates criteria for the progression and presentation of the scientific content in textbooks.Tarr et al. (2006) discuss three key dimensions that provide a general framework for reviewing and selecting mathematics textbooks, namely, content emphasis, instructional focus and teacher support.These dimensions can equally well be applied to the selection of science textbooks.The first framework emphasises the content and includes alignment with the learning expectations presented in the curriculum, in-depth presentation of topics with increasing sophistication across grades, and an appropriate balance of skills development and understanding of concepts and processes.Dreckmeyr et al. (1994), Leite (1999), andHubisz (2003) stress the following criteria: scientific accuracy, treatment of alternative conceptions, conceptual development, appropriate level of presentation, and suitable everyday applications of the content.
The second framework of Tarr et al. (2006) for textbook selection is instructional focus.This includes that activities should foster the development of science as a human endeavour and a way of thinking, that learners should share their ideas and that inquiry, reflection, critical thinking, problemsolving, and making sense should be promoted.Dreckmeyr et al. (1994) adds that activities should be interesting and relate to learners' experiences.Activities should link learners' pre-knowledge with scientific knowledge.In Project 2061 (AAAS, 2006), the analysts looked for the following instructional characteristics in the books they evaluated (Roseman et al., 1999), namely, whether the books • alerted educators to commonly held learner ideas, • provided learners with a variety of phenomena, • guided learner interpretation and reasoning, • provided practice in using scientific ideas, and • demonstrated the use of knowledge.With regard to educator support (their third framework) the question to be answered was whether the textbook enhanced the quality of instruction (Tarr et al., 2006).According to Leite (1999), textbook writers should make their pedagogical assumptions explicit in educator guides.They should explain the "how" and "why" of a textbook's organisation.Apart from assessment tools, exercises and projects, textbooks should provide opportunities for educators to increase their own understanding of scientific ideas and also include common alternative conceptions that should be addressed (Tarr et al., 2006).
In addition to these three frameworks, Dreckmeyr et al. (1994) and Hubisz (2003) refer to the technical aspects of textbooks.They emphasise the quality of the book itself as well as of the illustrations, sketches and graphs.The text should be well laid out and readable.Lemmer and Edwards (2007) add that textbooks should provide for cultural and gender differences and different learning abilities and styles.They found that most of the Grade 7 science textbooks explained any difficult or "new" English words that occurred in the text.
The Revised National Curriculum Statement (Department of Education, 2003) is learner-centred and recognises the principles of constructivism.Consequently, it supports the criteria for textbooks given by Tarr et al. (2006), Hubisz (2003), andDreckmeyr et al. (1994).The purpose of the natural sciences learning area is to promote scientific literacy through three learning outcomes, namely: 1.The development and use of science process skills in a variety of settings; 2. the development and application of scientific knowledge and understanding; and 3. appreciation of the relationship and responsibilities between science and society.The assessment standards for accomplishing the first learning outcome involve the planning of investigations, collection and evaluation of data, and communication of findings.For the second learning outcome, the learners need to recall meaningful information, categorise and interpret information and apply knowledge.The assessment standards for the third learning outcome entail the learner understanding science as a human endeavour, as well as the sustainable use of the earth's resources.Edwards (2007) and Lemmer and Edwards (2007) evaluated five natural sciences textbooks currently used for Grade 7 in South Africa.They found that outstanding features were the variety of activities, emphasis on collaborative learning, and the activities related to the learners' everyday experiences.However, the level and cognitive demands of the instructional activities and assessment tasks were generally too low.Most of the activities entailed direct application of knowledge, with few investigations.The science content was mostly presented in a fragmented way without proper progression and coherence.For instance, in four of the five textbooks evaluated, the concept of heat was presented without introduction of the concept of temperature.A temperature difference will cause heat flow.A discussion of the concept and measurement of temperature preceded the introduction of the concept of heat in most of the textbooks that Leite (1999) evaluated.
All but one of the five textbooks failed to apply the constructivist principle to take learners' alternative conceptions into account before a new concept is introduced (Lemmer & Edwards, 2007).None of the educator guides advised on how to remedy these conceptions.Errors and scientific inaccuracies that Edwards (2007) pointed out in the textbooks could cause or enhance alternative conceptions (Odom, 1993).Lemmer and Edwards (2007) warn that scientific accuracy should not be sacrificed when simplifying the work for lowergrade learners. .

Research design
A qualitative phenomenological study was conducted to probe into the way Grade 7 science educators select and evaluate their textbooks.Qualitative research approaches study phenomena that occur in natural settings (Leedy & Ormrod, 2005).A phenomenological study attempts to understand people's perceptions, perspectives, and understanding of a particular situation or event.In this study we focused on educators' perceptions and perspectives regarding the quality and appropriateness of their textbooks.
A questionnaire was compiled and permission to administer it to Grade 7 science educators was granted by the acting regional executive manager.The questionnaire was given to all 37 schools in the Potchefstroom region to be completed by the Grade 7 science educators.Only 5 educators responded, which was too low a response rate.According to Leedy and Ormrod (2005), a low return rate for questionnaires is typical.Consequently, the researchers personally visited schools in the region and the questionnaires were completed during structured interviews.Since this process was time-consuming, only 16 schools could be visited and the senior Grade 7 science educator of each school interviewed.Two researchers were present at each interview.All responses and comments of the educators were recorded.An advantage of the structured interviews was that the educators did not only answer the questions, but gave additional motivations, opinions, and impressions that provided depth to the investigation.
The questionnaire used during the interviews consisted of two parts: the first investigated the selection of textbooks and the second the educators' evaluation of the textbook they were using in class.The selection criteria of the educators were compared to those used by researchers such as Tarr et al. (2006), Hubisz (2003), and Dreckmeyr et al. (1994), while their evaluation of the presentation of the topic of heat was assessed against the requirements of the constructivist theory as implemented in the RNCS.

Results and discussion
The interviews revealed that 10 different Grade 7 natural sciences learners' books were being used in the 16 schools -every school used only one book.At the time of the study 18 books were approved by a governmental evaluation committee.Four of the 16 educators used the same book, while the other nine books were each used in only one or two of the schools.Edwards (2007) showed a diversity of instructional approaches, order and depth in the presentation of the content in Grade 7 textbooks.This diversity has consequences for learners that move to other schools during a phase.They not only have to adjust to a new school environment, but also to a different approach and sequence used in their textbooks, and consequently by their science educators.
The results of the questionnaire used in the structured interviews are summarised in Tables 1 and 2. In 10 out of the 16 schools (63%) the science educators selected their textbooks themselves, while the subject head selected the books in the other six schools (Question 1).Twelve of the educators (i.e.75%) preferred to select the books themselves (Question 2), since they knew their learners' needs and abilities as well as their environmental contexts.However, all the educators indicated that they would appreciate criteria and guidelines from the Department of Education.One of their largest challenges in selecting textbooks was that the educators were expected to select books even if they had yet to attend a workshop on the RNCS.Books were sent to schools for selection purposes early in the academic year.Workshops by subject specialists or advisors were conducted late in the year, long after textbooks had been selected for use in the following years.New textbooks were purchased approximately every five years in 10 of the schools (Question 3), due to financial constraints.A consequence was that a number of schools were not yet using textbooks based on the revised curriculum statement (Department of Education, 2003).In the fourth question of the questionnaire, the educators had to give criteria that they used when selecting a science textbook.We combined and grouped their criteria under the headings: content, instruction, teacher support, and technical aspects (Figure 1).These selection criteria to some extent

Instruction
• Activities should take learners' background and environment into consideration.
• A variety of activities should be included.Provision should be made for both group and individual activities.
• Activities, tasks and assessment should be at the appropriate level.
• Experiments should be executable.
• Tasks should be accessible.

Educator support
The educators' guide should give • Sufficient information for proper presentation of the work • Practical hints

• Assessment guidelines
Technical aspects

•
Textbooks must be affordable.
• Sufficient sketches should be included.
• Sketches should be clear.agreed with those given by other researchers, although there were essential differences.Alignment with the learning expectations and requirements presented in the curriculum statement (Tarr et al., 2006) was considered of utmost importance by all the educators.The learning outcomes and assessment standards should be indicated for all activities.Some educators mentioned scientific accuracy and the appropriate level of presentation (Leite, 1999, Hubisz, 2003) as criteria.Vital aspects that the educators did not consider with reference to the content included: in-depth presentation of topics with increasing sophistication across grades (Tarr et al., 2006); the scientific approach and historic development of science (Hubisz, 2003); streamlined texts that were focused around important ideas (Schmidt in Walton, 2002), interconnectedness across strands (Tarr et al., 2006), and treatment of alternative conceptions (Leite, 1999, Hubisz, 2003).
In connection with instruction, the educators focused on the variation, executability, and assessibility of activities and tasks (Figure 1).Other impor-tant requirements given by Tarr et al. (2006) were not referred to, namely, that activities should foster the development of the subject as a human endeavour and a way of thinking, and that activities should promote inquiry, reflection, critical thinking, problem solving, and sense making.Only one of the five instructional characteristics of science textbooks (Roseman et al., 1999) were mentioned by the educators, namely, that learners should be provided with a variety of phenomena.None of the interviewed educators used provision for gender and cultural differences or different learning abilities and styles (Lemmer & Edwards, 2007) as criteria for textbook selection.
The interviewed educators indicated that educators' guides helped them greatly in their preparations and made their task easier.However, they needed more guidelines to conduct experiments and facilitate activities.None of the educators asked for the inclusion of possible alternative conceptions and how these could be remedied (Tarr et al., 2006).Technical aspects referred to by the educators in the interviews focused mainly on sufficient, clear sketches (refer Figure 1).Care should, however, be taken that pictures, sidebars, and capsules do not interrupt what little text there is (Hubisz, 2003).While the educators expected a textbook to be affordable, Hubisz (2003) stresses durability of the book.
The remaining questions of the questionnaire (Questions 5 to 10 in Table 2) investigated the educators' evaluation of their textbooks' presentation of the topic of heat.Ten of the 16 educators indicated that they only used the selected prescribed book to present heat to their Grade 7s (Question 5).The other six educators used additional material, especially to obtain more challenging or interesting experiments or assessment activities.They complained that the experiments provided in their textbooks were too few and too simple.Some of the educators admitted during the interview that they preferred traditional experiments that gave step by step instructions.This was in accordance with the findings of Stoffels (2005), namely, that educators still tend to follow a typical 'cookbook' approach and even simplify most of the suggested problem-solving experiments by transposing them into educator demonstrations.
All but one of the educators asserted that their textbook adhered to the requirements of the RNCS (Question 6).The reason given by most of them was that outcomes were provided at the beginning of each unit or lesson or that learner activities and assessment tasks related to the learning outcomes and assessment standards.However, none of the educators investigated how the outcomes and assessment standards were interpreted by the writers of textbooks.Some of the educators checked whether the topics in the textbook covered the prescribed core knowledge, but none of them ensured that the text was presented in depth and was scientifically correct.Kulm et al. (1999) warned that although many different textbooks could cover the same topics, they could differ greatly in the specific ideas or substance that they covered.Edwards (2007) found errors and alternative conceptions in the Grade 7 textbooks that she investigated.The errors occurred mainly due to oversimplification of explanations or translations from English to Afrikaans.In this way alternative conceptions could be created or strengthened in learners, which would impede their learning of science.According to the educators, only 31% of the textbooks paid attention to the historical background on the development of the scientific concept of heat (Question 7).Reference to the history of science can enhance learners' understanding of the nature of science and can be used to address alternative conceptions (Lawson, 1999;McKinney & Michalovic, 2004).For example, the alternative conceptions that hot objects contain heat and that heat flows or moves already occurred in the historic caloric theory (Knight, 2004).
Although alternative conceptions were not addressed, eight of the educators indicated that their Grade 7 textbooks paid some attention to preconceptions of learners (Question 8).These books introduced new topics by revising learners' supposed existing scientific knowledge.The interviewed edu-cators admitted that they did not have the knowledge to detect and remedy alternative conceptions.Neither the learners' books nor the educators' guides made them aware of alternative conceptions, or taught for conceptual change.
Only nine of the educators (56%) were of the opinion that sufficient content knowledge regarding the topic of heat was presented in their textbooks (Question 9).The other seven educators (44%) wanted more content and more explanations of interesting phenomena.Most of the educators (63%) regarded the activities and assessment tasks to be in order (Questions 10 and 11), because they could be performed by most of their learners and related to their everyday experiences.The other third of the educators were of the opinion that they were not cognitively demanding.The activities were too few and too short, the discussion of the content lacked depth and the assessment tasks were too easy.These educators needed to use additional resources to keep their learners busy and challenged.Some of them assigned learners to find interesting information about phenomena and events that could be discussed or debated in class.Hubisz (2003) and Roseman et al. (2001) also reported a lack of depth in textbooks' presentation of content, activities, and assessment.
Half of the educators were satisfied with the progression in the presentation of the topic of heat in their textbook (Question 12).However, Edwards (2007) found that none of the Grade 7 books that she evaluated paid the necessary attention to all the concepts that should precede a study of the concept of heat.For example, only one of the textbooks explained and used the concept of temperature in terms of which the heat concept is scientifically defined.The other books simply referred to cold and hot objects.As Kulm et al. (1999) have found, concepts are often presented in the textbooks as loosestanding facts.

Conclusions
Since the quality of textbooks can promote or impair effective teaching and learning of a topic (Leite, 1999), an investigation was made into educators' ideas about the appropriateness of Grade 7 South African textbooks that they selected for use in their classrooms.Although they pointed out valid positive and negative features, major deficiencies occurred in their selection criteria and evaluation of the textbooks.
It followed from the interviews that the educators applied only a few of the selection criteria given in the literature.They mainly focused on alignment with the RNCS and everyday applications.These criteria are indeed very important, but are not the only selection criteria in the realm of the constructivist theory and for effective science education.For instance, the educators failed to apply the basic premise of the constructivist theory that science teaching and learning should start with what the learners already know and guide them to a higher level of knowledge that is broader, more general, and systematic.
The majority of the educators that participated in the study did not detect any errors or scientific inaccuracies in their textbooks.They assumed that the text was scientifically correct and taught the learners accordingly.Still, many of them regarded the textbooks as below standard.They criticised the level and cognitive demands of the activities and assessment tasks and were of the opinion that the content lacked depth.On the other hand, the educators were positive about the variety of activities and that the activities related to the learners' everyday experiences.However, to ensure scientific literacy it should be realised that instructional strategies (such as learner-centred activities) should be research-based and used to help learners learn the key science ideas (Roseman et al., 2001).
A high quality of science content knowledge and appropriate pedagogical knowledge are essential requirements for the ability to select, evaluate, and use science textbooks.The results of the study indicate the necessity for educator training programmes to spend more time on textbook selection and evaluation.Apart from preparing educators for this responsible task, an exercise in textbook selection is a contextual application of instructional theories.The educators should be guided to evaluate textbooks within the constructivist paradigm, instead of using a fragmented approach in which they say what they like, or dislike, without using a theoretical framework.

Recommendations
In the interviews educators requested more assistance from the Department of Education in the selection of textbooks.Selection criteria should be compiled by a competent team and workshops held timeously.The educators also expressed a need for co-operating with their colleagues from neighbouring schools.Educators from schools in the same geographical region could meet to choose textbooks.In this way they could learn from one another's experience and knowledge.Educators from the same geographical region could also form clusters where they exchange books and ideas with their colleagues.
Textbook authors must ensure that learner activities are challenging and contribute to the learning of the content, scientific skills, and processes.The textbooks should cover the core knowledge in appropriate depth and with scientific accuracy.The components of each unit (e.g.explanations, tasks, projects, and experiments) should form a coherent whole instead of a series of disconnected facts and activities, as is often found in modern textbooks (Kulm et al., 1999).Manuscripts should be peer-reviewed in a similar way as any other scientific high-standard publication.Textbooks are scientific publications that are used to train future scientists.A scientifically literate nation can only be ensured if a firm foundation is laid from an early age.
(e.g.energy and change, and matter and materials) separately•Not contain any non-relevant material, i.e.only include the work needed for the examination.

Figure 1 A
Figure 1 A combined list of selection criteria used by the Grade 7 educators that participated in the study.

Table 1
Science educators' responses to questions on their selection of textbooks

Table 2
Educators' responses to questions regarding the presentation of the topic of heat in their textbooks