Assessing students’ information literacy skills in two secondary schools in Singapore

Appropriately measuring information literacy (IL) skills is essential to understand their educational impact as well as exploring pedagogies to improve teaching of these skills. This paper reports findings of a baseline study to understand the IL level of secondary school students in Singapore. A comprehensive instrument encompassing a new dimension of ethical usage of information as well as major IL principles and guidelines was developed for data collection. A total of 298 responses were gathered through the online survey. The findings indicate that overall there was a scope to improve students’ IL skills. That stated, higher order IL skills (such as information use, synthesis, and evaluation) demand more attention, as comparatively lower levels of proficiency were observed in this study.


Introduction
Since Zurkowski coined the phrase information literacy (IL) in 1974 (Laxman 2010;Pinto et al. 2010)., IL and IL standards have been defined by many researchers in different ways which collectively covered skills and knowledge that enable individuals to recognise their information needs, to locate, evaluate and use needed information while filtering out the unwanted information, and to some extent, to evaluate one's own information seeking process and the consequent final information products (Wilson 1981;Kuhlthau 1993;Eisenberg and Berkowitz 1996;Eisenberg 2008;ACRL 2000). The importance of adequate IL skills has been linked to academic performance as well as in the work place (Majid et al. 2011). Appropriate measurement of IL is essential to understand its educational impact as well as to explore suitable pedagogies for imparting these skills. This paper reports on a baseline study to understand the IL level of secondary school students in Singapore. As part of the study, we developed a comprehensive instrument that, in addition to core IL competencies, covered a new dimension of ethical use of information.

Review of the literature
This section presents a review of IL-related literature in two parts: the definition and scope of IL, and the assessment of IL skills.

The definition and scope of IL
The term information literacy was used for the first time in 1974 by Paul Zurkowski (Laxman 2010;Pinto et al. 2010). A person who is information literate requires a very specific set of skills that include the location and use of information for making decisions and solving problems (Hubbard 1987). In the earlier use of the term IL, it was also linked to democracy as necessary to preserve democratic institutions and the liberty to make decisions freely (Owens 1976).
In the 1980s, the role the technology emerged as a key component of IL (Umbach, 1998). Learning basic computer skills became a necessity, which included managing word processors, spreadsheets, image editors, databases, and other applications. These skills turn out to be increasingly important in the workplace. In the context of IL, technology is an enabler, and there have been two tendencies: technology as instructional tool and technology as instructional object (Eisenberg and Johnson 1996;Eisenkraft 2010). Related to computing is the concept of network literacy, which focuses on finding, accessing and using information in a networked environment (Savolainen 2002).
The meaning of the concept has been modified and adapted over the time. There is a movement to expand literacy to include media literacy. A media user has to be capable of having a critical understanding of the scope, coverage and bias or slant of various media channels (Aufderheide 1992). Supporters of this movement recognise the effects of the media in our everyday life, and the need to include media as another component of literacy (Brill et al. 2007;Buckingham 2007;Stein and Prewett 2009). In addition, more specialised information literacies such as financial literacy, healthcare literacy and social literacy are gaining popularity.
In all situations, there is a need for education that allows our children to think for themselves and have the capacity for life-long learning (Massis 2011;Stripling 2010;Mancall et al. 1986). Also, students should be able to increase their ability to solve problems (Heider 2009), collaborate (Engstrom and Jewett 2005), and be independent learners. Such conditions have forced educators to look for new ways to educate such as resource-based learning (Tergan et al. 2006), authentic learning (Fitzsimmons and Lanphar 2011), and problem-based learning (Barrett 2010).
Traditionally, IL has been studied as a process (Eisenberg and Lowe 1997), which should be integrated into the curriculum and reinforced by other educational activities (Derakhshan and Singh 2011;Eisenberg and Johnson 1996;Kong 2008;Kuhlthau 1987), and as a set of vital skills for future success (Aufderheide 1992;Farmer 2010). Five major models have been developed for IL processes: Kuhlthau's information-seeking behaviour model (Eisenberg and Brown 1992); Eisenberg/Berkowitz's information problem-solving model (Eisenberg and Berkowitz 1992); Irving's information skills model (Irving 1985); Pitts/Stripling's research process (Stripling and Pitts 1988); and New South Wales' information process (Dawson and Kallengerger 2007).

Assessment of IL skills
The overall aims for assessing IL are to increase student learning, to provide accountability, and/or to strengthen instructional programmes (Oakleaf & Kaske 2009). With the added emphasis being placed on IL after the onset of the internet age, numerous IL assessments in various formats have emerged across the world. McCulley (2009) identified 3 major assessment approaches: (i) knowledge tests and surveys; (ii) performance assessments; and (iii) informal assessments.
Knowledge tests and surveys that measure one's IL ability typically include a list of questions for students to answer, more often than not in the form of multiple choice questions, with a standard list of correct answers for markers to check against. Comparatively, a fixed-choice test is easier to implement and widely adopted for assessing IL skills, as it is less resource-intensive, and the results can be used for comparison at different levels from individual to institutional level (Oakleaf 2008). Examples include standardised IL tests such as: Standardized Assessment of Information Literacy Skills (SAILS); Tool for Real-time Assessment of Information Literacy (TRAILS); and Research Readiness Self-Assessment (RRSA) assessing discrete IL skills (Abilock 2007;Schloman & Gedeon 2007;Ivanitskaya et al. 2008). These standardised tests allow efficient testing of large number of students and they can be easily adapted for use at other institutions. Nonetheless, fixed-choice tests are not the most comprehensive, valid and reflective of one's IL abilities as they examine individuals' recognition skills (Oakleaf 2008). Furthermore, tests put students in hypothetical situations that students may have been taught to answer and hence do not always examine their actual IL practices which may be a better reflection of their skills.
Performance assessment tests higher level skills but it is more resource intensive (Oakleaf 2008). This approach increases the relevance of IL instruction and uses authentic assessment. As an example of performance assessment techniques, rubrics involves using a set of descriptive scoring schemes to evaluate students' work, which allows measurement of higherorder thinking and enables students to understand the expectation of their instructors (Helvoort 2010;Oakleaf 2008). In the case of Dutch higher education (HE) institutes, IL assessments in the form of analytical rubrics were used when students were tasked with creating their own country profiles of a foreign country they were supposed to work or intern at (Helvoort 2010). Another example involved first-year university students where they had to take a compulsory module that served to hone their skills in finding, utilising and analysing information as they completed a bibliographic analysis of a research topic (Knight 2006). Rubrics are suitable for IL instruction that is integrated into other subject curriculum, as its flexibility allows easily transfer to any disciplines and learning environments (McCulley 2009). Nevertheless, using rubrics can result in more time and expertise needed on the part of the assessors as they need to have some basic understanding of the criteria and the subject matter under assessment.
Portfolios are a more exhaustive form of performance assessment, as a portfolio is a consolidation of a student's effort in various aspects. Portfolios contain work done over various time periods, and students' progress can be seen (Sonley et al. 2007). However, as an assessment tool consisting of multiple forms of assessment, it is likely to be more cumbersome and intricate to collect, create and assess than even rubrics. In the case of several modules conducted in the University of Teeside, training portfolios consisting of students' bibliographic records of books, websites, and journal articles were used to assess their level of IL (Sonley et al. 2007). Concordia College included an IL survey created by UCLA Libraries, a bibliographic analysis and a written paper (Flaspohler 2003). The University of Alabama included tests, quizzes, presentations and a project featuring a bibliography of ten items in a compulsory library instruction course on IL (Mackey & Jacobson 2007). The Washington State University Honors College course included case studies, citation exercises and examinations (Johnson et al. 2011).
Informal assessments are the easiest to incorporate into IL training sessions to assess various domains such as cognitive, affective and behavioural (Radcliff et al. 2007). Classroom assessment techniques, as an example of informal assessment, provide quick snapshots of whether students adequately understood the IL concepts just being presented, which could be used to make clarifications at the time or to make improvements for the future (McCulley 2009). However, informal assessments are by their very nature informal, which limits their value.

Statement of the problem and research objectives
What unites much of this literature is a geographical focus on North America and, to a lesser extent, Western Europe. Other areas of the world have received limited attention. Our aim is to extend the focus beyond the North American and Western Europe context, and rectify this imbalance by developing a comprehensive overview of the IL context of Singaporea country in Southeast Asia. Despite years of IL education in Singapore, no large-scale assessment has been conducted to find out students' levels of IL skills. So far, only a few IL training programmes in Singapore secondary schools have been reported. For example, in 1998, the Ministry of Education (MOE) collaborated with the then National Library Board Institute (NLBI) to conduct training programmes for the heads of department of the information technology and media resource library (HOD IT/MRL) from the high schools. A year later, the MOE and the National Library Board (NLB) organised the Library@School Conference in September 1999 with the specific objective of helping schools enhance their libraries and extend their services to both students and teachers in the schools (MOE 1999). This paper is a part of that larger goal to carry out a national IL skills study in Singapore.
In addition, research on collaborative information-seeking behaviour and ethical use of information also contribute to the understanding and new dimensions of IL, in which social context factors are included but not limited to the culture, distribution of authority, social norms, habits, practices, expectations, rewards, and preferences held by the group that an individual belong to (Robbins & Judge 2009;Cho & Lee 2008). There is a need to assess students' IL skills with coverage of these new dimensions. Therefore, this study aims to assess Singapore secondary school students' IL skills as well as their perceptions towards collaborative information seeking and ethical use of information. Specifically, this study aims to address the following research questions: 1) What is the level of IL skills of Singapore secondary students? 2) What is the students' level of knowledge about information ethics and collaborative information seeking? 3) What is the current usage of various libraries (e.g. school library, public library) and human information sources by those students?

Background of the study
In the Singapore education system, students in Grade 3 (age 8-9 years old) undergo initial streaming according to their academic abilities in English, mathematics, science and their mother tongue language. Students who perform well in these subjects can take their mother tongue language at the advanced level, while students who do not do well take these four subjects at the standard level. Students who do not perform well in certain subjects will take those subjects at the foundational level. In addition, at the end of Grade 6 (11-12 years old), students will sit for the national level high-stakes examination known as the Primary School Leaving Examination (PSLE) where their scores in this examination alone will determine which secondary school and academic stream they will be posted to. Students who are in the top 10% of the national cohort are posted to the special stream in top-performing schools, while those in the next 50% are posted to the express stream in both top-performing and neighbourhood schools. The subsequent 25% are posted to the normal academic stream and the bottom 15% are posted to the normal technical stream which focus more on technical and work skills. In addition to the PSLE, students also take two more high-stakes examinations -the Cambridge General Certificate of Education (GCE) Ordinary Level (O Level) examination at the end of Grade 10, and the GCE Advanced Level (A Level) examination at the end of Grade 12.
In view of the education system, our IL study would be useful to inform the competencies of the students across streams and the order of IL skills that have been acquired by them.

Methodology
This section presents the development of survey questionnaire, sample selection and data collection process.

Development of the survey questionnaire
An online survey questionnaire was used to collect data for this study. The review of various kinds of IL assessment tools reveals that each of them has their own strengths and weaknesses. This explorative study adopted the method of knowledge test and survey, as it is an economical way to cover larger sample size. Performance assessments can generate more reliable results but it has much higher resource requirements.
The questionnaire was developed by a team comprising information studies and education faculty members of Nanyang Technological University. Survey instruments used by previous studies were consulted. However, it was noted that most of them were designed with context that were familiar to students receiving education in North America. Thus, the development of our questionnaire was cautiously taken to have contents that were more relevant to students in Singapore. For instance, one multiple-choice question asked students to identify information resources that were least likely to be biased with answers including sites and organisations that were familiar to Singaporeans. The language in the survey was also carefully designed to be easy to understand so that the reading skill of individual students would not greatly affect the survey results. To ensure this, we pilot-tested the questionnaire with four secondary school students to clear up most of the language issues. We were also able to identify the degree of difficulty in each question and answer. Consequently it was decided to give different scores to each correct answer. In other words, each question was weighted according to its perceived degree of difficulty for students. For some questions, there was only one correct answer, and for others, the correct answer was allocated a higher mark while an incorrect (or less correct) answer was allocated lower marks.
The questionnaire was divided into three sections. The first section collected demographic information about the participants such as the school, gender, age, level of study and stream of study. The second section tested the IL skills, which included the classic Big 6 skills as well as ethics and collaborative information behaviour. The 6+3 model (Mokhtar et al. 2010) was chosen as theoretical reference to develop this section as it covered both soft and hard skills of IL, and was designed based on the local context. The third section sought information about participants' overall library use, use of human information sources, and access to technology. The scoring scheme mentioned above was applied to the second section of the questionnaire, comprising 41 questions (Table 1).

Sample selection and data collection process
As a preliminary study of a national survey, this study employed the convenience sampling method to collect data. The authors contacted and obtained approval from the principals of two secondary schools in Singapore. All students from Secondary 1 (13 years old) to Secondary 3 (15 years old) levels were recruited as sample for this study, which are equivalent to the grade 7 to grade 9 in the K-12 education system. A total of 223 students from School 1 and 730 students from School 2 where invited to participate in the study.
In March 2010, all students of the two schools were given the address of the online survey website through their teachers. By the end of April, 298 students in total completed the survey, yielding an overall response rate of 30.94%.

Data analysis results
The Statistical Package for Social Sciences (SPSS) was used for data analysis. This section presents the data analysis results. Among the 298 responding students, 113 were from School 1 and 185 were from School 2. The composition of students is shown in Tables 2a-d.

Access to technology
From the data collected, the average number of computers per household was 2.17, with minimum of zero and maximum of 10 (SD=1.55). Some 86.24% of the participants had Internet access at home.

Library visits and usage
A scale of 0 to 5 was used to record the frequency of visits to libraries and use of library resources in the past 12 months (0= Never; 1=Rarely; 2=2-3 times a month; 3=Weekly; 4=2-3 times a week; 5= Daily). Table 3 shows the average scores which indicate that school libraries were most visited and used. However, the majority of the participants reported that they rarely visit or use the library resources ( Figure 1).

Consulting others on IL tasks
While asked about the people consulted across the different stages of the information process, most students reported that friends, peers/classmates, teachers, and family are the frequently consulted human sources (Table 4). Friends were reported by the highest percentage of students for all tasks except information use, for which peers/classmates had the highest percentage.

Overall IL skills scores
A set of 35 questions were used to investigate the classic big 6 IL skills, and 6 additional questions were used to understand ethics related knowledge, collaborative information seeking behaviour and overall IL process. As described in Table 1, the scores of all items were not the same; therefore, the raw scores were converted into percentages in order to make a more meaningful comparison among the items. It was found that, of the six skills, the best scored skill was task definition (mean score: 45.39%), and the least scored skill is synthesis (35.87%). It roughly appears that the students generally have higher level of IL skills that are meant to be applied to the earlier stages of the information behaviour. Among the individual items, students were weak at the following areas as shown in Table 5 (with scores ≤30%). Students were also found to have a lack of knowledge about information ethics, as the average score of the four related questions was 30.19%. It is also worth noting that the average score for the question related to collaborative information seeking is even lower at 17.65%. Students tended to have insufficient knowledge in this dimension also.

Comparison of the two streams
As shown in Figure 2, among all the categorical scores, evaluation and collaborative information seeking (CIB) percentage scores of the two streams were significantly different, with t(287) = 1.89, p = 0.04, and 1.04, p = 0.04 respectively. Overall, the express stream did better than the normal academic stream. This result suggested students who have better academic performance also have better IL skills, which is not too surprising. Figure 2: Comparison of the two streams

Correlations of demographic variables and mean percentage scores
There were some slight correlations found among some demographic variables and mean percentage scores. Those with statistical significance at the 0.01 level (2-tailed) and 0.05 level (2-tailed) are marked with ** and * respectively in Table 6. Again, overall, the streams of study made the difference in most IL skills, especially the skills that are considered being applied at the earlier stages of information behaviour (i.e. task definition, information seeking, and location and access). However, it was observed that the respondents' father's highest educational qualification had a negative relation to the ethics score, and having internet access at home also contributed towards lower scores in information evaluation skill. While it is difficult to explain the former observation, the latter may suggest that students were less careful in judging the quality of information when considering online resources.

Correlations among mean percentage scores of each skill category
Statistical testing was also performed to investigate the possible relationship between the mean percentage scores of each skill category. Those with statistical significance at the 0.01 level (2tailed) and 0.05 level (2-tailed) are marked with ** and * respectively in Table 7. It shows that the skills at the earlier stages including task definition, information seeking strategies and location & access, affect some other IL skills, while the skills in the later stages including information synthesis, evaluation and use are more distinct. For instance, task definition will direct the sequence of information seeking and location & access. Information seeking strategies will determine the quality of resources retrieved and hence have an impact on the steps of information synthesis and evaluation. To summarise, the majority of participants reported that they rarely visit libraries or use library resources. According to the analysis results, out of the 6 IL skills, the best scored skill was task definition, and the least scored skill was synthesis. Regarding the human information sources consulted for conducting various information tasks, friends were selected by the highest percentage of students for all tasks except information use. It is not surprising to find that students from the express stream did better than the normal academic stream, which may suggest that students who have better academic performance also have better IL skills. The majority of students had internet access at home. However, the correlation analysis suggested that students may become less careful in judging the quality of information when considering online resources.

Discussion
The study findings reveal a number of issues that need to be addressed. To begin with, we note that students tend not to use the library, even their own school libraries. Earlier studies on information behaviour suggested that human information sources, rather than physical information sources such as print materials found in libraries, often are the ones that are most heavily used, even by professionals, such as lawyers and engineers (Leckie et al. 1996;Wilkinson 2001;Fidel and Green 2004). This same observation applies to the secondary school students who participated in the survey. The infrequent use of libraries by these students had deprived them of the opportunities to learn IL skills from the librarians.
The study also found that lower-order IL skills (e.g. task definition, information seeking strategy, and location and access) were possessed by more students than higher-level skills (e.g. information use, synthesis, and evaluation). This possibly suggests that there is sufficient awareness and effort in teaching these skills in the Singapore school curriculum. Conversely, it also suggests that more effort must be put into teaching higher-order IL skills through avenues such as project work. These are skills that take time and a lot of practice to develop. However, the current structure of the Singapore education system, which focuses more on academic achievements and high-stakes assessment, may create a challenge in incorporating these skills fully into the existing school curriculum. To elaborate, these high-stakes examinations (PSLE, GCE O level and A level) require some drill-and-practice and studying to the test, which tend to compromise the teaching the complete set of IL skills. As a result, schools have paid more attention to teaching lower-order IL skills.
Some Western studies also revealed that secondary (high) school students lacked information synthesis and evaluation skills. Many students simply sought a direct answer without critical judgment of its quality. For example, Adams (1999) found that high school students working in science courses had difficulty evaluating the veracity and objectivity of information. Similarly, Brem, Russell, and Weems (2001) discovered that high school students were unable to determine credibility and accuracy of web sites. According to Brill, Falk, and Yarden (2004), high-school biology students read scientific documents superficially, without critical thinking about their content. Heinström's (2006) research showed that many students tend to judge the relevance of information on the basis of convenient access and superficial criteria.
One of the most advocated methods to teach higher-order IL skills is through collaboration between teachers and school librarians. Working hand-in-hand with teachers, librarians can ensure the library resources are well-integrated with the needs of school curriculum system and provide good reference services. Laverty (2007) pointed out that librarians can support adolescent IL by creating authentic research topics, putting Googling in the context of a research strategy, teaching best methods for web searching, and discouraging plagiarism by personalising assignments that track the information-gathering process. Through a study in community college libraries, Small, Zakaria and El-Figuigui (2004) demonstrated the positive effect of teacher-librarian collaboration on students' motivation of learning IL. Another possibility is to train more teacher librarians. Excellent teacher librarians not only possess a detailed knowledge of educational pedagogy and a comprehensive understanding of school curriculum, but are also well-informed about IL theory and practice, which enable them to provide better IL instruction (ALIA 2012).
In the local context, Foo (1999) emphasised the critical and crucial role the school library played in "supporting the new age education", which is even more important in the new millennium when the nation propels itself towards a "truly information society". However, school libraries or school media resource centers have long been underused and detached from students' learning environment (Mokhtar and Majid 2005). In addition, librarians traditionally have little contact with teachers and their classroom objectives, except to locate books and other reference materials, or babysit students who use the library (Manzo 1997). Moreover, it was found that Singapore school libraries did not have full-time professionally-trained librarians. Instead, the heads of department for IT were in charge of the school library, and more than three-fifths of them were without any professional library training (Majid et al. 2002).
Perhaps a potentially more serious problem is the differences found between the two academic streams. If IL skills are so important in today's society, and that IL is necessary to preserve democratic institutions (Owen 1976), then we are witnessing the creation of an IL underclass, perhaps incapable of adequately participating in future society.

Conclusion
This paper has described an instrument for evaluating IL skills as well as assessing IL competency levels of secondary students in Singapore. The findings indicate that overall there is still scope to improve students' IL skills. That stated, higher order IL skills (such as information use, synthesis, and evaluation) demand more attention, as comparatively lower levels of proficiency were observed in this study. Singapore secondary schools should enhance the collaboration between teachers and school librarians, and promote the training of teacher librarian professionals, which will enable the IL components to be better integrated into the school curriculum with supplementary materials and services provided by school libraries.
In order to prepare for a more comprehensive national level study, this instrument has been subsequently refined based on focus group discussion with several participating students, a review of the analysed results, as well as expert opinion sought from other IL researchers. The focus group discussion provided feedback about the quantity and understandability of questions. The analysis results provided us a better insight into the difficulty levels of questions.
The instrument was also validated through assessment by three IL experts from Hong Kong, Kuwait and Thailand. Some long and complicated questions were deleted, and the language of some questions was simplified in order to minimise the effect of students' reading and comprehension abilities. Moreover, the points assigned to some questions had also been revised according to their difficulty levels.
Future studies can systematically investigate the various ways of incorporating higher order IL skills into the school curriculum. Beyond this, it would be useful to investigate how IL skills can be applied by individuals for the betterment of the society.