Differences between the genders in ICT skills for Finnish upper comprehensive school students: Does gender matter?

This study examined the ICT skills of Finnish upper comprehensive school students. The data has been collected from 65 municipalities around Finland for 5455 ninth graders (mean age 15.24). ICT skills were measured using a digital, performance-based ICT skill test. The test was based on the revised Finnish national core curriculum for basic education. Based on the results, there was a small, but statistically, significant difference between the genders in the total scores on the ICT skill test. More consequential differences between the genders were found in the item level analysis. As explicit item level analysis indicated, boys tended to get better scores from more technical-oriented items, while girls got better scores from school work-oriented and social interaction-related items. The results emphasize that gender differences in ICT skills are more item specific than general. More importantly, the variation between individuals in ICT skill items was extensive and in all likelihood more influential than the gender difference as such.


Introduction
Ongoing digitalisation is changing every aspect of society.The growing amount of (digital) data, digital infrastructures, online artefacts and transaction systems, among numerous other things, are changing the way that people study, work, communicate, and live their everyday lives.Today's Western societies are described as digital societies, where new information and communication technologies shape the landscape of everyday living.This digital citizenship demands new kinds of literacies and digital competencies for one to be able to operate and participate in society (Gallardo-Echenique et al., 2015).
There is continuing debate around what the essential skills in digital societies are.The DIGCOMP framework project uses the concept of digital competence and categorises these crucial competences for information, communication, content-creation, safety and problem-solving (Ferrari, 2013).In turn Binkley et al. (2012) use the 21 st century skills concept to describe the ways of thinking, working, and living in a digitalised world.Fraillon et al. (2013) apply the construct of computer and information literacy that relates to an individual's ability to use technologies to investigate, create, and communicate and participate effectively at home or school and in the workplace or society.
The purpose of this study is to examine the differences in information and communication technology (ICT) skills between the genders in the context of the Finnish national core curriculum for basic education.The intent of the core curriculum is to provide a common ground for the local curricula and promote equality and equity in education.It contains the guidelines for providing education as well as the objectives and key content of instruction.In the core curriculum, ICT skills are one of the seven transversal competences which are integrated into all subjects.ICT skills are considered to be an essential aspect of civic competencies and are seen both as a goal and a tool for learning.In practice, the goal is to offer every student the following skills: Understanding of the basic operations and concepts of ICT, knowledge to use ICT in a responsible, safe, and ergonomic manner; the skills to use ICT as a tool for information management and creative work; and the abilities to use ICT in interaction and networking.Overall, the aim is to offer the experiences in using different applications to understand their meanings for everyday life, communication and public influence.(FNBE, 2016.)In this study, we are particularly interested in the learning goals for actual ICT skills, instead of studying ICT as a tool for learning.
The research questions are: 1) Are there differences between the genders in ICT skills between upper comprehensive school students?2) What kinds of differences exist between the genders?

Concepts
Various concepts are used to define the skills required for ICT use.Concurrently with digitalisation, terms like IT, ICT, and computer literacy have become dominant.(van Laar et al., 2017;Bawden, 2008.)In most cases, these concepts 3 consist of a domain part (computer, ICT, Internet, etc.) and a knowledge perspective (competence, literacy, skill, etc.) (Hatlevik et al., 2015).In their systematic literature review on the different perceptions of 21 st century skills, van Laar et al. (2017) made a distinction between 21 st century skills and digital skills.Based on their results, 21 st century skills are broader than digital skills and are not necessarily underpinned by ICT, while digital skills involve that connection.Based on the 75 reviewed articles, the authors created a framework of seven core skills and five contextual skills.The seven core skills are: Technical usage, information management, communication, collaboration, creativity, critical-thinking, and problem-solving.The five contextual skills are: Ethical awareness, cultural awareness, flexibility, self-direction, and lifelong learning.
In the definition offered by van Laar et al. (2017), technical usage is seen as the skill to use devices and applications.Information management skills are used to search and use information, including knowing what sources of information to use.Communication skills are used to transmit information to others.Collaboration skills are used to create social networks and be able to work as a team using ICT.Creativity skills are used to create new ideas or redefine old ones.Critical thinking skills are used to evaluate gathered information and communicate with others.Problem-solving skills are used both for finding information to solve problems and using ICT to solve those problems.
In spite of the diversity in the concepts, ICT skills or competences are considered essential skills for social interaction, civic participation, information retrieval and processing, academic performance, and professional success (see e.g.Hoffman & Schechter, 2016;Pagani et al., 2016;Zhong, 2011).Consequently, ICT skills should be considered necessary educational outcomes for every student in digitalised societies, and thus, a valid assessment of ICT skills is crucial (Aesaert & van Braak, 2015).In this study, the term ICT skills refers to the digital skills or competencies in line with the definition of the seven core skills offered by van Laar et al. (2017) and the learning goals for ICT skills found in the Finnish national core curriculum (FNBE, 2016).

Measuring ICT skills
In her review, Litt (2013) classified the ICT skill assessment methods as: 1) survey and self-reported measures; 2) performance/observation measures; and 3) combined and unique assessments.Surveys and self-reports seem to be the dominant methods for ICT skill assessment.In these assessments, participants respond to a question or a set of questions about their own competence level or evaluate their ability to perform specific tasks on the Internet (e.g.Heerwegh et al., 2016;Livingstone & Helsper, 2010).Qualitative studies prefer observationbased measures or interviews, which incorporate ethnographic practices.These types of studies focus on, for example, observing a person's actions during information search tasks (see e.g.Hargittai, 2002).Interviews in their turn typically utilise open-ended questions like 'What are you strong in?' (Litt, 2013).Some performance-based studies have conducted task-based assessments and virtual environments to mirror the actual environment where the ICT skills are being used (e.g.Claro et al., 2012;van Deursen & van Dijk, 2009).
In their review of assessment instruments Siddiq et al. (2016) concluded that the majority of the utilised assessment instruments are based on self-reporting.Most were online tests with multiple-choice questions.Some also included questions in a dynamic format that required participants to interact with the tasks.A few had a dynamic task design that required interaction with the test environment.Most of the tests were evaluated using quantitative methodology.Siddiq et al. (2016) summarises by saying that the majority of these tests assess information search, retrieval, or evaluation, and technical skills; however, aspects like problem-solving with ICT, digital communication, and online collaboration are not covered to satisfaction.
It is well known that measurements based on participants' own evaluations face validity problems, as people tend to overrate or underrate their own levels of competence (e.g.van Deursen et al., 2016;Aesaert & van Braak, 2015;Litt, 2013).There is, as Litt (2013) and van Deursen et al. (2016) emphasise, an urgent need for more accurate, nuanced, and reliable assessment instruments for greater generalisable and diverse samples that can capture the phenomenon in its entirety.

Gender and ICT Skills
In their longitudinal cross-sectional study of representative samples from the Dutch population on Internet skills, van Deursen and van Dijk (2015) noticed men scoring higher than women on all skill domains: Operational skills (e.g., saving files, downloading programs, using the refresh button); formal skills (e.g., being familiar with web site structure); information skills (e.g., finding information, using search booleans); and strategic skills (e.g., gaining financial benefits, and making decisions based on retrieved information).To the contrary, among students, previous research results on gender differences in ICT skills have been quite diverse.
In their study, Aesaert and von Braak (2015) noticed that among primary schoolers, girls perform better in tasks that relate to communication-oriented activities and deliver digital information with understandable content to the receiver.In the study by Ritzhaupt et al. (2013), middle school girls were found to be outperforming boys in all technology literacy domains.The five examined domains were: Technology operations and concepts (e.g., creating new files, locating and opening files, selecting the best device to complete a given task); constructing and demonstrating knowledge (e.g., selecting the correct printer, setting page margins within a word processing document, editing images); communication and collaboration (e.g., practical keyboard skills, using e-mail, and creating new slides within presentation software); independent learning (e.g., using print preview, deleting data in a spreadsheet, creating concept maps as a learning strategy); and digital citizenship (e.g., maintaining password security, identifying security risks, displaying an awareness of potentially inappropriate media use).Lau and Yuen (2014) found that female students in junior secondary school perceived their Internet literacy (e.g., searching information, using instant messaging, downloading files) and computer literacy (e.g., setting headers in word processing software, plotting a graph with a spreadsheet, editing a photo with image processing software) to be higher than that for male students.5 In an earlier study, using the previous version of the ICT skill test used in this study, among Finnish upper comprehensive and secondary school level students, Kaarakainen et al. (2017) found that male students outperformed female students on tasks that required advanced information technology skills (e.g., tasks related to software installation and initialisation, information networks, server environments, programming, and database operations), while on tasks related to basic digital skills (e.g., basic operations, information seeking, word or image processing, social networking) gender differences were not significant.In the case of technology self-efficacy, males/boys often reported higher ratings than females/girls (see e.g., Huffman et al., 2013).For example, in their study on German secondary schoolers, Christoph et al. (2015) found boys' self-ratings (e.g., "I am able to install new programs...") and theoretical computer skills (e.g., terminological knowledge of basic computerconcepts, such as 'JAVA', 'IP-address' or 'FTP') to be higher than those for girls, while no gender differences could be found in basic computer skills (e.g., tasks related to digital contexts within web-browsers, text processing, and e-mailing).
It can thus be summarised that the question of gender difference in ICT skills is complex and still largely unclear.A notable problem is the variation in the used concepts and the diverse concept operationalisations.As Ritzhaupt et al. (2013) speculated, even though girls are successful in their studies, boys may still be more proficient in other ICT-related tasks not covered in their tests.Thus, more research is needed to identify the actual dimensions of the potential gender differences within ICT skills.This current study particularly aims to clarify this issue.

Participants
The data was collected in Finland at the beginning of year 2017 (January-March) as a project financed by the Finnish Prime Minister's Office (funding for Government's analysis, assessment and research activities).The participants were from 149 upper comprehensive level schools (grade 9/9) in 65 municipalities around the country, chosen based on a geographically representative sample of Finnish municipalities, as formed by the Finnish Education Evaluation Centre.Altogether, 5455 9 th -graders ages 15 to 17 years were tested, and of that group 47% were boys and 53% were girls.The mean age for the participants was 15.24.

Measurement
ICT skills were measured using the online test developed in the Research Unit for the Sociology of Education (RUSE) at the University of Turku (Finland).The original ICT skill test was developed in 2013, and it was completely revised during year 2016 when the Finnish national core curriculum for basic education was renewed (2014) and implemented in schools in August 2016 (Kaarakainen et al., 2017;FNBE, 2016).The test is bilingual because both Finnish and Swedish are official languages in Finland.The software used for testing is a web application, written in PHP and Javascript and using the TinyMVC-and Bootstrap-frameworks.The application is supported by PostgreSQL database 6 software for all data storage needs, and the test content (tasks and the specific questionnaires in each study) is included in the system as easily changeable XML-files.
The test consists of 18 items divided into 6 modules based on item topics (Appendix 1).The goal was to form coherent modules for a comfortable user experience.Test items were implemented in such a way that the user interface and graphics were intended to simulate common ICT applications and hence mirror real-life settings.The tested competency areas (18 items) were chosen based on the renewed Finnish national core curriculum (FNBE, 2016).The last module (requirements for the ICT study programs) was broadly based on the curricula for the ICT field of Finnish vocational institutions and the universities of applied sciences.The participants could achieve 2 points for each item, which could result in a maximum total score of 36.
In the ICT skill test, each item consists of multiple subtasks (1-6) and/or chains of actions, in which every action (selection or operation) is linked to the previous one; together they form a coherent item.In the items, a combination of closeended questions (conventional multiple-choice, true-false multiple-choice, multiple true-false multiple-choice, and matching) (see Haladyna, 2011) and open-ended questions or questions requiring participants to interact with the test environment (input the right values or select and click the right function icons) were applied (see Siddiq et al., 2017).The majority of items can be seen as context-dependent item sets (see Haladyna, 2011), as they consist of a problem scenario that participants have to solve by choosing right actions from given options beside the progressive storyline.Some items can be seen as constructed-response questions that require the test-takers to construct or develop their own answers (Miller, 2015).In the ICT skill test, items are assessed automatically based on specified options and actions or simple text mining algorithms.

Analysis
The reliability (internal consistency) of the ICT skill test was estimated using Cronbach's alpha.Reliability refers to the extent to which a test is a consistent measure of a specific concept.It is known that scales with only two or three items tend to exhibit smaller alphas than do those with more items (Peterson, 1994).As the items of the ICT skill test can consist of only one multi-phase interactive task or just two or three subtasks, other item measures are utilised instead of alpha values for item level analysis.
The corrected item-total correlation is widely used to examine if any item fails to correlate with a total score.The cut-off value of .3means that items that have item-total correlation below the threshold value are likely to be extremely easy or difficult or ambiguous, or otherwise that this item is not measuring the same construct being measured by the other items.(Nunnally & Bernstein, 1994.)To analyse item difficulty, an item difficulty index was used instead of the proportion of right answers (traditional item difficulty), as that choice was better suited for complicated items (Tiruneh et al. 2017).Here the interest is not simply on how many test-takers get the item (completely) right.The formula 7 used to compute the item difficulty index (P) was: where ~fX is the total number of scores earned by all test-takers on an item, n is the number of test-takers, Xmin is the smallest item score possible, and Xmax is the highest item score possible.
Item discrimination is another basic measure of the validity of an item.It is defined as the ability of an item to discriminate (or differentiate) between high and low achievers.The formula used to compute the discrimination index (D) is: Discrimination index, D =   −   whereandare the difficulty indexes for the highest performing (U) and lowest performing (L) groups.The widely used threshold of 27% was used to divide these upper and lower groups.(Adams & Wiemanc, 2011.)In order to respond to the research questions (Are there differences between the genders…?And what kind of differences are these…?), a two-tailed independent samples t-test was used to analyse the statistical differences between the genders.The t-test was used to compare the sample means from two independent groups for an at least interval-scale data when the distribution was approximately normal (see e.g., Warner, 2013).Analysis of the data was first executed using total scores of the ICT skill test, and then on each item separately in order to examine the gender differences in overall performance and also for each item one by one.

Results
The Cronbach's alpha for the ICT skill test (all 18 items) was 0.86, which clearly exceeded the threshold of .7 (Nunnally & Bernstein, 1994).Results of item-level analysis are presented in Appendix 1.As that appendix indicates, the item difficulty indexes varied from .01 to .61.Low-item difficulty indexes for the last four items indicated that all programming-related items were extremely difficult for the majority of 9 th -graders.Even elementary programming, which actually does not require any programming skills, but only the ability to follow and give orders was based on the given instructions.The P-values for all other items lay within the range of ~.2 and ~.6, indicating that none of these items was either too difficult (<.2) or too easy (>.8) for the participants to complete.
The item-total correlation varied between .22 and .65.The items related to programming, which were among the most difficult items, also had the lowest item-total correlation values.These item discrimination indexes ranged between .34 and .92,and thus were all acceptable (>.3).These values indicated that all items on the ICT skill test were able to distinguish between participants who mastered the skills required for a particular item and those who did not yet 8 have those skills.(SD 5.92).The difference between the genders was small, but statistically significant (t-value 2.712, p-value .007).
The five top scoring items for the students were information seeking (M 1.21), word processing (M 1.02), software installation and updates (M .93),digital 9 communication (M .86),and video and audio processing (M .84).Conversely, the five lowest scoring items for the students were information networks (M .34),web programming (M .15),elementary programming (M .12),database operations (M .10)and programming (M .01).
There were only two items where the differences between the genders were not statistically significant, namely, image processing and software purchasing.In the case of software installation and updates, and information security, the difference between girls and boys was only marginally significant (p < .05).In the case of all the other items, the differences between the genders were highly significant (p < .001).
The girls significantly outperformed the boys in information seeking, word processing, digital communication, video and audio processing, information security, cloud services and publishing, social networking, presentations, and spreadsheets.The boys in their turn outperformed the girls in software installation and updating, basic operations, information networks, web programming, elementary programming, database operations, and programming.The widest gender differences in item mean scores were found in word processing (for the girls, with a difference of .34points), basic operations (for the boys, with a difference of .29 points), information networks (for the boys, with a difference of .23 points), and information seeking (for the girls, with a difference of .18points).

Discussion
In this study, the ICT skill test was introduced, and the results of the reliability and item analysis were presented to assess the quality of the test items and the test as a whole.The results indicated that the ICT skill test was a reliable instrument, and its items were mainly adequate in their difficulty level and had appropriate discrimination power.Both item difficulty indexes and item-total correlation for elementary programming, database operations, web-programming and programming items indicated the need to consider renovation of these items.It is clear that these four programming-related items are challenging for a comprehensive school final graders.However, since August 2016, programming has been integrated into the Finnish national core curriculum.In the upper grades (7-9/9) programming has been integrated as part of the objectives set for mathematics, while in the lower grades, the fundamentals of programming can be taught in all subjects (FNBE, 2016).Because of this these programming-related items should remain a part of the ICT skills test, as they provide the ability to track whether the desired programming skills will increase in the future or will not.
Based on our results, upper comprehensive level students performed best in schoolwork-oriented items, such as information seeking and word processing.
Conversely, the lowest scoring items for all participants were technical-oriented items, such as basic operations, information networks, different kinds of programming, and database operations.The discovery that students performed relatively well on items related to word processing, computer security, social networking and communication is encouraging in light of the desire to achieve the learning goals of transversal competences in the Finnish national core curriculum (FNBE, 2016).On the contrary, the noticed lack of basic operational skills and knowledge deficits in information networks, spreadsheets, and software purchasing indicated the need for formal education on the very elementary use and knowledge of computers and mobile devices.It should also be noted that relatively low scores on the basic operations items, which relate to the technical usage of computers, can be explained to some degree by the fact that these items stress the use of computers and keyboards when Finnish youth today are more experienced with Smartphones or other mobile devices (see e.g., Taipale, 2014).For example, the use of shortcut keys (a subtask in the basic operations item) requires computer specific knowledge.
The current study answered the question on whether there were differences between the genders in ICT skills.Based on the analysis in this study, there was only a small, but still significant, difference between the genders in the total scores of the ICT skill test.Particularly, this paper sought to identify the actual dimensions of potential gender differences within ICT skills.When examining item level differences, the results are in line with previous studies (e.g., Aesaert & von Braak, 2015;Lau and Yuen, 2014), as girls were found to outperform boys, particularly on items that related to using learning-related software and tools like word processing, spreadsheets, presentation software, image processing, and items related to communication, social networking, and security issues.In contrast, in this study, boys were found to perform significantly better compared to girls on items that required more technical knowledge, like basic operations, information networks, programming, and database operations.This result is also in line with previous studies (Kaarakainen et al., 2017;Christoph et al., 2015).Above all, these discoveries emphasise that gender differences in ICT skills are particularly item specific.
Students' success was found to be weakest on the last module (requirements of ICT study programs) of the ICT skills test.This is also the module where the boys outperformed the girls by the largest margin.This module contained items related to database operations, web programming, and computer programming, which are common areas of computer science studies in secondary and tertiary level education.It is well known that in the field of computing, a lack of diversity has existed for several decades, and female participation in computing studies has remained low (McGill et al., 2016).However, after more profound consideration, this is the module that distinctly revealed that differences between individuals outweigh the difference between genders, as even among the boys, there was a huge variation in individual performance.It is worth noticing that the standard deviations for the scores on the items in the last module were at least twice as big as the mean.In fact, there were only 19 boys and 3 girls among all the participants, who got at least half of the available points in that module.Therefore there was only a very small group of students who mastered the most technical items in our ICT skills test, and this success related more to individuals than gender as such.
Individual technology usage habits and experiences played a notable role in the above-mentioned phenomenon.As Aesaert and von Braak (2015) assumed, the reason behind females' success in communication-oriented activities and information delivering can be found in their ICT use and experiences.Similarly, Lau and Yuen (2014) presumed that because female students tend to engage in more learning-and social networking-related activities at home, they achieve skills like information searching, instant messaging, word processing, image editing, etc., which were classified in their study as Internet and computer literacy.When it comes to more technical skills, Huffman et al. (2013) found that gender roles in particular (not just biological sex) play a large role in technology self-efficacy; in particular masculinity is a strong predictor of technology self-efficacy.
As van Laar et al. (2017) suggest, ICT skills are essential for performing tasks that are necessary in a broad range of occupations in a digital society.Typically ICT skills delivered in an educational context will have relevance also in 21 st century digital skills for work.However, as van Laar et al., (2017) summarised, the changing labour market and job demands in a knowledge society pose serious challenges to educational systems, when they are asked to prepare students for jobs that may not yet even exist.In the Finnish core curriculum for basic education these ICT skills are decentralised under traditional school subjects (FNBE, 2016).This creates a situation where the responsibility for teaching ICT skills is fragmented between several teachers, at worst without adequately expressed and internalised common goals.This scenario increases the risk that not all students are getting sufficient digital preparedness for the future in their basic level education.All things considered, however, the notable role of individuals' (leisure time) experiences and usage habits in ICT skills and digital preparedness, when combined with decentralised teaching of those skills, may compromise the ideal of equality between the genders and principally between individual students.This raises important questions to study in future research.

Table 1 .
The ICT skill test scores by items and gender differences for the students.Table1presents the ICT skills test scores by items.The table is presented in descending order by the achieved average item scores of the students.The mean score for all test items for all participants was 10.45 (standard deviation 5.32), for girls 10.64 (SD 4.74) and for boys 10.24