The effects of digital learning material on students’ mathematics learning in vocational education

Abstract This study investigates the effects of Digital Learning Material (DLM) including instructional clips, online guidance, structuring of content, and a collaboration tool on students’ mathematics learning in Dutch vocational education. A pretest–posttest design was used. Apprenticeship students were asked to complete assignments and to discuss them with their peers and the online teacher. The results showed that DLM can enhance students’ mathematics learning in vocational education. The learning enhancement was mostly due to the use of instructional clips and structuring of the content of the mathematics tasks. Elaborations of these results, implications, limitations and recommendations for further research are provided.


Introduction
Dutch vocational education prepares students, from 16 years of age to over 40, for an occupation and provides them with requisite education-based vocational qualifications. These qualifications are developed by education and industry and meet job-entry requirements that pertain to each sector: ABOUT THE AUTHOR Diana P Zwart is a senior lecturer at Windesheim University of Applied Sciences and a PhD student at Utrecht University in the Netherlands. Her work includes the design of Digital Learning Material to support teachers and students with mathematics in online and blended learning environments. In November 2015 she gained a NWO Grant for her PhD proposal on "Training nursing students' mathematics in a virtual learning environment".

PUBLIC INTEREST STATEMENT
Digital Learning Material (DLM) can be a valuable dimension to online learning and help students to practice without limitations of time, space and place. However, the use of DLM by itself does not guarantee benefits for learning. Scientific literature suggests that it should be accompanied with appropriate instructional approaches that facilitate students' selection, organization and integration of new information and knowledge. This article describes the effect of such approaches on learning mathematics in vocational education. We used instructional clips, online guidance, collaboration tools and restructured the content of mathematics because of students' prior knowledge. It was found that involvement of students in the online learning tasks showed greater learning gains. Yet, it isn't just about DLM. As in traditional face-to-face forms of learning, we envisaged a central role for the online teacher in giving impetus to online learning.
Health and Welfare (H&W), Economics and Services (E&S) and Technology (T) (Ministry of Education, Culture & Science, 2015). This is how the Dutch Government ascertains that students learn the knowledge required to perform productively at their future job. This knowledge comprises an occupation's domain-specific conceptual, procedural and dispositional knowledge (Billett, 2013). Since the national mathematics-and language tests are also part of the curriculum as a standard procedure for assessment in vocational education, there is also a lot of attention focused on students' mathematics and language knowledge. Within that, mathematics is generally the biggest headache for students.
The national mathematics test was meant to support student's vocational education learning of mathematics in the context of their profession. Results on national mathematics test still raise cause for concern (see Examenblad, n.d.;Ministry of Education, Culture & Science, 2015). The disappointing results come as no surprise: vocational education supports learning through work practice (Billett, 2013) and mathematics in non-vocational education settings is now taught as a collection of separate elements instead of teaching them in an integrated fashion (Van Merriënboer & Kirschner, 2007). As a result, students lose track of the interconnections between these elements. Students therefore produce flawed algorithms, which might explain their disappointing examination results.
Students' characteristics in vocational education are quite different from each other in terms of their age level, prior education and differences in general cognitive abilities, specifically, Working Memory (WM) (Alloway, Bibile, & Lau, 2013). And since WM is a key predictor for mathematical achievement (Wei, Yuan, Chen, & Zhou, 2012), it is difficult for a vocational mathematics teacher to support students in a "one size fits all" manner. An alternative to facilitating mathematics teaching and learning in vocational education is to use instructional design with Digital Learning Material (DLM).
DLM offer a wide range of educational opportunities that could not be achieved in traditional faceto-face forms of learning and instruction (Kalyuga & Liu, 2015). DLM enhance efficiency of teaching and offer more diversified learning experiences without limitations of time, space and place (Lee & Hung, 2015;Noroozi, Busstra et al., 2012). Despite the advantages of DLM, the use of technology by itself may not guarantee sufficient benefits for learning to warrant investigation into the development of such applications: mixed findings have been produced by empirical research on the use of DLM (see Kalyuga & Liu, 2015;Noroozi, Busstra et al., 2012;Payne et al., 2009). Therefore, scientific literature suggests that DLM should be accompanied with appropriate instructional approaches (see Noroozi, Busstra et al., 2012). DLM such as instructional clips, online guidance, content structuring and collaboration tools in technology-based environments can be used to create learning environments in which students can interact meaningfully with the learning materials and their peers . It can also facilitate the students' selection, organization and integration of new information and knowledge. Using such instructional approaches for learning mathematical concepts and solving mathematical problems for students in vocational education is rather sparse. Furthermore, it is not clear how DLM can foster the integration and acquisition of domain-specific knowledge to students in vocational education. Therefore, this study examined the impact of an instructional design approach with DLM including instructional clips, online guidance, content structuring and collaboration tools to facilitate mathematics learning of students in vocational education. And since WM is a key predictor for mathematical achievement (Wei et al., 2012), this study is also concerned with the extent to which WM influences students' mathematics learning outcomes.

Mathematics in Dutch vocational education
In vocational education, mathematics starts with the instruction of the underlying unifying principles of domain-specific knowledge within the domain of "numbers" (e.g. notation, naming and meaning of both positive and negative (large) whole numbers, decimals and fractions, as well as connecting, organizing and calculating numbers, with or without calculator). After eight weeks, students are tested and regardless the scores of the test, students start with the next domain "proportions", in which, for example, fractions and percentages take an important role. Most students encounter problems with fractions and calculating the amount of 100%. They also find it difficult to count backwards from a given percentage of 100%. The instruction of this new domain-specific knowledge is highly significant to encode and retrieve information from the understanding of the part-whole relation, measurement interpretation of fractions, and fractional quantities (CVE, 2015). After the domain "proportions" students start with geometry, again regardless their scores on previous domains. Geometry has a stronger relation with performance on mathematics tasks with a strong visuospatial component (Peng, Namkung, Barnes, & Sun, 2016). In the domain "geometry", students have difficulty with converting and they often have insufficient understanding of compound (ratio) sizes (CVE, 2015). The importance of the underlying (prior) domain-specific knowledge and skills should not be underestimated. Since prior achievement tends to go along with the level of prior knowledge a student has accumulated (Kollar et al., 2014), this is particularly problematical when students do not understand the underlying (prior) knowledge and skills thoroughly. When students are knowledgeable in this particular domain, they can encode and retrieve information specific to it more efficiently than they can encode and retrieve information from a domain in which they are less knowledgeable (Peng et al., 2016). It should thus be recognized that there are students who need more time and more instruction than the standardized eight weeks to understand the concepts of the domains, because when this particular domain is less knowledgeable to the student, it is difficult to make connections with the new domain-specific knowledge. In addition, it should also be mentioned that if a student is already able to perform a certain task, the information provided becomes redundant (Sweller, 2010) and extra instruction is not needed.
The individual differences in mathematics success might be a result of the educational design, but the success can also be the result of students' different WM capacities, which could determine how vocational education schools should shape their mathematics education. Investigating the strength of the relation between WM and different types of mathematics performance among students may be important for instructional design (Peng et al., 2016). Because WM can be seen as an important facilitating or inhibiting factor for mathematical achievement, it will be discussed more thoroughly.

Working memory and mathematics
WM is an important cognitive skill; capacity differences between the highest and lowest scoring individuals correspond to five years of normal development (Alloway et al., 2013). It is considered important for mathematical performance (Alloway et al., 2013;Wei et al., 2012). Burgess and Hitch (2005) point out that a higher WM capacity can help students to better encode items. WM is a factor that has proven critical for general individual differences between the efficiency of filtering irrelevant information (Jost, Bryck, Vogel, & Mayr, 2010), and it integrates domain-specific skills, knowledge and procedures to meet the particular demands of learning tasks within a particular domain (Peng et al., 2016). Too many new elements can overburden WM, causing cognitive overload (Kalyuga, Chandler, Tuovinen, & Sweller, 2001). In particular, WM load caused by elements of the learning environment that are necessary neither to solve the task nor for schema acquisition is called extraneous load (Sweller, 2010;Van Merriënboer & Kirschner, 2007). This is another reason why WM limitations must be borne in mind in the instructional design of DLM, so that these materials effectively support students to be able to succeed.

Digital learning material (DLM)
The instruction in DLM environments can be designed to help students identify useful information, understand how materials fit together and see how materials relate to prior knowledge (Mayer, 2001). In DLM environments, instruction shifts from a traditional teacher-or system-controlled environment, to on-demand education (Van Merriënboer & Kirschner, 2007). The instruction by the online teacher should encourage students to be as cognitively active as possible, and discuss their ideas and conceptions from different perspectives (Noroozi, Weinberger, Biemans, Mulder, & Chizari, 2012), as in face-to face settings. The instruction in DLM can also be delivered by the use of instructional clips, which present the instructional message to the learner auditorily and visually. Each clip involves a concise statement of a crucial step in the mathematical process. The sounds of the clips verbalize and reason the connections between the different elements of the domain-specific knowledge. For example, in the domain of "proportions", the clips repeatedly show the relationships between ratio, percentages, and fractures before the instruction on domain-specific knowledge and skills starts. By clearly arranging the steps and providing, verbalizing and reasoning thought models, the instructional messages may prime appropriate organizational processing by the viewer. Using this process of verbalizing and math reasoning the modelling examples do not explicitly provide the schemata for solving a particular problem, but models the actions and strategies used to find a solution (Kollar et al., 2014). The modelling examples consist of sequences of steps the learners need to follow as key decision steps, which are (1) provide a rational structure, (2) reduce complexity, (3) give verbal help and (4) provide modelling. Chunking information in short clips may prevent loss of information from WM (Driscoll, 2005), but this has the consequence that multiple instructional clips are needed to connect domain-specific mathematics knowledge to richer networks of knowledge and to interconnect different elements. Students should be able to solve abstract problems through mathematical manipulation (Canobi, 2009).
To answer the question of how to support students in vocational education with mathematics, the first step is to answer the question of how to work with the "one size fits all" approach. This study aims to investigate whether and how a DLM environment enriched with instructional clips, online guidance, content structuring, and a collaboration tool, as a combination, can be designed to enhance student mathematics' competence in vocational education. The ultimate goal is to reframe instructional design in the mathematics curriculum in vocational education, through direct application and testing. Therefore, the following research questions are formulated to test the effects of DLM including instructional clips, online guidance, structuring of content and a collaboration tool on students' mathematics learning in vocational education: (1) What are the effects of DLM on vocational education students' mathematics learning outcomes for the separate and combined domains of numbers and proportions in the sectors H&W and E&S (study 1) and the domain of geometry in the sector Technology (study 2)?
(2) What is the influence of WM capacity on mathematics learning outcomes for study 1 and study 2?
Because students are able to watch clips that repeatedly show the relationship between ratio, percentages, and fractures and clearly arrange the steps visualizing and reasoning thought models, it is expected that students' pretest mathematics learning outcomes will be improved in the posttest measurements after the DLM intervention. We expect that such positive differences are reflected in the posttest results for all programmes of the H&W and E&S and Technology sectors. Although gains are expected in the sectors, no difference in gains is expected between the sectors because all sectors have the same opportunities for disposal for the materials. The final expectation is that higher WM has a positive influence on both mathematics learning outcomes and students' satisfaction (Alloway et al., 2013;Wei et al., 2012).

Context and participants
This study focuses on students in Dutch vocational education who obtain their qualification by apprenticeship training. i.e. they learn in school for one day per week, and work and learn the other four days in the workplace (Baartman & de Bruijn, 2011). Students in Dutch vocational education must attain a sufficient mathematics level. The level of the programme in vocational education determines the complexity and height of the mathematics level that students need to achieve for their final degree. The difference in complexity and height depends on the single steps a student has to make to solve the problem. For example, reading tables with different types of data require calculating with time for problems with high complexity, in contrast to problems with low complexity that only demand students to map the tables.
The first study took place at "Drenthe College" in Assen (the Netherlands), a school for vocational education. The participants were 18 students from the sectors H&W (N = 12) and E&S (N = 6), selected because they were in the mathematics group taught by the two teachers who participated in this study. The mean age of the participants was 29.5 years (SD = 10.3 years). The minimum age was 20 years and the maximum age was 52 years. Three students were male and 15 were female.
The second study took place at "Deltion College" in Zwolle (the Netherlands), a school for vocational education. The participants were 12 students from the sector Technology (T), selected because these students were in the mathematics group taught by the teacher who participated in this study. The mean age of the participants was 18.2 years (SD = 1.9 years). The minimum age was 16 years and the maximum was 23 years. All students were male. A mathematics course was a required part of the curriculum for all students. The goal was to gain insight and practice tasks to qualify for the national mathematics test. Each sector had a teacher who guided the course online.

E-learning environment
Students used online learning environments (see Figures 1 and 2). The DLM in the course was designed, recoded, and implemented in the learning platforms for the online learning environment. Students were able to complete assignments online and interact within the DLM environment. They could watch instructional clips if they were not able to solve the problems from the book or from the online mathematics assignments. If students were still not able to solve the problems after watching the instructional clips, they could ask questions by using the collaboration tool. Teachers guided the students online by answering questions or re-explaining problems in a different way in the collaboration tool. Students were also allowed to answer or discuss their peers' questions. Each week had the same structure.

Instructional clips in E-learning environment
The instructional clips were built with modelling examples and recorded with audio. The design principles in the instructional clips consisted of sequences of steps that helped the students to solve math problems by key decision steps: (1) offer a rational structure, (2) reduce complexity, (3) give verbal help and (4) provide modelling (Kollar et al., 2014). The instructional clips for domain-specific knowledge related to numbers, proportions and geometry were constructed to range from easy to complex and linked to the content of the mathematics assignments for the week that were used by the school's mathematics programme. The programme's mathematics content was shifted at times, because of need for interconnection between the different elements of the domain-specific knowledge in the instructional clips. Recorded audio in the instructional clips also referred students to earlier clips for prior knowledge, so they were able to choose an earlier clip needed for solving the problem in a more complex task. For problem-solving support, modelling examples were used in the clips (visual and audio).

Online guiding in E-learning environment
The teachers guided the students online, twice a week over eight weeks for each domain. Their role was to give assistance when students had questions about assignments or to re-explain problems in a different way in the collaboration tool. Teachers had two days training on online guiding.

Collaboration tool
In study 1 the online (free) collaboration tool, "Titanpad", was implemented in the learning environment. It is a users friendly web-based application to work online synchronously on a document and chat with other students at the same time. Each student is assigned to a colour that makes awareness for others in terms of who contributes. In study 2, the online collaboration tool was a forum in the learning environment. It is also an online discussion space but students communicate a-synchronously with each other. In both studies, students could interact and ask questions within the learning environment, and teachers and peers could respond to the questions. Each question and response was visible for all users.

Structuring of content
The content was designed to have the same structure every week, so that students could identify useful information, understand how the material fit together and see how the material relates to prior knowledge (Mayer, 2001). On the left side of the screen were the content of the week's assignments (either online or the book) including instructional clips, answers to the assignments and the collaboration tool. The instructional clips, available on demand, were displayed on the right side of the screen, next to the list of assignments (see Figures 1 and 2).

Procedure
Prior to working with the DLM, students of each department were given information on the DLM environment and the online course with DLM during a live class session, which took an hour. Researcher and teacher explained the goal of the experiment and introduced the DLM environment. In study 1, students were enrolled in an eight week course addressing the domain-specific knowledge of numbers and in an eight week course addressing the domain-specific knowledge of proportions. In study 2, students were enrolled in an 8-week course addressing the domain-specific knowledge of geometry. After the information session, students were tested online for their WM with a speed-math test (IDAA, Bekebrede et al., 2010). Students needed to mentally calculate the answers for 30 problems, where the answer was selected from five possibilities. The items varied in difficulty and students had a maximum of five minutes to complete the test. Since higher WM capacity can help students to better encode and manipulate items, as well as critical efficiency of filtering irrelevant information, the test scores show that the coding and interpretation of information is facilitated by WM. The reliability coefficient of this test is 0.89. In both studies students also took a mathematics equivalent pretest and posttest specific to the mathematics programme at the vocational education school (Lagendijk et al., 2010): 20 tasks about domain-specific knowledge of numbers and proportions each (study 1), and 20 tasks about domain-specific knowledge of geometry (study 2).

Statistical tests
A one-way analysis of variance (ANOVA) test was used instead of a t-test to ascertain whether the means of more than two groups significantly differ. We analysed if mean scores students' mathematics grades are significantly different for the separate and combined domains of numbers and proportions in study 1 and the domain of geometry in study 2. Furthermore, an ANOVA test was conducted to compare mean differences between students from the H&W and E&S sectors on their separate and combined mathematics gains for numbers and proportions in study 1. For both studies, regression analysis was calculated to determine the relation between WM score and students' mathematics learning outcomes. Table 1 shows an overview of descriptive analysis of the data for both studies in terms of gender, mean age, level of study, sector, mathematics scores for the pretest and posttest on domain-specific knowledge, and WM score.

Results for research question 2
There was no significant correlation between WM scores and the differences between the pretest and posttest mathematics results in the first study for numbers (r = .30, p = .30), proportions (r = .10, p = .74) and their combination (r = .34, p = .26). Similar to the first study, no significant correlation between WM and geometry learning outcomes was seen for the second study (r = .04, p = .91).

Conclusion and discussion
The present studies were designed to determine the effect of the DLM on the mathematics learning outcomes of apprenticeship students in vocational education. Furthermore, the relationship between WM scores and mathematics results was investigated. It was hypothesized that learning gains were expected for both studies. Moreover, it was expected that students with high WM would achieve higher mathematics results. Based on these two studies, the general conclusion can be drawn that DLM can foster positive learning outcomes and knowledge construction (e.g. Noroozi, Busstra et al., 2012). Furthermore, the DLM environment including instructional clips, online guidance, structuring of content and a collaboration tool helped students improve their mathematics scores, especially those in study 1. The implementation of DLM improved students' domain-specific mathematics knowledge for both numbers and proportion domains except for the geometry topic. It appeared that in contrast to the hypothesis, differences in mathematics results could not be accounted for by WM scores.
With regard to the second research question, it was hypothesized that no differences would be found between the two sectors. On the contrary, students in the H&W sector demonstrated significantly higher learning gains for proportions and for numbers and proportions combined than students in the E&S sector. The students of H&W were involved in the assignments and collaborated as a group, and if they found technical and organizational problems the teacher was immediately notified. This had an effect on their results, because these students showed greater learning gains than the students of E&S. Best, Miller, and Naglieri (2011) argue that mathematical problem-solving depends on involvement of students in the learning tasks. The students of E&S were not interested in collaborating and working with DLM. They did not see the added value of general mathematics in their future profession. Students in the second study from the Technical sector also failed to see the added value of mathematics in education. These students worked with the DLM because their teacher asked them to do so. Furthermore, they argued that time was a big issue. These students work four days (sometimes even more) a week, and they only had one day a week for their study. Students need that time to finish their workplace assignments. Though the students had the possibility to practice online mathematics during seven days of the week, the DLM environment did not motivate them that much to practice more because it did not support their workplace learning. At this point, we might have overestimated the teachers' expertise in giving students guidance on technology use (Knezek & Christensen, 2008). The teachers' role to close the gap between the innovation of the DLM environment and classroom curricula did not sufficiently emerge during and after sessions. We ignored key system factors that cannot be manipulated by the innovation such as assessments, technology policies and infrastructure (McKenney, 2013). In a productive DLM environment, in order to construct knowledge, students need to work together instead of on their own (Kanselaar, Andriessen, de Jong, & Goodyear, 2000). However, collaborative learning needs to be structured and guided appropriately (e.g. Gillies, 2004;Kollar, Fischer, & Slotta, 2007), otherwise students often engage in low-level learning processes because of low-level argumentation (Kollar et al., 2007;Noroozi, Kirschner, Biemans, & Mulder, in press;Noroozi, Weinberger et al., 2012). We envisage a central role for the online teacher here, in giving impetus to this collaboration. For this to occur successfully, the teacher needs to be able to scaffold student participation.
This study had a small sample of participants for making claims that are not based on coincidence. However, it was a first effort to research the development of the educational concept of DLM in Dutch vocational education. Questions arise as to whether difficulty in attending to key task dimensions and the passive approach to task completion are variables that should be taken into account, instead of using only a WM test. The present study did not analyse the different assignments on the students' pretests and posttests. Furthermore, replication of this study in vocational, secondary, or higher education with more students, a control group and under more controlled conditions would be needed to confirm the results of this study.