The Impact of a Proposed Mathematics Enrichment Program on UAE Students’ Mathematical Literacy Based on the PISA Framework

Abstract

The main objective of this study is to investigate the impact of a proposed mathematical enrichment program based on the Programme for International Student Assessment (PISA) framework for mathematical literacy. Mathematical literacy is essential because it helps students understand and use real-world mathematics frequently tested in many international assessments. PISA has received special attention because of its place on the national agenda of the United Arab Emirates (UAE), which aspires to become a globally preeminent nation. This quantitative research employed a non-equivalent pre- and post-test quasi-experimental method. A mathematical literacy test was used to collect data from a sample of 102 students of 10th grade in the UAE. The findings revealed a statistically significant difference between the experimental and control groups and an effect size above the mean according to the covariate analysis. The female students recorded greater improvement than the males. Overall, the results obtained from this study revealed that implementing mathematics enrichment programs based on the PISA framework might be one of the solutions to enhance students’ mathematical literacy.

1. Introduction

In 1998, the National Research Council [1] stated that mathematical skills are needed to participate fully in everyday life. Although this statement was written over twenty years ago, many still consider it a valid suggestion. Mathematics is not only the language of science, but also the language needed to engage with domains such as finance, business, and health. Additionally, it provides nations with the knowledge to compete in the technological sphere [2].

The literature around the goals of mathematics education reflects the increasing use of practical mathematical knowledge of quantitative problems in everyday life. Gellert and Jablonka [3] argue that the “mathematisation of society” has, more than ever, created the need to equip students with the appropriate mathematical knowledge and skills to allow them to participate effectively in this “mathematised world”, and to understand the quantitative problems they face in their lives. Consequently, appropriate and relevant goals in mathematics education is on the political agenda of many countries and, indeed, the UAE is no different; it is an ambitious country which is focused on equipping a new generation which can face life’s challenges at the international level. Education in the UAE is considered an essential element in the development of the nation and the best investment in its youth; as such, it aims to invest in the knowledge economy instead of relying on oil and gas [4]. For this reason, the National Agenda for UAE Vision 2021 emphasized that the current education system must become first-rate and have teaching methods suited to the country’s future needs [5].

National governments use the results of international assessments to guide educational policy, often under the slogan of “raising standards”. Based on their experience as researchers conducting international studies, Stigler, Gallimore and Hiebert [6] emphasized how this valuable kind of research permits intercultural comparisons. By discovering practices which are not observed everywhere, we can reflect both on these and our own ways of doing. As such, this provides researchers and educators with alternative ways to improve mathematics teaching [7].

The National Agenda of the UAE has set eight major educational goals, one of which is for UAE students to be among the global best at reading, mathematics, and science in international tests such as PISA (Programme for International Student Assessment). PISA was created by the Organization for Economic Cooperation and Development (OECD) to assess achievement in reading, mathematics, and scientific literacy, with its first cycle in 2000. PISA is of particular interest as it is implemented every three years for students aged 15, who are thus approaching the end of compulsory education in most participating education systems. PISA is not curriculum-oriented and goes beyond the school curriculum to measure the use of knowledge in daily tasks and challenges. For this reason, it measures young people’s success in acquiring knowledge and their ability to use it in specific areas to meet real-life challenges. Although all three subjects are evaluated in each cycle, one major topic is assessed in depth, and the other two are minor areas of that cycle; the major topic is rotated in each cycle. Mathematical literacy (ML) is the main subject of the postponed PISA exam of 2021, which will be held in 2022 due to the COVID-19 pandemic.

The UAE National Agenda [8] set a goal for the UAE to be among the top 20 countries in the PISA assessment. PISA allows the UAE to compare its students’ achievements with those in other countries and to examine the effectiveness of their educational systems. According to Sanderson ([9], p. 1), the PISA 2018 results showed that the UAE ranks highest in the Arab world for all three subjects of reading, mathematics, and science literacy; however, no Arab country exceeded the OECD average in these subjects. Moreover, looking at the same 2018 results, UAE students had generally low performance and ranked 50th in mathematics out of nearly 80 countries, despite an increase in score of about eight points compared to the 2015 cycle [10]. Thus, the UAE must do more to rank among the top 20 countries, as its results indicate that the general path of the UAE is “stable” [10], as supported by UAE results in previous PISA cycles.

Table 1. UAE results in previous PISA cycles cycle [10,11].

	PISA 2009		PISA 2012		PISA 2015		PISA 2018
	Score	Rank	Score	Rank	Score	Rank	Score	Rank
Reading	431	42	442	44	434	47	432	46
Mathematics	421	41	434	48	427	47	435	50
Science	438	41	448	46	437	46	434	49

shows the results for each main subject area.

Additionally, the existing literature on gender and academic achievement have different perspectives and findings. For example, the trend in OECD countries has been that male student achievement in mathematics in PISA outperforms females with males scoring five points higher than females [12]. However, student results in the UAE were inconsistent with the OECD trend where females demonstrated better mathematical literacy than males. The results showed girls outperforming boys in mathematics by nine points [12].

Statement of Problem

The UAE is making great strides towards education reform and developing the first educational system to achieve both Vision 2021 and the National Agenda goal of being among the global best. The aim of education reforms is to ensure that all students maximize their potential at school [13]. As such, the UAE has invested heavily in educating its next generation as it seeks to reduce its dependence on oil and gas. In January 2017, Hussain Ibrahim Al Hammadi, UAE Minister of Education said, “We want to move from an economy based on oil to a new economy based on the human knowledge of both nationals and expatriates alike who will use knowledge as a tool to compete and move the country forward” ([14], p. 1). Those students who benefit from this educational reform must use mathematics in their everyday lives because helping students become successful people outside of the classroom is just as important as teaching the curriculum.

Enhancing students’ competencies and abilities in various literacy skills in reading, mathematics and science by creating an ideal, high quality educational foundation is a key pillar of the UAE National Agenda [8]. Andreas Schleicher, Director for Education and Skills at the OECD, said of the UAE’s performance: “In mathematics, we are seeing the continuation of a positive trend, here the UAE is broadly on track of achieving its ambitious performance targets” [9]. However, this ranking shows that students’ ability to use mathematics to think about their lives, make plans for their future, and think about important problems and issues in their lives is insufficient to the achievement of the UAE’s goal of scoring among the top 20 countries. Moreover, the PISA 2018 results revealed that only about 5% of UAE students can perform at the fifth and sixth levels [15]. These alarming findings indicate the need for educational intervention to enable UAE students to perform at higher levels of mathematical literacy because they are the primary force for nation-building in many areas of the country.

Nevertheless, an educational environment is required which meets the needs of all learners, including the gifted and high achievers. In this regard, the National Council of Teachers of Mathematics (NCTM) proposed that “all students be guaranteed equal access to the same curricular topics [and] does not suggest that all students should explore the content to the same depth or at the same level of formalism” ([16], p. 131). Previous research on the current state of mathematics education for the gifted in the UAE has revealed limitations concerning specialized mathematics programs for gifted students. Moreover, mathematics teachers were also negative regarding the effectiveness of these gifted programs, where available [17]. Thus, a mathematics enrichment program (MEP) is proposed in this study, to address these limitations and increase students’ readiness for the future and their ability to use what they learn effectively. The effectiveness of this MEP was tested with tenth grade students.

2. Literature Review

2.1. What Is Mathematical Literacy?

The conversation about the importance of being “mathematically literate” began in the early eighties and continues even today [18]. Literacy goes beyond the ability to read and write as it also includes mathematics, which is considered of equal importance in the definition of literacy [19,20,21]. Undoubtedly, the ability to use numbers and interpret quantitative information is an important component of literacy, in addition to speaking, writing, and reading. The use of the term “literacy” may refer to a certain level as it does in other compound phrases, such as “statistical literacy” or “computer literacy” [22].

The NCTM Standards [16] describe ML and being mathematically literate but without giving an explicit definition, although the NCTM has set five broad goals for ML for all students: “(1) That they learn to value mathematics; (2) that they become confident with their ability to do mathematics; (3) that they become mathematical problem solvers; (4) that they learn to communicate mathematically; and (5) that they learn to reason mathematically” ([16], p. 5). Overall, people who can apply mathematics in real-life situations and arrive at a solution can be considered mathematically literate [19]. The OECD [22] suggests that ML implies the functional use of mathematical knowledge and skills rather than just mastering them as part of a school curriculum. In this sense, ML goes beyond curricular mathematics. However, ML assessment is inseparable from current curricula and teaching methods because students’ knowledge and skills depend, to a large extent, on what and how they learned them in school and how that learning was assessed [23,24].

2.2. PISA Definition of Mathematical Literacy

ML is gaining more focus in curriculum reform as it is measured in some international comparative assessments such as PISA and Trends in International Mathematics and Science Study (TIMSS). According to Jablonka [19], the first attempt to define mathematical literacy was within the initial OECD framework for PISA, seeking to test students’ ability “to put their mathematical knowledge to functional use in a multitude of different situations” [25]. PISA adopts a “real-life literacy” perspective rather than a curriculum-driven one ([26], p. 3). However, this definition was modified for PISA 2012 and, more recently, the definition of ML continues to have the same focus, with slight changes each cycle until reaching the definition proposed for the postponed PISA 2021, where the major subject will be ML. Here, the definition adopted by the OECD is: “Mathematical literacy is an individual’s capacity to reason mathematically and to formulate, employ, and interpret mathematics to solve problems in a variety of real-world contexts. It includes concepts, procedures, facts and tools to describe, explain and predict phenomena. It assists individuals to know the role that mathematics plays in the world and to make the well-founded judgments and decisions needed by constructive, engaged, and reflective 21st century citizens” ([12], p. 8).

2.3. PISA Framework of Mathematical Literacy

ML in the PISA framework for 2021 consists of two inseparable parts, namely mathematical reasoning and problem-solving [12,26]. Mathematical reasoning is at the centre of the problem-solving cycle (modelling cycle). While ML plays a vital role in the ability to use mathematics to solve real-life problems, mathematical reasoning goes beyond problem-solving in its traditional sense to include making judgments about societal problems that can be solved using mathematics. Figure 1 represents the ML of the PISA 2021 framework.

The mathematical modelling cycle (the problem-solving cycle) takes place with a “problem in context”. Contextual problems are defined as problems that involve a setting that exists outside of pure mathematics [27]. According to Stacey [28], mathematical modelling contains three processes: formulating, solving (employing), and interpreting. To begin solving the contextual problem, the individual attempts to formulate the situation mathematically based on the relevant mathematics identified in the problem situation. In this stage, the problem solver transforms the “problem in context” into a “mathematical problem” to apply the mathematical treatment. Then, mathematical concepts, procedures, facts, and tools are employed to find “mathematical results”. This is the stage where mathematical reasoning, manipulation, transformation, and computation take place. In the next stage, the “mathematical results” need to be interpreted in terms of the original problem as “results in context”. The problem solver needs to “interpret, apply, and evaluate” the mathematical solution in the real-world context of the problem [25]. In this regard, Jablonka [19] stated that “ML is connected to learning how to think, but not to learning what to think about” (p. 82). The primary implications of a pedagogical emphasis on ML for mathematics teachers are clear: mathematics must be logical for students to understand, and it must be based on their previous and current experiences, as well as likely future ones [29]. In addition, in research from Hendroanto et al. [30] who believe that students’ success in acquiring problem solving skills required to solve PISA-like problems is determined by teachers’ abilities in developing students ML furthermore, mathematics educators believe that ML skills are influenced mainly by student’s ability to solve a contextual mathematical problem similar to PISA-like problems.

This study sought to answer the following questions:

• What is the impact of the mathematics enrichment program on the mathematical literacy of tenth grade female students?
• What is the impact of the mathematics enrichment program on the mathematical literacy of tenth grade male students?
• Are there any gender-based significant differences in mathematical literacy in response to the mathematics enrichment program?

3. Methods and Materials

A quantitative, quasi-experimental design was used to determine the effect of an enrichment program on students’ mathematical literacy. It was not possible to sample individuals within groups because of the obtaining administrative approval difficulty for the random selection and removal of a small number of students from their classes for this study. A non-equivalent pre- and post-test design was chosen to examine the establishment of a cause–effect relationship between two or more variables [31,32]. The experimental groups of males and females were identified and enrolled separately in the math enrichment program, but the control groups were not enrolled in the MEP.

3.1. Participants

This study was conducted in UAE state schools for boys and girls. State schools were purposefully selected to reduce differences between participants, because students in public schools study the same curriculum and are exposed to the same assessment methods. In addition, the teaching in state schools is standardized by the Ministry of Education (MoE), while students enrolled in private schools follow different curricula that could influence their knowledge of mathematics. Moreover, the advanced stream was chosen for this study to broaden the definition of gifted to include a much larger group of students and potentially those with outstanding mathematical abilities, rather than just those identified with pre-existing mathematical expertise and passion [33]. Thus, the choice of the advanced stream was appropriate for this study as it normally includes gifted students. The sample of male and female students were in the tenth grade, in which most pupils are 15 years old.

A random sampling technique was used, whereby intact groups (not individuals) were randomly selected. Two schools were selected, one for males and one for females. In each school, two groups were randomly assigned as control and experimental groups. Three females and five males were excluded due to missing more than three lessons from the program. The final number of participants was 102, of whom 9 (8.8%), 73 (71.6%), and 20 (19.6%) were students aged 14, 15, and 16, respectively. Of the 102 participants, 53 (51.96%) were males and 49 (48.03%) females, as shown in

Table 2. Distribution of participants.

Groups	No. Students		Total
	Male	Female	Subtotal	Overall Total
Experimental	27 (50.9%)	24 (48.9%)	51	102
Control	26 (49.1%)	25 (51.0%)	51
Total	53 (100%)	49 (100%)	102

.

3.2. Study Instruments

3.2.1. Mathematics Enrichment Program (MEP)

This proposed MEP aimed to improve the students’ mathematical literacy. The content of the enrichment program consisted of two stages, namely a review of the basics of prior knowledge required for each lesson, and the relevant PISA-released items. PISA-released items were appropriate to the students’ cognitive level as they are designed to test 15-year-old students. The MEP consisted of the four PISA mathematical literacy content areas of quantity, uncertainty and data, change and relationship and space and shape. Moreover, the solving of these problems can be performed by formulating, employing, and interpreting, with reasoning being essential to all these processes. Additionally, the four contexts based on PISA framework (personal, occupational, scientific, and societal) were covered. For this purpose, the scope of the content for the MEP in this research was restricted to the comprehensive framework of mathematical literacy prescribed in PISA. Considering the framework of mathematical literacy in PISA (Figure 1).

The main components of mathematical literacy involve mathematical thinking, such as reasoning, modelling, and making connections between ideas [28]. According to Piaget’s theory of constructivism, students of this age are cognitively capable of reasoning and solving problems that support the relevance of the MEP program, because students aged 15 are in the formal operational stage [34].

The program duration was eight weeks plus two weeks for the pre- and post-tests. The development of the MEP took several steps to reach its final form. After its initial development, the MEP was presented to a group of experts in teaching and learning mathematics, consisting of a professor of mathematics education, a professor of mathematics, and five expert mathematics teachers with more than five years’ experience of teaching grade ten. All the experts’ comments were addressed before administering the MEP to students.

3.2.2. Mathematical Literacy Test (MLT)

The distribution of Mathematical Literacy Test (MLT) items are presented in

Table 3. Distribution of test items in MLT [35].

Part 1: Questions 1–26				Problem Solving
No.	Item Name	Item Difficulty	Level of Proficiency	Process			Content				Context
				Formulate	Employ	Interpret	Change and Relationships	Quantity	Space & Shape	Uncertainty & Data	Personal	Occupational	Scientific	Societal
1	Charts Q1 *	347.7	BL1
2	Charts Q2	415.0	L1
3	Charts Q5	428.2	L2
4	Which Car? Q1 *	327.8	BL1
5	Which Car? Q2	490.9	L3
6	Which Car? Q3	552.6	L4
7	Garage Q1	419.6	L1
8	Garage Q2.1	663.2	L5
9	Apartment Purchase Q1	576.2	L4
10	Drip Rate Q1	610.0	L5
11	Drip Rate Q3	631.7	L5
12	Revolving Door Q1	512.3	L3
13	Revolving Door Q2	840.3	L6
14	Revolving Door Q3	561.3	L4
15	Sauce Q2	489.1	L3
16	Sailing Ships Q1	511.7	L3
17	Sailing Ships Q3	538.5	L3
18	Sailing Ships Q4	702.1	L6
19	Climbing Mount Fuji Q1	464.0	L2
20	Climbing Mount Fuji Q2	641.6	L5
21	Climbing Mount Fuji Q3	591.3	L4
22	Helen the Cyclist Q1	440.5	L2
23	Helen the Cyclist Q2	510.6	L3
24	Helen the Cyclist (E) Q3	696.6	L6
25	Ferris Wheel Q1	592.3	L4
26	Ferris Wheel Q2	481.0	L3
	Total number of Problems solving items		26	8	14	4	7	7	8	4	8	4	6	8
	Percentage %		100	31	54	15	27	27	31	15	31	15	23	31
	PART 2: Q27–34		8	Reasoning

* BL1 = below level one. Source: [35].

.

This test consisted of 34 problems drawn from released PISA materials published on the OECD website [35]. The test problems first part primarily attempted to assess the students’ problem-solving at six proficiency levels presented in 26 of the test problems that are addressing the process, content, and context based on the PISA framework. For example, the problems covered the process as follows: 8 items (31%) assessed the formulate process while 14 items (54%) assessed the employ and 4 items (15%) covered the interpretation process. The same 26 problems also covered the four content areas and the four context areas as presented in Table 1.

In addition to eight problems that measured their reasoning skills, in PISA, each question is assigned a difficulty level [35]. Using item response theory and these difficulty levels, raw scores were converted to a score on the PISA scale. The PISA scale in mathematics was also divided into six mathematical literacy levels to represent degrees of proficiency, with level six being the highest. This test served as a pre- and post-test and aimed to measure the students’ mathematical literacy levels based on content, context, and processes.

3.3. Methods of Data Analysis

Descriptive and inferential statistics were used to address the research questions posed. An analysis of covariance (ANCOVA) was conducted on the quantitative achievement data to identify any differences in the ML mean scores of the respective male and female experimental and control groups [31,32]. The effects of the students’ mathematical literacy level were controlled for by setting the pre-test as a covariate. A p-value of less than 0.05 is considered statistically significant. In the UAE, male and female students attend separate schools, and so this study was implemented in the female school and repeated at the same time in the male school to study the effect of MEP on both.

4. Results

The results included descriptive statistics and the analysis of group and gender differences to compare groups, as shown below.

4.1. Descriptive Statistics

The means and standard deviations were calculated for both the pre- and post-tests of ML for both the males and females, as shown in

Table 4. Descriptive statistics for the ML of the tenth grade students.

Mathematical Literacy Test		Experimental Group					Control Group
		Pre-Test		Post-Test			Pre-Test		Post-Test
	N	M	SD	M	SD	N	M	SD	M	SD
Female students	24	11.79	3.87	18.17	5.00	25	9.20	3.58	10.80	3.89
Male students	27	5.37	2.39	12.81	4.39	26	5.69	2.31	8.85	1.71

.

Table 4 shows that the post-test ML results for males and females increased in both the experimental and control groups, but that the increase in the experimental group was much greater for both genders. The female students averaged 18.17 in the experimental group and 10.80 in the control, while the males averaged 12.81 in the experimental group and 8.85 in the control. This was a good starting point for inferring the positive impact of the MEP. Hence, with the experimental group scoring higher than the control in the post-test, it was expected that this could be due to the intervention, provided that other confounding variables were controlled. Consequently, to ensure that this post-test difference was indeed a result of the treatment and not random variation in the pre-test between groups, a one-way ANCOVA was used to examine the effectiveness of the MEP in controlling the pre-test of ML [32].

4.2. Analysis of Group Differences

The effect of the MEP on the ML of the male and female students, who were taught in separate schools, was investigated using ANCOVA by controlling the pre-test. Thus, the effect of the MEP on the adjusted ML post-test of the females was examined using one-way ANCOVA, as

Table 5. ANCOVA results for the female students’ ML.

Source	Df	Mean Square	F	p	η2
Pre-test	1	503.311	53.099	0.000	0.536
Group	1	281.653	29.714	0.000	0.392
Error	46	9.479
Total	49

shows.

Table 5 shows that the ANCOVA test was significant F(1, 46) = 29.714, and p = 0.000 < 0.0005, partial η² = 0.392. The effect size eta-squared is interpreted as small, medium, and large if it possesses the values 0.01, 0.06, and 0.14, respectively [36]. Hence, for this study, the effect size is large, meaning that 39.2% of the ML post-test results were due to the MEP. Similarly, the effect of the MEP on the adjusted ML post-test of the males was examined using ANCOVA and is shown in

Table 6. ANCOVA results for the male students’ ML.

Source	Df	Mean Square	F	p	η2
Pre-test	1	41.863	5.886	0.019	0.105
Group	1	220.790	31.045	0.000	0.383
Error	50	7.112
Total	53

below.

Table 6 shows that the ANCOVA test was significant F(1, 50) = 31.045, and p = 0.000 < 0.0005, partial η² = 0.383. The effect size is large and means that 38.3% of the ML post-test results were due to the MEP, indicating that the implementation of MEP on the experimental group was positive regarding the ML of both the female and male tenth grade students.

A post-hoc comparison performed using the Bonferroni method to control for Type 1 errors [32] showed a statistically significant difference between the experimental and control groups (p < 0.0005), which also indicates that the implementation of the MEP had a positive effect on the ML of both sets of students. This can also be seen clearly from

Table 7. Adjusted and unadjusted means for ML.

Mathematical Literacy Test		Experimental					Control
		Unadjusted		Adjusted			Unadjusted		Adjusted
	N	M	SD	M	SE	N	M	SD	M	SE
Females	24	18.17	5.00	17.00	0.648	25	10.80	3.89	11.92	0.634
Males	27	12.81	4.39	12.88	0.514	26	8.85	1.71	8.78	0.524

. The experimental group outperformed the control when the adjusted mean scores of the experimental and control groups were compared with pre-tests as a covariate for both female and male students. Table 7 shows the adjusted mean scores for ML for the students in the experimental and control groups using pre-test as a covariate.

There is a demonstrable difference across the means of the experimental and control groups, and the results of the experimental group in the post-test of ML were better than those of the control group for both females and males. In other words, the ML of the females and males in the experimental group improved as a result of the MEP.

4.3. Analysis of Gender Differences

Similarly, to study the gender differences, ANCOVA was also used to compare the difference in the effect on females and males with the pre-test as a covariate to control for students’ previous levels.

Table 8. ANCOVA results for all students’ ML.

Source	Df	Mean Square	F	p	η2
Pre-test	1	469.388	52.394	0.000	0.522
Group	1	5.193	0.580	0.450	0.012
Error	48	8.959
Total	51

shows that the ANCOVA test was not significant F(1, 48) = 0.580, p = 0.45, partial η² = 0.012, indicating that there was no statistically significant difference in the post-test of ML results between the female and male groups when adjusted for the pre-test results. This indicates that both the males and females gained a similar increase in their level of ML. Although the adjusted means using the Bonferroni method [32] show that the females outperformed the males, the difference was very small and so insignificant.

Table 9. Adjusted and unadjusted means for ML for all students.

		Unadjusted		Adjusted
Groups	NO.	M	SD	M	SE
Females	24	18.17	5.00	15.77	0.764
Males	27	12.81	4.39	14.85	0.706

shows the adjusted mean scores for ML for the experimental groups of females and males using the pre-test as a covariate.

Based on the previous ANCOVA tests, it was found that the MEP had a positive effect on the students in the experimental groups, whether male or female.

Furthermore, students were asked the following question, “Do you recommend running this program for students to improve their mathematical literacy?” Most student responses showed that they would recommend it based on their experience. The frequencies and percentages of all students who would recommend the program are presented in

Table 10. Percentages of students who would recommend the program.

Students Who Would Recommend the Program		F	%
Females	Yes	22	91.7
	No	2	8.3
	Total	24	100.0
Males	Yes	22	81.5
	No	5	18.5
	Total	27	100.0
All students	Yes	44	86.3
	No	7	13.7
	Total	51	100.0

.

The number of students of both genders who would recommend the program was forty-four (86.3%), while only seven would not (13.7%). Most of the students who would not recommend were males (five males versus two females). In short, the large percentage of students who would recommend the MEP is another clue to its positive impact on students’ ML.

5. Discussion and Conclusions

Most students perceive mathematics as a difficult subject due to the lack of real-life connection and unattractive teaching methods [37]. However, the proposed MEP, which mainly consisted of contextual mathematical problems, was successful and increased the students’ mathematical literacy. Thus, the increase in students’ achievement in ML is likely due to more meaningful learning, which allows information to be stored more quickly and remembered more easily for retrieval [38].

All potential confounding variables such as time difference, teacher influence, and topics to be covered were controlled. Thus, it is appropriate to draw meaningful conclusions based on the effect of the treatment. The study showed that student participants in the MEP were more likely to improve their ML. This study demonstrated the positive impact of an MEP designed to focus on solving contextual problems and reasoning as the main components of ML. With the goals of improving ML and meeting the UAE Vision 2021 of being among the top 20 countries for PISA, this study is a major first step in establishing data-based decision-making protocols and processes for analysing instructional programs.

The mean scores for both genders on the post-test of ML were increased, reflecting the positive impact of the MEP on the students’ ML. This increase was explained by the results of the post-test compared to the pre-test, with the latter being a covariate to control for students’ previous levels. This finding supports those of other studies, for example [39,40], which indicated that using contextual problems improved students’ achievement in mathematics. Our results showed that increased ML was more evident among females than males.

The results of the current study also revealed no significant difference between male and female students, in line with other studies [41,42]. However, the females in our study showed slightly better ML than the males. Similar results were revealed by Ajai and Imoko [43], who also employed a quasi-experimental pre-test. Moreover, our results were consistent with the PISA 2018 results for the UAE, as girls outperformed boys in mathematics by nine score points; this contrasted with the trend in OECD countries, in which males scored five points higher than females [15].

More recently, much previous research has laid great stress on producing valid and practical PISA-like questions, because the PISA test is considered one of the most powerful measurements of literacy in mathematics. These questions were designed and examined by various researchers [44,45,46,47], who developed PISA-like problems in different content areas of mathematics and reached the same conclusion as our study on the positive impact of these problems on the students’ ML.

Implications and Future Research

The implications of the study indicate that the mathematics curriculum should regularly include contextual problems. Mathematical modelling, such as with contextual problems, is recommended for all curricula and grades and curriculum writers are encouraged to consider the potential of modelling in promoting mathematical proficiency and engagement when problems are meaningful to students.

The NCTM [48] has indicated that assessment and education should be complementary, such that the assessment provides information for the teacher to use in making educational decisions. Thus, it may be necessary to apply higher level tests with more difficult elements than achievement tests appropriate for the grade level. Thus, if the test does not contain sufficient elements of difficulty appropriate for the student, the result may not indicate the true level of their understanding [49].

In fact, it is believed that adding meaningful context to mathematics problems has the potential to promote student motivation [50]. For further research, students could be interviewed to gain more insights to enhance such programs, in addition to measuring students’ actual engagement in the classroom through observations, teacher reports and questionnaires. Furthermore, research should study the students’ self-regulated learning skills as these could be another important factor that explains the results of studies such as this. This view is also supported by [51].