Prediction of Gender-Biased Perceptions of Learners and Teachers Using Machine Learning

Abstract

Computers have enabled diverse and precise data processing and analysis for decades. Researchers of humanities and social sciences are increasingly adopting computational tools such as artificial intelligence (AI) and machine learning (ML) to analyse human behaviour in society by identifying patterns within data. In this regard, this paper presents the modelling of teachers and students’ perceptions regarding gender bias in text books through AI. The data was collected from 470 respondents through a questionnaire using five different themes. The data was analysed with support vector machines (SVM), decision trees (DT), random forest (RF) and artificial neural networks (ANN). The experimental results show that the prediction of perceptions regarding gender varies according to the theme and leads to the different performances of the AI techniques. However, it is observed that when data from all the themes are combined, the best results are obtained. The experimental results show that ANN, on average, demonstrates the best performance by achieving an accuracy of 87.2%, followed by RF and SVM, which demonstrate an accuracy of 84% and 80%, respectively. This paper is significant in modelling human behaviour in society through AI, which is a significant contribution to the field.

1. Introduction

The advent of new computational tools such as artificial intelligence (AI) has made it possible to analyse diverse and big data belonging to different disciplines of applied sciences. The same trend is now emerging in many areas of social and behavioural sciences [1,2]. AI has made it possible to replicate human intelligence in machines and to provide accurate prediction and decision-making [3,4]. The interdisciplinary crossroads of AI are now increasingly transforming the way the data is analyzed and solutions to different critical problems and investigations are sought. According to Bostrom [5], a machine trained with AI could supplant academicians, physicians, lawyers and social workers in the future.

A report by UNESCO 2019 [6] highlighted the bridging of gender discrimination and digital assistance, featuring recommendations on overcoming gender gaps in digital skills focusing coding on numerous AI applications. According to the critical findings of UNESCO 2019, global dialogue on the nexus of AI and gender bias is required and AI approaches skew significantly towards gender impartiality, which can lead to constructive changes in society. For example, an instance of this would be AI-powered recruitment software, which can be deployed to eliminate gender biases against women and, at the same time, should benefit employers in incorporating gender-sensitive language for writing more inclusive job postings.

Research on highlighting gender discrimination can play a crucial role in eliminating gender biases in society. Different attempts have been made at the government level in Pakistan to deal with this issue, especially in the education sector. The government of Pakistan is reinforcing international commitments to include gender justice in Pakistan’s Education Sector Reforms (2001–2005) and also in the National Plan of Action (2001–2015). The focus is to make the curriculum align with the needs of learners and remove gender discrimination from the text books and curricula [7].

Researchers have analysed different text books in Pakistan at various levels and have established that gender partiality prevails in these text books [7,8,9]. Primarily, the studies cited above have examined the depiction of gender in the text books. According to Sunderland [10], the analysis of gender representation in text books only cannot provide a practical solution to the issue of gender discrimination. She points out that what is inside the minds of learners and teachers need to be taken into consideration to understand the phenomenon comprehensively. Keeping this aspect in focus, this paper examines the gender perceptions of teachers and learners in addition to the analysis of text books taught at undergraduate level in the Khyber Pakhtunkhwa (KPK) province of Pakistan. The gender perceptions of the teachers and learners are modelled with AI, which can be used to predict the gender bias in the text books, which has not been studied before. The paper addresses the following research questions:

• How is gender portrayed in the text books, and what are the perceptions of teachers and learners regarding gender depiction?
• To what extent does AI-assisted gender-portrayal prediction vary according to different themes?

The main contributions of this paper are as follows:

• An analysis of the two text books taught at intermediate level to discern gender portrayal in these text books.
• An investigation of the perceptions of teachers and learners regarding gender portrayal in the English text books.
• A comprehension and discussion of the AI-assisted gender-portrayal predictions as perceived by the teachers and learners.
• A reflection on the implications of the integration of human–AI analytical disparities around gender inequality.

The paper is significant to bridging the gap between the nexus of AI and gender-bias prediction which has yet to be explored, to the best of our knowledge. The remaining of the paper is organized as follows. Section 2 presents an overview of related work. Section 3 presents the material and proposed methodology. Section 4 presents experimental results. Finally, the paper is concluded in Section 5.

2. Literature Review

AI takes advantage of human experience to design intelligent machines [11,12,13]. There is a global policy program to endorse the attainment of United Nations Sustainable Development Goals by 2030 and to improve quality of life [14]. AI has been used to supplement human intelligence through natural language processing, image analysis, sensors, internet of things and many other applications which are impactful in considering the synthesis of humans and AI and the effects of this nexus on society [15,16].

AI, for several years, has been applied successfully in the applied sciences domain but now it is finding its path in diverse fields such as social and behavioral sciences [17]. This significant upsurge is due to the availability of data for AI training and the improvements in computational tools for data analysis. Social and behavioral sciences are employing research methodologies driven by AI [18,19]. Many examinations of AI usage have been employed in the recent times, for instance, in the field of health sciences [20]. AI is seen as a way to provide countless opportunities for the betterment of society. It is considered a fundamental area for attaining the UN Sustainable Development Goals and has the potential of providing cost-effective, user-friendly and precisely targeted services [21].

When different social groups interact, stereotyping is evident and one manifestation of this is gender stereotyping [22]. It associates conventionally and socially perceived attributes and qualities with specific genders [23,24]. On one hand, this stereotyping substantiates overgeneralized social categorizations [25] and on the other hand, it increases influence on the social behavior and attitudes leading to discrimination, prejudice and other social conflicts [24]. As asserted by Raudman et al. [26], gender stereotyping can be seen in mass media as well as in the general public. For instance, gender differences are accentuated, whereas in fact the different genders have more similarities than differences [27]. Moreover, there is varying intensity of gender stereotyping across different cultures [28].

Gender bias has been identified in academia [29,30]. Nevertheless, the implementation of gender-equality policies in society and academia is hardly visible, even in this modern world [31]. Women are persistently underrepresented in higher ranks of academic institutions (European Commission [29,32]), as well as in editorial bodies [33]. Men are relatively better paid than women [34], and the research funding of women is less likely to be granted than that of men [35]. In addition, while analyzing the comparison of promotion patterns, men are seen to be promoted to higher positions more often than women [36].

Gender discrimination denotes that the treatment of genders is unequal and disproportionate, specifically, the unequal treatment of women by men based on the belief that men are superior to women. Roussel [37] asserts, regarding gender discrimination, that gender discriminators regard men as privileged and superior to women. In addition, it is based on the physiological dissimilarities among women and men that their economic and social ranks are established [38].

Language is employed to share, express and establish the thoughts and beliefs of the people of a society [39], and if the society is patriarchal, gender discrimination is rooted in the language. This needs to be studied not only to understand the phenomenon but also to eliminate it [39]. Gender discrimination and bias has been researched in educational materials, especially in text books, of schools around the world. Studies showed that gender bias is a prevailing aspect of these text books [40,41,42,43,44,45].

Zakka [40] examined text books of Mathematics taught at a primary level in Nigeria, employing content analysis, and concluded that women are underrepresented by approximately one to seven. Stockdale [45], after analyzing English as a Foreign Language text books, reported that even text books taught for complete oral communication courses are biased towards males. A recent work conducted in Afghanistan [42] investigating English text books concluded that males are over-represented, which promotes gender inequality and which was also shown in the study conducted in Iran [44]. Hamdan [43] also reported culturally prevalent gender bias in English text books taught at public schools in Jordan.

Investigations on various facets of gender portrayal in text books refer to visibility, which is the presence of female and male characters in the books [46]. It is considered one of the vital aspects of gender research, according to Stockdale [45]. Porreca [47] employed the term ’omission,’ referring to less representation of one gender in comparison to other. Moreover, the unequal gender representation in text books reinforces that the females are shown weaker than men [7]

In this paper, teachers and learners’ perceptions regarding gender bias in the text books of Khyber Pakhtunkwa (KPK), Pakistan, are investigated and modelled with AI and, also, an analysis of text books taught at intermediate level is carried out to identify the gender portrayal in these text books.

The culture of KPK is relatively more conventional and conservative in comparison to other regions of Pakistan. It is considered, in KPK, that the male members of the society are trendsetters and decision-makers due to the culture and the ideology of the Pakhtunwali. It is interesting to examine the extent to which such practices are reflected in the text books and perceived by teachers and learners and then modelling these with AI so that predictions regarding gender perceptions of teachers and students can be made.

3. Materials and Methods

This section presents methodology used to use AI to model the gender-biased perceptions of the respondents (teachers and learners) as perceived by them for the English text books.

3.1. Methodology

Figure 1 presents the block diagram used to implement AI for the prediction of gender bias.

3.2. Data Collection

The data for the present study includes two English text books taught at intermediate level. For the perception of teachers and learners, responses from 470 teachers and learners from twenty randomly selected colleges were collected. First of all, gender portrayal was investigated in English text books taught at the intermediate level, and then the perceptions of students and teachers were analysed.

3.3. Data Description

A Likert-scale questionnaire was adapted from Refs. [7,48] to collect data from the students and teachers. Permission was taken from college authorities and informed consent was taken from the participants. The researcher herself distributed 1000 questionnaires to twenty randomly selected colleges. Out of 1000, 680 were returned. The questionnaires were analysed and questionnaires with missing information or partially completed were discarded. Out of 750, 470 filled questionnaires representing an equal number of male and female students and teachers were taken and analysed for the present study.

Table 1. Themes and topics concerning in the study’s questionnaire for AI-based predictions.

Sr.	Themes of Inquiry	Description
1.	Emphasis on male character	Representation of more male characters and as higher being in the text books
2.	Females in traditional roles	Presentation of females in traditional roles in society
3.	Male preference	Preference of males over females in society
4.	Societal beliefs	Reinforcement of the societal belief that women are not entitled to social privileges
5.	Equal participation	Men participating equally to females in society

provides the themes upon which the questions are focused in the questionnaire. All questions focused on the respondents perceptions of gender-bias embedding in the text books.

Table 2. Respondents statistics.

Respondent Group	Number of Respondents	Gender		Age
		Male	Female	(In Years)
Teachers	10	5	5	25–60
Students	460	230	230	16–19

provides the statistics about the respondents, i.e., number of teachers, learners, gender and the age group.

In total, 19 different questions were asked to each respondent. The respondents were asked to register their answers to each question in the discrete scale from 1 to 5. After each respondent had responded to all 19 questions, the data of the each respondent was labeled by experts as agreeing or disagreeing with the existence of the gender biases in the text books through interviews and discussions with each respondent.

3.4. Data Labeling and Train–Test Split

The dataset was collected from 470 respondents, each answered 19 different questions. The responses of each respondent were analysed by three experts in order to classify the respondents, based on their responses, into two classes, i.e, gender bias—True or False.

The data set was randomly split into two disjoint sets: train set and test set. The train set was used for the training of AI/machine-learning (ML) techniques whereas the test set was used for the testing of the AI/ML techniques. The test set is helpful in analysing the generalization error of the ML techniques, i.e., how the machine responds to data that is not seen/learnt during the training phase. In this paper, 20% of the data was used for testing purpose and remaining 80% to train the ML techniques.

Additionally, K-fold validation with K = 10 was used for parameter tuning as well as for the regularization of the ML techniques. The K-fold validation was applied on the train set only. All the techniques were implemented in Python, and the hyperparameter tuning for these techniques was performed by a random search process using the RandomizedSearchCV function of the scikit-learn library of Python. To obtain the results for different themes listed in Table 1, only the labels along with the responses collected against the themes’ questions were used for the training, validation and testing.

3.5. AI/ML Training and Evaluation

AI/ML techniques used in this paper are support vector machine (SVM), logistic regression (LR), decision trees (DT), random forest (RF) and artificial neural networks (ANN). Each technique was trained on train set only and then evaluated on test set. The performance of each technique was compared with the others using evaluation metrics such as accuracy, precision, recall and f1 scores. All these techniques were implemented using AI/ML libraries of the Python. For SVM, LR, DT and RF, the scikit-learn Python library was used. Hyperparameter tuning for these techniques was performed by a random search process using RandomizedSearchCV function of the scikit-learn. In case of ANN, tensorflow library was used. A brief description of each of the AI/ML techniques is given below:

3.5.1. Support Vector Machine

SVM is a supervised learning technique and a binary classifier [49]. SVM maps training instances to higher dimensions in a manner to maximize margin or the width of the gap between the classes. It constructs a hyperplane(s) in a high-dimensional space which maximizes the margin. A large margin means a lower generalization error and better performance. SVM fits well on a complex and non-linear, but medium-size, dataset. In this paper, SVM was trained using two different kernels, ie., linear and radial basis function (RBF). The parameter tuning for these kernels as well as regularization for controlling overfitting was implemented. The hyperparameter tuning for SVM-Linear was carried out on penalty (L1 or L2), loss function (hinge or squared_hinge) and soft margin (C: 0.1 to 5) with the help of RandomizedSearchCV. In case of SVM-RBF, the parameters such as soft margin (C: 0.1 to 5) and RBF Gamma (auot or scale) were also tuned with the help of RandomizedSearchCV.

3.5.2. Logistic Regression

Logistic regression is a supervised learning technique capable of classifying two or more classes [50]. It uses a logistic function to model the output-dependent variable. The logistic model has a dependent variable with two or more possible values and combines one or more independent variables to predict the value of the dependent output variable. For hyperparameter tuning, the parameters such as penalty (L1 or L2) and inverse of regularization strength (C: 0.1 to 5) were tuned with the help of RandomizedSearchCV.

3.5.3. Decision Trees

Decision tree (DT) is a versatile machine-learning technique used for binary and multi-label classification tasks [51]. It is based on applying thresholds to the feature values for decision making. The feature and its threshold for the split is calculated with the help of gini, entropy, or log_loss impurity scores. DT performs a series of decisions on the observations starting from the root node to the leaf nodes in order to identify the target class for the observation. DT is known for its intelligibility and simplicity and easy understanding. The hyperparameters’ tuning for DT is performed on parameters such as split criterion (gini, entropy, or log_loss), the minimum number of samples (2 to 5) required to split an internal node and the minimum number of samples (2 to 5) required to be at a leaf node with the help of RandomizedSearchCV.

3.5.4. Random Forest

Random forest (RF) is an extension of decision trees. It is a powerful machine-learning technique based on the ensemble learning technique [52]. Compared to DT, RF deploys multiple decision trees for reaching a prediction decision. It can be used for binary as well multi-label classification tasks. It generalizes well on complex and nonlinear data sets and is easy to train. DT and RF are considered less complex than SVM. In this paper, the number of decision trees denoted as estimators used for RF training varies from 3 to 15. Additionally, the hyperparameters, such as the function to measure the quality of a split (gini, entropy or log_loss), the minimum number of samples (2 to 5) required to split an internal node, the minimum number of samples (2 to 5) required to be at a leaf node and the percentage of samples drawn randomly (40% to 80%) for each estimator, were identified through RandomizedSearchCV. RF combines the output of multiple decision trees to reach a single decision, which means that the aggregation of decision of the multiple estimators, in an ensemble way, tends to give better results compared to a single decision tree.

3.5.5. Artificial Neural Network

Artificial neural network is a supervised learning technique inspired by the working of biological neural networks [53]. It consists of connected units or nodes called artificial neurons, which receive inputs and multiply them with certain weights to produce output. The output is then fed into another neuron. This forms a network of neurons through connections. The training of the ANN is carried out by specifying the learning rate, number of hidden layers, number of neurons per layer and the batch size. ANN normally fits well on different types of linear/non-linear and complex datasets compared to the other above techniques. ANN is considered to be a data-hungry algorithm and produces low generalization errors in big data. In this paper, the number of layers varies from 1 to 4 with the number of neurons per layer varying from 1 to 128 neurons and Relu activation function was used for the neurons. The output layers contained single neuron due to binary classification with a Sigmoid activation function.

3.6. Performance Metrics

The performance of each AI/ML technique was compared with others using Accuracy, Precision, Recall and F1 scores. These scores were calculated as follows:

$$\begin{array}{ccc}\hfill \mathit{Accuracy}& =& \frac{TP+TN}{TP+TN+FP+FN}\hfill \end{array}$$

(1)

$$\begin{array}{ccc}\hfill Precision& =& \frac{TP}{TP+FP}\hfill \end{array}$$

(2)

$$\begin{array}{ccc}\hfill Recall& =& \frac{TP}{TP+FN}\hfill \end{array}$$

(3)

$$\begin{array}{ccc}\hfill F1& =& \frac{2\times Precision\times Recall}{Precision+Recall}\hfill \end{array}$$

(4)

where $TP+TN+FP$ and $FN$ represent true positive, true negative, false positive and false negative, respectively.

4. Experimental Results

This section presents an analysis of the two text books used in this paper, along with the experimental results. The experimental results are presented theme-wise. In the end, all the themes are combined to compare the performance of ML for the prediction of gender bias, with that of individual themes.

4.1. Analysis of Text Books

An analysis of the two text books taught at college level was carried out to identify the portrayal of gender in these books. The two books are taught at first and second year in all the Government colleges of the province. Various aspects of gender portrayal in the text books were analysed by researchers. For the present study, an integrated model based on pioneering research by [47] and further enhanced by Hasan [54] was used to identify gender portrayal in the text books. The following categories were identified after content analysis of the two books based on the integrated model:

• Presence of male and female characters in the books. For this category, all the books were thoroughly read. The number of male and female characters were counted. Their frequencies and percentages are presented in a table.
• Allocation of different roles in the books. For this category, the frequency of main, supporting and minor characters was analyzed.
• Use of nouns for males and female characters. For this category, an analysis of nouns used for male and female characters was carried out.

The number of male and female characters is shown in

Table 3. Number of male and female characters in two text books.

Books	No. of Female Characters	Percentage of Female Characters	No. of Male Characters	Percentage of Male Characters
Book 1	16	25	47	75
Book 2	10	14	61	86
Total	26	20	108	80

. In the first book, three quarters of the characters are male as compared to one quarter being female characters. The number of female characters in the second book is even lower than the first book, which makes a total of eighty percent for males and merely fourteen percent for females. The character representation of the two genders in the designated books tends to endorse the fact that males preponderate in these settings. In Book 1, about three quarters and in Book 2 more than three quarters of the mentions include men and their male counterparts. This persistent occurrance is inclined towards the establishment of a dominant–recessive relationship, in which one gender (often male) is conveniently used to create an inconspicuous effect on the other gender (usually female).

In the second category, the distribution of characters is also biased towards females.

Table 4. Distribution of characters in Book 1.

Roles	Number of Females in Book 1	Percentage of Females in Book 1	No. of Males in Book 1	Percentage of Male in Book 1
Main	9	29	22	71
Supporting	2	25	6	75
Minor	7	29	17	71
Total	18	29	45	71

and

Table 5. Distribution of characters in Book 2.

Roles	Number of Females in Book 2	Percentage of Females in Book 2	No. of Males in Book 2	Percentage of Male in Book 2
Main	4	20	16	80
Supporting	0	0	3	100
Minor	5	31	11	69
Total	9	23	30	77

shows the distribution of different characters in Book 1 and Book 2, respectively.

The characters were also evaluated in light of the different parts played during their appearance, which still relates to the point that the reader is encouraged to develop a male-dominated image of whatever text put forward. According to the data collected from Book 1, the leads as well as the supporting persons are males 70% of the time while female depiction is scarce and usually undermined.

In Book 2, on the other hand, the gender bias further worsens, as about 90% of the leading and supporting characters are males while women contribute an insignificant 10%. In the minor roles, however, women have a 30 percent share and, hence, they often remain hidden from the eyes of the audience.

In the last category of nouns and adjectives, the number of males overshadows females. The data is shown in

Table 6. Use of nouns and adjectives in the books.

Nouns	Nouns for Females	Frequency of the Nouns for Females	Nouns for Males	Frequency of the Noun for Males
Book 1	Woman	00	Man	37
	Wife	04	Husband	00
	Mother	14	Father	50
	Grandmother	06	Grandfather	27
	Sister	11	Brother	16
	Daughter	01	Son	02
Book 2	Woman	34	Man	45
	Wife	22	Husband	28
	Mother	05	Father	36
	Grandmother	00	Grandfather	00
	Sister	31	Brother	04
	Daughter	42	Son	03

. An analysis of the occupational roles of the male and female figures rendered the difference in gender portrayals extensive. In Book 1, it was observed that women are less likely to be represented with no role associated with her. On the other hand, men are depicted in abundance in whatever role they are cast in, save for the role of a husband. They outweigh the women in numbers and appearances, confirming a blatant gender discrimination. Lastly, the gender representation in Book 2 describes men and women as individuals as well as suited to their filial occupations. Though husband and father are glorified as compared to the role of a wife and mother, it can be observed from the stated data that sisterly characterization outweighs the representation of brothers. In addition, the role of the daughter was also more acknowledged and exalted as compared to that of the son.

4.2. Emphasis on Male Characters

This theme pertains to the representation and emphasis on male characters in the text books along with considering male as a higher being than woman. There were a total of four questions which were asked to all the 470 respondents. These questions focus on the portrayal of more male characters in the text books and also their portrayal in better roles than women.

Table 7. Results for gender-bias prediction under male characters theme.

Method	Accuracy	Precision	Recall	F1 Score
SVM (Linear)	0.415	0.498	0.499	0.498
SVM (RBF)	0.585	0.568	0.571	0.569
Logistic regression	0.426	0.47	0.473	0.471
Decision tree	0.638	0.624	0.631	0.627
Random forest (estimators = 3)	0.585	0.558	0.559	0.558
Random forest (estimators = 5)	0.649	0.626	0.627	0.626
Random forest (estimators = 8)	0.574	0.569	0.574	0.571
Random forest (estimators = 10)	0.628	0.62	0.628	0.624
Random forest (estimators = 15)	0.606	0.594	0.599	0.596
ANN (layers = 4, neurons = 128)	0.606	0.594	0.599	0.596

shows the experimental results. The results presented are based on accuracy, precision, recall and F1 scores. The results show that respondents have mixed perception regarding whether the portrayal of male characters in the books leads to gender bias. Such mixed perceptions results in ML techniques having difficulty making correct predictions, as indicated by the low prediction accuracies in the range from 41% to 64% achieved by the ML techniques. It can be seen that SVM with linear kernel attains an accuracy of 41.5% whereas with RBF kernel it achieves an accuracy of 58.5%. In case of DT and LR, the accuracy is 42.6% and 63.8%, respectively. In case of RF, the number of estimators is important, as it can be seen that with estimators equal to 5, the RF achieves the highest accuracy of 64.9% compared to 3, 8, 10, and 15 estimators. The RF also achieves better results than ANN (layers = 4, neurons = 128), which demonstrates an accuracy of 60.8%. DT achieves better results than RF in this theme. RF combines the output of multiple decision trees to reach a single decision, which means that the aggregation of the results of multiple estimators tends to decrease the performance of RF.

The performance of ANN also varies with respect to the number of layers and number of neurons, as shown in Figure 2. The figure shows the performance of different ANN architectures (layers, L; neuron, N) to predict the gender-biased perceptions of the respondents. It can be observed that ANN 4-128 demonstrates the best results compared to all the other ANN architectures. The gender representation manifested in the text books is the most crucial facet of sexism, as is established in [7,47,55,56]. However, the modelling of the perceptions of respondents using ML when considering increased male representation is not established as a matter of concern for gender biases in the results.

4.3. Females in Traditional Roles

Under this theme, the respondents were asked about the portrayal of females in traditional roles in society, which may lead to gender bias, in the text books. The experimental results under this theme are shown in the

Table 8. Results for gender-bias prediction under females-in-traditional-roles theme.

Method	Accuracy	Precision	Recall	F1 Score
SVM (Linear)	0.723	0.723	0.739	0.731
SVM (RBF)	0.745	0.728	0.733	0.73
Logestic regression	0.755	0.748	0.764	0.756
Decision tree	0.777	0.765	0.746	0.755
Random forest (estimators = 3)	0.745	0.727	0.721	0.724
Random forest (estimators = 5)	0.734	0.72	0.73	0.725
Random forest (estimators = 8)	0.745	0.728	0.733	0.73
Random forest (estimators = 10)	0.777	0.762	0.77	0.766
Random forest (estimators = 15)	0.766	0.754	0.767	0.76
ANN (layers = 1, neurons = 16)	0.787	0.782	0.801	0.791

. It can be seen that the accuracy of ML techniques vary from 71% to 78%. This is an indication that the respondents agree with the perception that females in traditional roles lead to gender bias and is depicted by the results of the ML techniques, which show good accuracy. ANN 1-16 obtains the best accuracy, i.e, 78.7%, compared to all the other techniques under this theme. In Figure 3, the performances of different ANN architectures are shown. It can be observed that ANN 1-16 provide the best results. RF performed slightly better than DT. As RF combines the output of multiple decision trees to reach a single decision, the aggregation of the results of multiple estimators tends to improve the performance of RF under this theme.

From Table 7 and Table 8, it can be established that there are no fixed architectures/configurations of ANN that will always perform well under different themes. For instance, ANN 1-16 obtains the best results under the females-in-traditional-roles theme but under the Men-emphasis theme, the best accuracy is demonstrated by ANN 4-128. The same argument is also true for RF, whose performance varies with respect to the number of estimators. The results show that the performance of RF decreases as the number of estimators either decrease or increase to around 10.

It is also established in the literature that male characters are not shown as involved in limited or domestic activities as frequently as females [57], who are assigned the more traditional roles. Similar findings were established in Ref. [56] with regards to the examination of Khyber Pakhtunkhwa’s English text books.

4.4. Male Preference

Under this theme, the preference of males in society was investigated using four different questions. The experimental results for gender-bias predictions are shown in the

Table 9. Results for gender-biased prediction under male-preference theme.

Method	Accuracy	Precision	Recall	F1 Score
SVM (Linear)	0.457	0.526	0.521	0.523
SVM (RBF)	0.628	0.62	0.628	0.624
Logestic regression	0.606	0.615	0.623	0.619
Decision tree	0.681	0.667	0.676	0.671
Random forest (estimators = 3)	0.66	0.646	0.653	0.649
Random forest (estimators = 5)	0.681	0.657	0.653	0.655
Random forest (estimators = 8)	0.723	0.71	0.675	0.692
Random forest (estimators = 10)	0.649	0.626	0.627	0.626
Random forest (estimators = 15)	0.691	0.671	0.673	0.672
ANN (layers = 2, neurons = 64)	0.702	0.681	0.681	0.681

. It can be seen that random forest (Estimators = 8) achieves the best accuracy of 72.3%. From this, it can be concluded that respondents have a mixed view about male preference which leads to the existence of gender bias in the text book. ANN (layers = 2, neurons = 64) achieves accuracy of 70.2%, whereas the performance of other ANN architectures are shown in Figure 4 for performance comparison.

4.5. Societal Beliefs

Under this theme, four questions were asked pertaining to the reinforcement of the societal belief that women are not entitled to social privileges. The performance of the ML techniques under this theme is shown in the

Table 10. Results for gender-bias prediction under societal-belief theme.

Method	Accuracy	Precision	Recall	F1 Score
SVM (Linear)	0.766	0.764	0.721	0.742
SVM (RBF)	0.713	0.698	0.707	0.702
Logestic regression	0.723	0.703	0.698	0.7
Decision tree	0.638	0.628	0.636	0.632
Random forest (estimators = 3)	0.67	0.668	0.679	0.673
Random forest (estimators = 5)	0.617	0.623	0.631	0.627
Random forest (estimators = 8)	0.681	0.676	0.688	0.682
Random forest (estimators = 10)	0.67	0.668	0.679	0.673
Random forest (estimators = 15)	0.649	0.646	0.656	0.651
ANN (layers = 2, neurons = 8)	0.755	0.742	0.753	0.747

. It can be seen that the best accuracy score was achieved by SVM linear with an accuracy of 76.6%. From such results, it can be asserted that there exists a gender bias. The performance of different ANN architectures are shown in Figure 5. As reported in [47,56,57], the representation of women and men has not changed in the last thirty five years and, especially in the Pakistani context, the ones who suffer poor representation and preference are women, which is also depicted in the results.

4.6. Equal Participation

Under this theme, three questions were asked pertaining to equal participation of both genders in society. Women are generally presented in the text books as being busy in jobs such as child caring and nursing [58]. This is also affirmed in [59], that women are found in the roles of teachers and nurses while men are shown in situations needing problem-solving skills. The prediction results under this theme are shown in

Table 11. Results for Gender prediction under Equal Participation theme.

Method	Accuracy	Precision	Recall	F1 Score
SVM (Linear)	0.574	0.592	0.597	0.594
SVM (RBF)	0.67	0.651	0.656	0.653
Logistic regression	0.649	0.633	0.639	0.636
Decision tree	0.638	0.607	0.601	0.604
Random forest (estimators = 3)	0.638	0.624	0.631	0.627
Random forest (estimators = 5)	0.638	0.613	0.613	0.613
Random forest (estimators = 8)	0.628	0.607	0.61	0.608
Random forest (estimators = 10)	0.67	0.648	0.65	0.649
Random forest (estimators = 15)	0.66	0.646	0.653	0.649
ANN (layers = 2, neurons = 64)	0.67	0.646	0.644	0.645

and Figure 6.

4.7. All Items

In this section, all the themes’ questions are used to check the performance of ML for predicting gender bias. The experimental results are shown in

Table 12. Results for gender prediction using all items.

Method	Accuracy	Precision	Recall	F1 Score
SVM (Linear)	0.777	0.761	0.758	0.759
SVM (RBF)	0.809	0.795	0.807	0.801
Logestic regression	0.745	0.728	0.733	0.73
Decision tree	0.787	0.773	0.767	0.77
Random forest (estimators = 3)	0.766	0.75	0.744	0.747
Random forest (estimators = 5)	0.798	0.784	0.787	0.785
Random forest (estimators = 8)	0.83	0.829	0.8	0.814
Random forest (estimators = 10)	0.787	0.779	0.755	0.767
Random forest (estimators = 15)	0.84	0.83	0.826	0.828
ANN (layers = 4, neurons = 32)	0.872	0.867	0.858	0.862

and Figure 7. It can be seen that better prediction results are obtained compared to individual themes. The best accuracy is demonstrated by the ANN (Layers = 4, Neurons = 32), i.e, 87.2%, followed by RF based on 15 estimators. RF combines the output of multiple decision trees to reach a single decision, which means that the aggregation of the results of multiple estimators, i.e, 15, tends to improve the performance of RF.

From these results, it can be concluded that the gender bias as perceived by teachers and learners can be modeled with the help of ML techniques, which can be deployed to predict bias with high accuracy on future data.

5. Conclusions

This paper employed machine learning to model gender-biased perceptions of teachers and learners regarding English text books. The data used was gathered through a questionnaire based on five different themes. The respondents were teachers and intermediate-level learners. Five different machine-learning techniques were explored. Their performances were then compared with the help of accuracy, precision, recall and f1 scores. The experimental results show that machine-learning techniques achieve low accuracy under themes such as more representation of men and showing men in higher roles. This is due to the mixed perceptions of the respondents. However, in the case of the other themes, the responses of the respondents are well-divided, which leads to good performance. The best results achieved are under females in traditional roles, where ANN achieves 78% accuracy. In the societal-belief theme, SVM demonstrates the best accuracy, of 76.6%. The experimental results show that even better results are obtained when all themes are combined. In such a case, ANN demonstrates an accuracy of 87.2%, followed by random forest, which achieves an accuracy of 84%. This shows that artificial intelligence can be used efficiently to model gender-biased perceptions if data is collected from teachers and learners through questionnaires about text books. Additionally, the analysis of textbooks also confirmed gender discrimination which called for the opinions of learners and teachers regarding this issue.