Assessing Argumentative Representation with Bayesian Network Models in Debatable Social Issues

This study seeks to obtain argumentation models, which represent argumentative processes and an assessment structure in secondary school debatable issues in the social sciences. The argumentation model was developed based on mixed methods, a combination of both theory-driven and data-driven methods. The coding system provided a combing point by which the theoretical framework and data integrated into the model. Bayesian networks were used to assess and update student progress. The study examined how to explore argumentation knowledge structure and argumentative skills as a whole via data analysis and theoretical considerations. Effective argumentative representation is a crucial step to bridge argumentative learning and assessment. variable. For student one, the Perspectives was 52.44%, Stand Point was 50.61%, Position and Claim was 53.45% and Argumentative Representation was 51.79%. We can examine the student’s performance at both general level at Argumentation Representation and sub-level at such as Perspective, Stand Point, and Position and Claim.


Introduction
Argumentation as a cognitive theme has caused recent attention in education (Kuhn & Udell, 2003, Lu & Zhang, 2013a. Researchers and educators have applied the elements of argumentation to cognition and learning in adolescents and young adults (Brem & Rips, 2000;Lu & Zhang, 2013b). These students should develop such knowledge and skills formally. However, serious weaknesses have been observed in argumentative learning in such age groups (Kuhn, 1991(Kuhn, , 2001. From a cognitivist perspective, argumentation includes argumentative structure and cognitive skills (Alishahi & Stevenson, 2008). Argumentative structures can be seen in the formats of argumentative representations (Dung, 1995) and models (Erduran, Simon, & Osborne, 2004). Argumentative cognitive skills consist of thinking, reasoning and discourse skills (Kuhn & Udell, 2003;Lu & Zhang, 2012;Shaw & Felton, 1997).
This study examined argumentation learning of secondary school debatable issues in social sciences. We found that effective representation was critical to form and facilitate argumentative knowledge structure and cognitive skills. Based on such a representation structure, we established an argumentative learning model to describe students' progress and to examine their cognitive errors. The researchers used Bayesian network model to differentiate students' competences in argumentative processes and to describe distributions of different argumentative cognitive skills. This Bayesian model is a dynamic, multi-functional assessment tool to show student progress.

Argumentative Structures and Models
Different models describe argumentative structures. Toulmin's (2003) argumentative model provides a theoretical framework for analyzing argumentative structure. This model asserts that most arguments consist of five elements: claims, grounds, warrant, backings, and rebuttals. In fact, the core parts of the model consist of grounds, claims, and warrant. The grounds provide evidence for the claims. A warrant links the data to a claim, legitimizing the claim by showing that data to be relevant; backings and rebuttals are two extra supports from positive and negative aspects. Toulmin's model aims at creating a stronger epistemological and empirical logic that takes into account all sorts of topics in both everyday life and academic research. Eemeren questioned: "Is there one universal system of norms, by which all sorts of arguments in all sorts of fields must be judged, or must each sort of argument be judged according to its own norms (p. 1)?" In argumentative applications to the sciences and social sciences in recent decades, limitations distinguish categories of grounds (data), warrants and backings and further leave the ambiguity of argument structures (Erduran, Simon, & Osborne, 2004;Kelly, Druker, & Chen, 1998). With the advancement of cognitive theories and educational technologies, there are increasing demands for alternative argumentative models. However, the basic ideas and the components of Toulmin's argumentative model are still applicable in the scientific fields and academic research.

Argumentative Skills and Discourse
Argumentative skills are developed from critical thinking skills. Kuhn and Udell (2003) suggest that argumentative skills are mainly expressed in reasoning skills and also include discourse skills. Educators would likely evaluate the quality of argumentations to determine student proficiency in advancing, critiquing, and defending claims in reasoned discussion. These argumentative skills are developed in collaboration with others. Also, discourse skills as a separate dimension contribute to argumentative development in claims with a great deal of data evidence during peer conversation (Kuhn, 2005).
Argumentative skills enable students to deal with debatable issues with appropriate claims, coherent reasoning, sound evidence, and well-elaborated concepts (Kuhn, 1991). Studies indicate that argumentative skills have not been sufficiently developed in secondary schools (Kuhn, 1991) and that students have difficulty grasping the gist of argumentative structure and skills from argumentative discourse (Reznitskaya et al., 2001;Stein & Bernas, 1999).

Argumentative Tasks and Learning Trajectories
Argumentative knowledge and skills can be acquired and developed when learners accomplish a set of cognitive tasks following a set of rules. Thus, cognitive task analysis and argumentative representation become crucial to exam students' argumentative proficiency in advancing, critiquing, and defending claims (Kuhn & Udell, 2003).
From the perspective of cognitive tasks analysis, argumentation is a process to go about a set of cognitive tasks, which contain argumentative goals. Most cognitive tasks are ill-structured and selected from authentic scientific and social issues. In other words, the cognitive tasks are not developed from a strict cognitive task design. Instead, a learner's trajectory can be acquired by argumentative representations based on the cognitive task analysis (Lajoie, 2003;Zhang & Lu, 2014). Effective argumentative representations are crucial to describe argumentative knowledge structures and skills towards proficiency advancing, critiquing, and defending claims in an arguing process. Thus, researchers can use cognitive task analysis techniques to develop variables from cognitive tasks and Bayesian networks can strengthen argumentative representation.

Research Questions
The purposes of the study were to explore an alternative method in students' learning assessment based on their interactions focusing on a social issue theme. Bayesian network processes represented students' learning trajectories and performance results on both evidence variables and explanatory variables. The following research questions were addressed: 1) How is the argumentative representation model used to represent students' learning processes in debatable social issues?
2) How is the Bayesian network used to provide dynamic assessment information?
3) How are the students' conceptual understanding and perspectives of social debatable issues assessed diagnostically?

Research Methods
This study investigated debatable issues in social sciences, "a concerned group for women immigrating to an urban area," as cognitive tasks to examine students' formation of argumentative knowledge representation and development of argumentative skills.
The following issue is an example: We are the "concerned group for women immigrating to an urban area", and we are not accepted appointments on birth delivery services from "non-local pregnant women." As a result of this arrangement, we were shut out from the services of public hospitals; and this deprived our right to give birth to their children… Based on the issue description, students were asked to present their opinions and suggestions to solve this problem.

Participants
Fifty-two eighth grade students participated in this study (ages 13 to 14). The school was chosen as it participated in a university-school partnership project involving the use of an online platform to teach liberal arts studies. The students were in two classes; 30 girls and 22 boys participated. The curriculum of the liberal arts studies consisted of modules that focus on specific issues in three major areas: (a) personal and self-development, (b) society and culture, and (c) science, technology and environment. Students studied a variety of topics such as interpersonal relationships, the city today, and public health.

Cognitive Tasks about Social Debatable Issues
The cognitive tasks consist of three independent paragraphs regarding information about "women immigrating to an urban area." Basically, the women in the scenario were from Mainland China and they wanted to deliver their babies in Hong Kong and they did not want to pay for the delivery. In this scenario local women (from Hong Kong) could deliver their babies in hospitals without delivery costs because they were considered local. The issue involved several social issues, such as politics, poverty, and discrimination. The cognitive task structure in this scenario consisted of two parts: issue description and essay questions.
The first item of information was about the equality of charging delivery fees to pregnant women who immigrated to the special urban area in Hong Kong. The concerned groups request the government to treat them equally. For instance, the pregnant members of the concerned group request the government to allow them to deliver their babies in a local hospital without charging delivery fees.
The second item of information is about an individual case that describes a woman of the concerned group who was charged the delivery fee. Three judges of the Court of Appeal then unanimously ruled that it was legitimate for the Hospital Authority (HA) to charge delivery fees to non-local pregnant women.
The third item of information reflects a voice from a local pregnant woman. She said that the delivery room was overbooked because the concerned group of women who immigrated to the special urban area.
Following these paragraphs of information, three questions have been posed to the students: a. By combining information 1, 2, and 3, and your background knowledge, try to identify the different stakeholders and their perspectives/comments they would have regarding the women from Mainland China immigrating to the special urban area." b. Do you agree with the views of the pregnant woman in information 2? c. How does the case mentioned in information 2 reflect the spirit of the "rule of law" in the special urban area?

Argumentative Task Analysis
An argumentative tasks analysis is a cognitive task analysis (CTA) in argumentation. Clark and Estes (1996) suggest that methods can be considered from two dimensions: knowledge elicitation and data analysis. In this study, we took a CTA-grounded theory pair. From the knowledge elicitation dimension, we identified aspects of representation based on the content knowledge, argumentative knowledge, and skills. From the data analysis dimension we used grounded theory to focus on basic argumentative components. We used three steps in the coding process: open coding, selective coding and theoretical coding (Clark, Feldon, van Merrienboer, Yates, & Early, 2008;Clark, Feldon, & Yates, 2011;Zhang & Lu, 2014). Based on the analysis a coding system has been developed (see Appendix 3).

Argumentative Representations
Argumentative models can be represented in Bayesian networks (Koski & Noble, 2009;Lynch, 2010). The argumentative model is presented in Figure 1. There are two kinds of variables in the model: evidence variables and explanatory variables. The evidence variable can be used to collect evidence directly from the student completion of current debatable issues; the explanatory variable is a latent variable that is unobservable. In total, there are four explanatory variables and 11 evidence variables in this study. The evidence variables include: conceptual clarification, identifying stakeholders (in all perspectives), multiple perspectives, stakeholder (identified in single perspective), state of standard point, confirmation of standard point, matching, apply information from case, apply outside information, explanation, and conclusion. The explanatory variables are argumentative representation, perspectives, standard point, and position and claim.

Simulated Evidence
A random sampling method was applied to test evidence states, from one true evidence to 10 true evidence observations in the simulation. The true status meant the variable acquired a positive value based on each student's completion of one piece of argumentation knowledge or skill. In order to reduce the tedious process of the test, the random process started from a zero mastery model, which meant that the 11 evidence variables did not present any positive values. The student had zero evidence of these variables. At the next step, the simulated data of the evidence variables demonstrated there are one, two, and three evidence until a complete model, which meant the student had all 11-evidence variables. That meant the student scored 78.55% in argumentative representation in the argumentative Bayesian model. The argumentative progress status is listed in Table 1.

Student Evidence
For illustration purposes, we only choose six students from the 52 students to discuss here. Students' argumentation scores on the debatable social issue can be seen in Table 2. The full argumentation scores from 52 students can be seen in Appendix 1. Table 2 shows that scores are the records of students' argumentative completion across 11 evidence variables. For example, student one scored the highest score on Multiple Perspectives (with 4 points) and the lowest score on Identifying Stakeholders (with 0 point). Each student has a unique combination of all 11 evidence variable values, which consists of a vector. For example, for student one the vector is V s1 = (1, 0, 4, 3, 2, 1, 2, 2, 0, 0, 0). All of these evidence variables are used to provide evidence for updating these explanatory variables.

Students' Argumentation Scores in Explanatory Variables
Students' argumentation scores on 11 evidence variables indicated their proficiency in the argumentative process. A student's argumentation proficiency was represented by four explanatory variables: argumentative representation, perspectives, standard point, and position and claim. The argumentative representation measured a general argumentative proficiency. The completing levels of other three explanatory variables were directly related to the general argumentation representation.
As soon as we entered values of evidence variables based on the data in Table 2 into the Bayesian network, it was updated (see Figure 2). When we examined the output of the Bayesian network model (Jouffe & Conrady, 2011), we can see the updated probabilistic value of each evidence variable. The output can be seen in Figure 3, which reports the result of student one's performance. When the student one's evidence variable vector, V s1 = (1, 0, 4, 3, 2, 1, 2, 2, 0, 0, 0), was entered into the Bayesian network we received the updated value of each explanatory variable. For student one, the Perspectives was 52.44%, Stand Point was 50.61%, Position and Claim was 53.45% and Argumentative Representation was 51.79%. We can examine the student's performance at both general level at Argumentation Representation and sub-level at such as Perspective, Stand Point, and Position and Claim.   Table 3 we see there are different argumentative proficiency levels. We examined Table 3 in two dimensions. One dimension is the row dimension, which reflects students' scores across all argumentative explanatory variables. From student 3, for example, the explanatory variable values report the learner's different levels of the argumentation proficiencies. Although the Argumentative Representation is 0.8227, the perspective indicates 0.2718, which is very low. This provides a diagnostic assessment information for each student's progress in argumentation learning.
The column dimension reflects a distribution of explanatory variables. For example, when we examine the perspectives column, we can see that student 16 receives the lowest score in Perspective at 0.0620, and student 33 receives highest score at 0.9274. The lowest Stand Point score was from student 38 at 0.2055 and the highest Stand Point score was from student 36 at 0.8562. The lowest Position and Claim score was from student 39 at 0.0165 and the highest was from student 4 at 0.9691. Last, the lowest Argumentative Representation score was from student 50 at 0.0939 and the highest one was from student 36 at 0.9705.

Students' Argumentative Representation and Assessment
We examined students' argumentative representation and relevant perspectives based on their scores in Table 3.
We developed classification rules in terms of norm-based assessment. In other words, we assessed students' argumentative scores based on the 52 students' performance. We linearly divided students' score based on the range of each explanatory variable into 3 equal distance (Zhang & Lu, 2014). Thus, we received three score levels of each explanatory variable and we coded them from lowest, medium to highest three levels. That means: 1 = lower level, 2 = medium level, and 3 = higher level. For example, we use Perspectives to illustrate this process. The range of these variables is 0.8654. We can get the equivalent distance 0.2885 (cut-off), which is from 0.8654 divided by 3. Based on this cut-off we classified all students into three levels for Perspective. The similar computing is applied to other three explanatory variables and we received three levels of these four explanatory variables. The results can be seen in Appendix 2.

Discussion
This study explores argumentative development in a secondary school. It examines the debatable social issues in argumentation learning that secondary students must master. An argumentative representation model has been established based on argumentative task analysis, which is further represented in a Bayesian network argumentative model to depict student progress.
We describe students' perspectives of argumentation in a series of argumentative concepts. These concepts are a mixture of argumentative knowledge and skills in both evidence and explanatory variables. Each variable is a cognitive component in the argumentative model, which represents students' proficiencies in dealing with the debatable social issues.
The Bayesian network consists of 15 variables in total. Eleven of these variables are evidence variables and four of them are explanatory variables. The explanatory variables dynamically represent students' perspectives in argumentative learning based on the evidence received and by propagating the evidence vectors to eleven-evidence variable. The authors established an effective argumentative representation, which reflected the features of the students' argumentative learning processes and results.
The hierarchical argumentative representation model has effectively identified different types of students' perspectives and beliefs in debatable social issues. The model provides moment-by-moment information differentiation for both students and argumentative representation in two dimensions. From a student-learning dimension, we can observe a student argumentation trajectory across these explanatory variables. From the argumentative representation we can examine which aspects of the argumentative learning need to be further improved. For example, in group 9, student #48 receives the score vector which was (1, 3, 2, 1). This means the student had lower scores in Perspectives and Argumentative Representation. The argumentative representation model is currently employed in debatable social issues and this model can be used in different social issues and domains.

Limitations
Fifty-two students are still not sufficient for Bayesian network to learn to become a robust network. The results are limitedly generalized to different argumentative learning settings and learning environments due to the small sample size. However, argumentative representation with Bayesian network model is very effective and can be used as an alternative assessment model in similar learning environments. The argumentative representation model can be used in individual learner assessment and for diagnostic purposes.    Vol. 7, No. 11; Refer/integrate the case

Copyrights
Copyright for this article is retained by the author(s), with first publication rights granted to the journal.
This is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).