Elsevier

Computers & Education

Volume 99, August 2016, Pages 1-13
Computers & Education

Closing the gender gap in STEM with friendly male instructors? On the effects of rapport behavior and gender of a virtual agent in an instructional interaction

https://doi.org/10.1016/j.compedu.2016.04.002Get rights and content

Highlights

  • A virtual agent most successfully supports learning when it shows rapport behavior.

  • Although participants do not perceive increased rapport, the agent displaying rapport fosters performance.

  • A virtual agent of opposing gender, not same sex, is most successful in fostering participants' performance.

Abstract

While numerous research endeavors address the effects of pedagogical agents, the role of the agent's gender and its rapport behavior has been neglected. We hypothesize that a minimal amount of behavioral realism induced by display of rapport is necessary for any social effects to occur in human-computer interaction. Further, in line with results from STEM research on female role models, we assume that especially for female learners a same sex agent will be beneficial. In a 2 (student gender) × 2 (agent gender) × 2 (rapport behavior yes/no) between subjects design, we investigate whether virtual agents can help enhance participants' performance, effort and motivation in mathematics. Female and male participants (N = 128) interacted with a male or female virtual agent that either displayed rapport or no rapport. Our results confirm the expected main effect of rapport. However, against expectations, our results do not support the assumption that a same sex agent is beneficial for female learners. Participants’ performance and effort were significantly enhanced when interacting with an agent of opposite gender that displayed rapport. Our results have implications on designing agents for education and training purposes.

Introduction

A common goal in educational endeavors is to increase interest and performance in science and math related subjects. Compared to the need for future experts in the realm, there is a lack of students in STEM fields (Science, Technology, Engineering and Mathematics) and especially women are underrepresented (Hill, Corbett, & Rose, 2010). It is therefore of high societal relevance to encourage women to enter STEM related fields. However, previous findings suggest that due to socio-cultural factors like implicit and explicit gender stereotypes, a gender difference exists in STEM fields especially with regard to motivation and willingness to enter the corresponding disciplines (Correll, 2004, Nosek et al., 2002a, Nosek et al., 2002b, Steele and Aronson, 1995). These stereotypes are mainly perceived and learned through interaction with other social beings (Eccles et al., 1998, Jacobs, 1991). Women seem to underestimate their competence in mathematics (and other STEM fields) because of the socially common belief that women have lower abilities in those fields than men (Beyer and Bowden, 1997, Bong, 1999). Apparently, educating women about the inappropriateness of those gender stereotypes is not sufficient since those beliefs can continue to stay implicitly present in their minds (Nosek et al., 2002a, Nosek et al., 2002b). According to the expectation-value-model (Eccles et al., 1983), women underestimate their own competence and have lower expectation for success and therefore tend to have a lower preference for a career in STEM fields.

Recently, there has been an upsurge of interest in educational technology which exploits social and motivational factors that enhance math performance in general, and reduces gender inequality in particular (Baylor and Ryu, 2003, Kim, 2004). Specifically, pedagogical agents have been suggested as a means to individually motivate and instruct students. These autonomous agents have been widely expected to transfer the benefits of human–human tutoring and instructional communication to the area of computer-supported learning. Future systems are anticipated to enhance e-learning programs for individual learning with a human-like motivator or to support teachers in the classroom by attending to small groups of students (Lester et al., 2000, Moreno, 2004). Advocates of embodied automatic tutors claim that one major advantage of these pedagogical agents is that they can, or will be able in the future, communicate via verbal and nonverbal means, thus facilitating and personalizing the interaction with an e-learning program (e.g., Lester et al., 2000). Furthermore, increased motivation is expected: Baylor and Ryu (2003) suggest that the key advantage is that human-likeness creates more positive learning experiences and provides a strong motivating effect (Krämer & Bente, 2010). This assumption is also in line with the phenomenon that people often treat computers as social actors (Reeves & Nass, 1996). Here, it has been demonstrated that people automatically display social behavior towards computers and virtual agents even if they are convinced that the artificial interlocutor does not warrant human-like treatment (Krämer, 2005, Nass and Moon, 2000). Therefore, it can be assumed that psychological factors which improve people's performance in traditional face-to-face instructional settings (Andersen, 1979) can be successfully simulated by technologies in form of virtual learning companions or virtual instructors – at least in the sense that people will tend to react to them in a similar way as they would towards human instructors.

One factor that has been shown to be beneficial in terms of facilitating interaction, relationship building and instructional communication is rapport. In social psychology, rapport is described as the establishment of a positive relationship among interaction partners by rapidly detecting and responding to each other's nonverbal behavior (Gratch, Wang, Gerten, Fast, & Duffy, 2007a). This includes displaying behaviors that indicate positive emotions (such as head nods and smiles), showing mutual attentiveness (such as mutual gaze) and certain coordination behaviors (such as postural mimicry and synchronized movement, Tickle-Degnen & Rosenthal, 1990). Not only is rapport beneficial in human-human-interaction but also within human-agent-interaction: Niewiadomski, Mancini, Hyniewska, and Pelachaud (2010) have shown that when an agent displays appropriate and socially adapted emotional expressions it is perceived as more human-like than an agent that shows human expressions which are inappropriate or not socially adapted. This indicates that rapport is an important feature in order for the agent to be perceived as human-like and for any social effects to occur.

In this study, our goal is to focus on the agent's social role and social effects within instructional settings. Therefore, we do not employ a virtual agent as a tutor as in most other research on pedagogical agents (see Graesser et al., 1999, Lester et al., 2000). Instead, the agent's role is to motivate by observing the learners' success and – by asking about the experiences – giving the learner the opportunity to self-reflect on one's own performance. We therefore do not primarily focus on the role of the agent in terms of guiding and instructing cognitive processes but rather address the agent's social role and the corresponding processes (Krämer & Bente, 2010). Although traditionally, research on pedagogical agents has been dominated by a cognitive perspective, recent approaches explicitly state the importance of complementing this by a social psychological framework. For instance, Kim and Baylor (2006) argue that learning environments should provide situated social interaction since it is well documented that the cognitive functioning of learners is framed by social contexts. Kim and Baylor (2006) sum up that teaching and learning are highly social activities. They conclude that the reason for the weak impact of virtual agents (compared to human tutors) concerning learning outcomes might lie in the lack of empathetic social encouragement and caring aspects.

In this paper, we seek to address two goals. First, we aim to show that employing socio-emotional cues can enhance math performance in a human-computer setting. Specifically, we aim to show that the presence of an agent which provides rapport and fosters self-reflection can improve performance on standardized math tests. By this, we seek to provide further evidence that people do treat computers as social actors and help elucidate the design principles that foster this effect. Second, we strive to contribute to the question whether the gender of the agent matters and influences the effects in interaction with the student's gender. Here, we do not strictly follow up on the research on competent role models and their potential to motivate young women to engage in STEM fields but – in order to provide a new foundation to the discussion – rather focus on the effects of the mere presence of a pedagogical agent which is either of same or different sex as the learner.

Section snippets

Theoretical background

The importance of motivating factors in instructional processes has long been demonstrated. Especially the expectation to excel in terms of self-efficacy plays a crucial role: according to the expectancy-value-model of academic motivation, an individual's engagement and performance with regard to an academic task is best predicted by the two factors expectation and value of success (Eccles, Wigfield, Harold, & Blumenfeld, 1998; Pintrich and Schrauben, 1992, Pintrich and Schunk, 1996). Marks

Method

The goal of the study was to investigate whether a virtual agent can motivate participants and help to improve their performance in a mathematical task. For this purpose, we examined whether beneficial effects occur when participants interact with virtual agents and how gender of the agent as well as of the learner and the rapport displayed by the agent influence these effects. In order to test the assumptions that a female agent will be especially beneficial for women and that rapport will

Results

We conducted all analyses using Statistical Package for the Social Sciences (SPSS, Version 18). Before starting the analyses of hypotheses, we verified that agent appearance did not affect the results. As anticipated, there were no significant differences between agents with the same gender but different appearances. Therefore, the data was collapsed for further analysis. Also, variances were not significantly different for any test (ps > 0.51), therefore the equal variances assumption holds

Discussion

Our results yield several interesting insights. The aim of the study was to demonstrate the effectiveness of rapport during instructional communication in fostering performance and effort in STEM learning as well as to find further evidence for the advantage of same sex instructors for women. While the former was clearly supported, the latter was not only unsupported but the opposite of our assumptions was substantiated. In the following, we will discuss these results in greater depth.

As

Conclusion

In summary, contributions of this work are three-fold. First, the study adds to literature on human-computer-interaction by showing that virtual agent rapport is an important factor for achieving desirable outcomes such as effort and performance with regard to mathematical tasks. Virtual Agents with rapport can contribute to improve people's performance, specifically with standardized math tests. This observation may support the development of useful and effective applications in mathematical

Acknowledgements

This research was supported by the National Scientific Foundation under grant # IIS-0916858. Two of the authors received support by the PROMOS Program of the German Academic Exchange Service (DAAD).

References (94)

  • J.F. Andersen

    Teacher immediacy as a predictor of teaching effectiveness

    Communication Yearbook

    (1979)
  • J.M. Arhar et al.

    Interdisciplinary teaming in the middle level school: creating a sense of belonging for a-risk middle level students

  • A. Bandura

    Self-efficacy: The exercise of control

    (1997)
  • A. Baylor

    The impact of pedagogical agents' image on affective outcomes

  • A.L. Baylor et al.

    Pedagogical agent design: the impact of agent realism, gender, ethnicity, and instructional role

  • A.L. Baylor et al.

    Pedagogical agents as social models for engineering: the influence of appearance on female choice

  • A.L. Baylor et al.

    The effects of image and animation in enhancing pedagogical agent persona

    Journal of Educational Computing Research

    (2003)
  • D.E. Betz et al.

    My fair physicist? feminine math and science role models demotivate young girls

    Social Psychological and Personality Science

    (2012)
  • S. Beyer et al.

    Gender differences in self-perceptions: convergent evidence from three measures of accuracy and bias

    Society of Personality and Social Psychology Bulletin

    (1997)
  • J. Blascovich

    A theoretical model of social influence for increasing the utility of collaborative virtual environments

  • M. Bong

    Personal factors affecting the generality of academic self-efficacy judgments: gender, ethnicity, and relative expertise?

    The Journal of Experimental Education

    (1999)
  • J. Cassell et al.

    Coordination in conversation and rapport

  • S. Cheryan et al.

    Do female and male role models who embody STEM stereotypes hinder women's anticipated success in STEM?

    Social Psychological and Personality Science

    (2011)
  • R. Chouinard et al.

    The impact of expectancy and value of mathematics on students' commitment

  • R.L. Collins

    For better or worse: the impact of upward social comparisons on self-evaluation

    Psychological Bulletin

    (1996)
  • J.P. Connell et al.

    Competence, autonomy, and relatedness: a motivational analysis of self-system processes

  • S.J. Correll

    Constraints into preferences: gender, status, and emerging career aspirations

    American Sociological Review

    (2004)
  • J.S. Eccles

    Understanding women's educational and occupational choices: applying the Eccles et al. model of achievement-related choices

    Psychology of Women Quarterly

    (1994)
  • J.S. Eccles et al.

    Expectancies, values, and academic behaviors

  • J.S. Eccles et al.

    Social forces shape math attitudes and performance?

    Signs: Journal of Women in Culture and Society

    (1986)
  • J.S. Eccles et al.

    Stage-environment fit: developmentally appropriate classrooms for young adolescents

  • J.S. Eccles et al.

    Age and gender differences in children's self- and task perceptions during elementary school

    Child Development

    (1993)
  • J.S. Eccles et al.

    Motivation to succeed

  • M. Eisenhart

    We can't get there from here: high school girls consider engineering

  • C. Elliott et al.

    Integrating affective computing into animated tutoring agents

  • M. Garau et al.

    The responses of people to virtual humans in an immersive virtual environment

    Presence: Teleoperators and Virtual Environments

    (2005)
  • J.E. Grahe et al.

    The importance of nonverbal cues in judging rapport

    Journal of Nonverbal Behavior

    (1999)
  • N.A. Granitz et al.

    Now it's personal: antecedents and outcomes of rapport between business faculty and their students

    Journal of Marketing Education

    (2009)
  • J. Gratch et al.

    Virtual rapport

  • J. Gratch et al.

    Creating rapport with virtual agents

    Proceedings of the 7th International Conference on Intelligent Virtual Agents (IVA)

    (2007)
  • J. Gratch et al.

    Can virtual humans be more engaging than real ones?

  • B.A. Greene et al.

    Goals, values, and beliefs as predictors of achievement and effort in high school mathematics classes

    Sex Roles: A Journal of Research

    (2002)
  • W.S. Grolnick et al.

    Parent styles associated with children's self-regulation and competence in school

    Journal of Educational Psychology

    (1989)
  • R.E. Guadagno et al.

    Virtual humans and persuasion: the effects of agency and behavioral realism

    Media Psychology

    (2007)
  • F. Guay et al.

    On the assessment of situational intrinsic and extrinsic motivation: the situational motivation scale (SIMS)

    Motivation and Emotion

    (2000)
  • G. Hackett

    Role of mathematics self-efficacy in the choice of math-related majors of college women and men: a path analysis

    Journal of Counseling Psychology

    (1985)
  • A.L. Hayes et al.

    That avatar is looking at me! social inhibition in virtual worlds

  • Cited by (67)

    • A systematic review of pedagogical agent research: Similarities, differences and unexplored aspects

      2022, Computers and Education
      Citation Excerpt :

      The MSLQ measures intrinsic goal orientation, extrinsic goal orientation, task value, control of learning belief, self-efficacy for learning and performance, and test anxiety. In addition to the more common ARCS and MSLQ questionnaires, a few other studies also used the Questionnaire on Current Motivation (Domagk, 2010; Sträfling et al., 2010) and the Situational Motivation Scale (Krämer et al., 2016). For studies that utilized author-developed questions, some evaluated motivation from different perspectives, for example, Chen and Chen (2014) designed a motivation questionnaire for elementary school students, measuring attention, goal, enjoyment, and challenge, whereas some other measure motivation with one or two questions (Grivokostopoulou et al., 2020; Li et al., 2019; Tai et al., 2020).

    • An analysis of children’ interaction with an AI chatbot and its impact on their interest in reading

      2022, Computers and Education
      Citation Excerpt :

      This study found that the storyline questions extracted by the AI techniques acted as a promoter helping students narrate the important storyline in terms of the characters and events, and thus enriched the content of the book talk. Regarding the social affective cues, previous studies have suggested that a chatbot with rapport behaviors is more likely to create a positive social experience attracting students to continuously engage in the learning activity (Krämer et al., 2016). The chatbot of this study applied several social affective cues to build connections with students including personalized greeting, praise, asking about feelings, feeling sharing, and recommendation.

    • Stop Copying Me: Evaluating nonverbal mimicry in embodied motivational agents

      2023, Proceedings of the 23rd ACM International Conference on Intelligent Virtual Agents, IVA 2023
    View all citing articles on Scopus
    View full text