Perceptions and usage of AI chatbots among students in higher education across genders, academic levels and fields of study

AI chatbots have ignited discussions and controversies about their impact on teaching and learning practices in higher education. This study explores students ’ adoption and perceptions of ChatGPT and other AI chatbots in higher education. Based on survey data from a large sample (n = 5894) across Swedish universities, the study employs descriptive statistical methods to analyze usage, attitudes, and concerns, and inferential statistics to identify relations between attitudes and usage and background variables (gender, academic level, and field of study). The results reveal broad awareness and use of ChatGPT among students, but not of other AI chatbots. More than half of the students expressed positive attitudes towards the use of chatbots in education, but almost as many expressed concerns about future use. Statistically significant differences were found across all examined grouping variables, particularly between genders and fields of study. Female students and students from the humanities and medicine consistently expressed more negative attitudes and concerns about AI ’ s role in learning and assessment, while males and technology and engineering students showed higher usage and optimism. These findings not only validate the continued relevance of student backgrounds as a determinant of technology adoption but also expose several challenges and considerations surrounding AI and chatbot usage in education. The study supports the development of local solutions to AI in education tailored to student attributes and needs, and it provides insights for developers, educators, and policymakers.


Introduction
Artificial intelligence (AI) plays an important role in higher education and is profoundly influencing the academic and everyday lives of students (Chen et al., 2020).Foremost in developed countries and China, AI applications are increasingly implemented in education with significant potential to impact teaching and learning across all levels (Tahiru, 2021).Examples include AI-based adaptive learning platforms that provide personalized learning experiences to students, automated assessment, or AI-powered writing tools that enhance students' writing quality by providing real-time feedback on issues with grammar, punctuation, and style (Holmes & Tuomi, 2022;Zawacki-Richter et al., 2019).Concomitantly, AI in education (AIED) has emerged as a vibrant academic field, extending the capability of AI not just to learners but also to educators and institutions alike (Chen et al., 2020;Hwang et al., 2020;Nemorin et al., 2023).
Among current AI innovations, chatbotsconversational agents simulating human dialogue through natural language processing and machine learning algorithmshave gained particular traction through the launch of OpenAI's ChatGPT in November 2022 (OpenAI, 2022).This represents a paradigm shift in the domain; with over 100 million users within the first two months, the chatbot has become the fastest-growing application worldwide (Dennean et al., 2023), forcing other major tech companies to push their AI development programs.ChatGPT is based on the large language model Generative Pre-trained Transformer (GPT), which is trained on massive collections of data in the form of books, articles and openly accessible webpages.In contrast to most previous chatbots, ChatGPT has not only impressed with the quality of its responses, but also showcased its unique capability to "remember" a certain number of previous interactions within the same conversation.The versatility of ChatGPT as a conversational AI extends well beyond specific, single-purpose applications, and its potentials to increase efficiency, accuracy and cost savings were quickly highlighted (Deng & Lin, 2022).This seemingly opens up a multitude of opportunities and new challenges, and the potentially disruptive impact of generative AI is subject to intensive discussion in academia and the public (Li et al., 2023;Lo, 2023).

Advantages of AI chatbots in education fields
In education, ChatGPT's user-friendly and intuitive interface potentially reduces barriers to its wide adoption across educational settings and for different groups of teachers and learners (Kasneci et al., 2023), thereby overcoming many of the obstacles reported in the AIED literature.ChatGPT and similar AI applications can serve as self-study tools (Nisar & Aslam, 2023), assisting students in acquiring information, answering questions (Chen et al., 2023), facilitating group discussions, and resolving problems instantaneously (Rahman & Watanobe, 2023), thereby enriching students' learning experiences, offering personalized support, and potentially boosting academic performance (Kasneci et al., 2023).ChatGPT has also been effectively deployed in the design of educational materials and creative assessments, providing new avenues for content creation and curriculum development (Cotton, Cotton, & Shipway, 2023;Dijkstra et al., 2022).

Limitations of AI chatbots in education fields
Nonetheless, as the advancement and influence of ChatGPT permeate teaching and learning, concerns are being voiced by stakeholders and scholars.Critical issues in higher education involve assessment, examination, and academic integrity (Cotton et al., 2023;Eke, 2023;Yeadon & Halliday, 2023).Advanced generic AI tools such as ChatGPT pose a significant challenge as they are able to closely mimic students' work and are therefore difficult to distinguish from the students' own contributions.This raises concerns about untraceable plagiarism and cheating, prompting a reevaluation of many established assessment methods (Farazouli et al., 2023).Critics have also pointed out ChatGPT's technical limitations, as it is known to sometimes create incorrect information and hallucinate (Weise & Metz, 2023), leading to reliability concerns.Bogost (2022, p. 1) notes that "ChatGPT and the technologies that underlie it are less about persuasive writing and more about superb bullshitting."Large language models like ChatGPT heavily depend on training data sourced from the internet, resulting in outputs that can reflect existing biases present in the data.This raises concerns about the potential reinforcement of societal biases, imbalances, or prejudices that are prevalent in online content (Deng & Lin, 2022).Consequently, a lack of understanding of AI capabilities and limitations amongst end users might lead to misuse and over-reliance on AI technologies by teachers and learners (Kasneci et al., 2023).As AI becomes more deeply integrated into educational systems, there are also significant concerns surrounding copyright issues, data privacy, and security (ibid.).On the other hand, the power of ChatGPT and other AI tools is likely to provide an advantage to some users over non-users, creating an imbalance in the educational landscape (Cotton et al., 2023).This can further reinforce inequalities, as students with access to and experience with sophisticated AI tools can outperform those without them (Adiguzel et al., 2023).Lastly, the sustainability of these technologies and their energy consumption cannot be ignored, since the substantial power needs of AI systems can contribute to a significant environmental footprint (Kasneci et al., 2023).These considerations indicate that the emergence of ChatGPTin addition to the euphoria about its potentialhas also resulted in a more critical discourse about AI and its impact on education and society; it is likely that AI deployment in education will affect diverse groups in various educational contexts differently, raising questions of power, disadvantage and marginalization (Selwyn, 2022).

Aim of the study
Comprehensive and systematic empirical research to support or reject claims made about the benefits and challenges of AI in education, and in particular large language models is, however, scarce, and research on AI chatbots in education is still very much in a state of evolution (Hwang & Chang, 2021;Rudolph et al., 2023).Data from stakeholder perspectives are sorely needed to provide a basis for more informed discussions and decision-making (e.g., Bates et al., 2020;García-Peñalvo, 2023;Rudolph et al., 2023).To this end, this paper aims to examine students' familiarity with, usage of and attitudes towards ChatGPT and other AI chatbots for different student populations based on gender, academic level and field of study.We thereby draw from a dataset of nearly 6000 questionnaire responses across many universities and diverse academic disciplines in Sweden.A preliminary summary of the survey data (i.e., general frequencies of AI chatbot usage and attitudes and respondent comments) has been reported in Malmström et al. (2023).In this study, we focus on descriptive, inferential, and correlational statistical analyses to provide a nuanced and comprehensive understanding of different groups of university students' AI usage and attitudes.Specifically, the study is guided by the following four research questions.
• What is the overall prevalence and pattern of AI chatbot usage among students in higher education?(RQ1) • What are the general attitudes towards AI chatbots among students in higher education?(RQ2) • How do gender, academic level, and field of study influence the usage and attitudes toward AI chatbots in an educational context?(RQ3) • What is the relationship between students' attitudes towards AI chatbots and student's reported use of ChatGPT in their learning process?(RQ4) RQ 1 and RQ2 highlight the present state and impact of AI chatbots in higher education, focusing on AI chatbots as student-facing tools.Prior research has identified generally positive attitudes of the public towards the use of AI in education (Latham & Goltz, 2019).Given the recency of the breakthrough of generative AI, little empirical research is available from within the higher education sector, though there are a few recent reports to build on.A recent student survey from a Belgian university found that, while a large majority of students had used some forms of AI tools for coursework, only 13% of the students had used ChatGPT (Lobet et al., 2023).A U.S. survey (Welding, 2023) among college students revealed that 43% of the respondents had experience with ChatGPT or similar applications and about one third (32%) indicated that they used or planned to use AI tools for assignment completion.About half (47%) of the American students were concerned about the impact of AI on their education and 60% reported that their instructors or schools had not (yet) specified how AI tools could be used ethically or responsibly.Nevertheless, further monitoring of students' use and attitudes toward artificial intelligence in educational settings is essential to enhance informed decision-making by stakeholders in higher education.
Technology adoption and usage have been a long-standing theme within theoretical and empirical information systems research.Several theories and models have been widely employed due to their extensive predictive and explanatory power, among them Davis' (1989) Technology Acceptance Model (TAM), the Theory of Planned Behavior (Ajzen, 1991), andVenkatesh et al.'s (2003) Unified Theory of Acceptance and Use of Technology (UTAUT) as a consolidated framework of several earlier models and theories including the two others.These theories link perceptions and beliefs with a user's intent and actual use of technology moderated by individual and contextual factors (e.g., Liu & Ma, 2023).Loosely building on this theoretical tradition, RQ3 investigates the link between demographics, attitudes, and AI chatbot usage.By examining the influence of variables such as gender, academic level, and field of study on AI chatbot usage and attitudes, we aim to uncover existing disparities or biases that may need to be addressed by educators and educational institutions.This is vital for ensuring that all students, regardless of their demographics or academic backgrounds, can benefit from AI chatbots in education.Additionally, it offers a crucial background for understanding the social dynamics that might be at play in the adoption and acceptance of AI chatbots in a learning environment.Gender has been shown to be a moderating variable in AI adoption (e.g., Nouraldeen, 2022).However, gender effects tend to be stronger for older people as gender stereotypes are less likely to be prominent among younger generations (Morris et al., 2005).Further, technology adoption has been linked to performance expectancy for males and ease of use for females (Venkatesh & Morris, 2000).Both effects are relevant for the context of this study as students tend to be younger and ChatGPT stands out with its ease of use.Thus, while gender effects might be present among students, we expect the differences to be low. 1 The relationship between academic level and technology adoption in general, or AI in particular, has received very little scholarly attention.While Abu-Shanab (2011) found that the educational level did play a moderating role, Sandu and Gide (2019) could not identify a relationship between gender, age, or the level of education of the students and the adoption of chatbot technology in higher education.Similarly, research on the academic discipline of students as a moderating factor is scarce (Chiu et al., 2023), though existing evidence suggests that there are greater barriers to technology adoption within the fields of arts and humanities compared to technological fields (e.g., Mercader & Gairín, 2020).
RQ4 delves into the interplay between perception and practice, a crucial aspect for the effective integration of AI chatbots in education.Investigating the relationship between students' attitudes towards AI chatbots and their actual usage of these tools in their learning process helps us understand the behavioral dynamics behind the adoption of AI in education.As an analysis for all AI chatbots would exceed the scope and length constraints of this article, we focus on ChatGPT as the currently most popular AI chatbot.In line with UTAUT, positive attitudes regarding chatbots are expected to be substantially correlated with chatbot usage (Alzahrani, 2023); conversely, concerns about perceived risks and ethical issues that reflect a lack of trust in current AIED are anticipated to diminish the adoption of chatbots (e.g., Qin et al., 2020).
In answering these four questions, this paper will contribute to a more nuanced understanding of the role of AI chatbots in higher education, in the context of the potentially disruptive impact of ChatGPT.We expect our findings to guide developers, educators, and policymakers in exploring the potential of AI while remaining cognizant of students' perspectives and needs.

Research approach
The methodology employed for this study follows a quantitative research paradigm, suitable for examining usage, attitudes, and correlations across a large sample of participants (Creswell, 2014).Our research design centered on survey research.We applied single-item measures to keep the survey doable within less than 5 minutes.While this method introduces a limitation, as reliability measures cannot be applied, it reduces respondent burden, survey fatigue and attrition.Additionally, past research supports the predictive validity of single-item measures for constructs that are concrete and easily and uniformly imagined (Bergkvist & Rossiter, 2007;Rossiter, 2002) as we argue is the case in this study.
The survey items, partly inspired by the groundwork laid by Welding's (2023) study on AI usage in American colleges, were developed and piloted among a small sample of the target population with the objective to make sure the questions were understandable and to verify that the time it took to complete the survey.Subsequently, some minor refinements were made to a few items based on the pilot feedback.The finalized survey was then launched through the online survey platform Questback during the spring of 2023.The survey was split into two main sections addressing chatbot usage and attitudes towards AI in education.

Instrument and data collection
The survey questions can be found in Appendix 1.The first section of the survey aimed to gauge the students' familiarity with and usage of ten AI chatbots that supposedly are used for educational purposes, including but not limited to ChatGPT.2For each chatbot, respondents were asked: 'Rate your familiarity and frequency of use with a selection of AI chatbots,' followed by a four-item ordinal scale as answer categories: Unfamiliar; Familiar but never use it; Familiar but rarely use it; and Familiar and regularly use it.
The second section focused on student attitudes towards AI in education.This covered general attitudes towards chatbots in education, perceived effects of chatbot use on learning and academic performance, ethical concerns, and issues related to institutional guidelines on chatbot usage.The latter section used a response format comprising ten agreedisagree statements, along with a "don't know/prefer not to say" option.Background information pertaining to gender, field of study, and academic level was also collected from respondents (see Table 1).Although we also asked about university affiliation, this aspect was not analyzed in this study.A link to the survey was disseminated via multiple channels, encompassing networks with various Swedish universities and a promotional campaign on social media platforms, such as LinkedIn and Facebook.The survey was open from April 8, 2023, to May 5, 2023, securing a sample of 5894 students; this convenience sample broadly mirrored the national distribution of students in terms of gender, academic level, and discipline.

Data analysis
The analysis of the collected data encompassed both descriptive and inferential statistical methods using the Statistical Package for the Social Sciences (SPSS) software, version 28.0.1.1 ( 14) developed by IBM.Descriptive statistics were employed to address RQ1 and RQ2 by summarizing the distribution of responses about chatbot usage and attitudes 1 It is important to recognize that gender is not a binary concept and many individuals do not strictly identify as male or female.Unfortunately, our data on non-binary individuals was insufficient to be included in the main analysis, but the mean values for both chatbot usage and attitudes were consistently inbetween those of the male and female populations.
To answer RQ3, we applied chi-square tests to test for differences between the sub-groups, complemented by Cramer's V as nonparametric effect size measure and a post hoc evaluation of corrected standardized residuals.Those tests were suitable as they correspond to the categorial and ordinal nature of our variables without the need to meet the assumption of normality.Non-binary students were not included in the analysis of gender effects due to the low response frequency.
To address RQ4, we adopted the Kruskal-Wallis Test as a nonparametric equivalent of the one-way analysis of variance (ANOVA) without the need to meet assumptions of normality and homogeneity of variance.Post-hoc analyses were conducted using Mood's Median Test with a significance level of 0.05.and an additional component to identify homogeneous subsets of our data.
Finally, we conducted a correlational analysis to assess the relationship between the usage of ChatGPT as the currently most popular AI chatbot in this analysis and students' attitudes towards AI in education.Again, we employed Spearman's rho correlation as the best fit for the ordinal nature of our data.
Overall, we chose a conservative approach as a rigorous statistical foundation for our analysis.In choosing to analyze our data as ordinal, we minimize the risk of Type I errors (falsely rejecting the null hypothesis) by prioritizing the inherent order of the response options, while not imposing assumptions of continuity and interval equality of the response options.However, this methodological choice comes with the trade-off of potentially increased Type II errors, where we might fail to detect actual differences or relationships due to the tests' more stringent criteria for significance.Despite this, we believe that this approach strengthens the validity of our findings.By adhering closely to the ordinal nature of our data, we reduce the interpretative leaps and assumptions required by parametric tests, such as ANOVA, which would treat the data as quasi-continuous.At the same time, this ensures that any observed differences or relationships are not artifacts of the analytical method but are indicative of the phenomena under study.By opting for a conservative statistical approach, we aim to enhance the credibility and reliability of our findings, acknowledging the limitations of our data's scale of measurement.

Descriptive analysis
The descriptive analysis of students' usage of different AI chatbots, as illustrated in Table 2, reveals a varied usage pattern.The most frequently used chatbot is ChatGPT, with 35.4% of students being familiar and regularly using it.It is the only chatbot with a Median higher than unfamiliar.By contrast, other AI chatbots such as YouChat, ChatSonic, DialoGPT, Socratic, and Jasper Chat were rarely or never used by the majority of participants (in many cases, over 90% of the respondents claimed to be unfamiliar with these other chatbots). 3able 3 provides a descriptive summary of students' attitudes toward the use of AI chatbots in education.More than half of the students (55.9%) had a positive attitude towards the use of AI chatbots in education.However, almost as many (54.2%)expressed concern about the future impact of AI chatbots on students' learning, with the result that the Median is Agree for both statements.
Regarding the effect of AI chatbots on learning and performance, 47.7% of respondents agreed that the chatbots they used made them more effective learners, whereas only 17.3% confirmed the positive effect of chatbots in improving their study grades.Meanwhile, only 26.8% thought that chatbots improved their general language abilities.Even fewer (17.9%) believed that AI chatbots generated better results than they could produce on their own.
Concerning ethical aspects and academic integrity, a majority of students (61.9%) expressed the view that using chatbots to complete assignments and exams amounts to cheating.However, 58% disagreed with the statement that using chatbots goes against the purpose of education, and 60.3% disagreed with the prohibition of chatbots in educational settings.Lastly, only 19.1% of the students reported that their teachers or universities have rules or guidelines on the responsible use of AI chatbots.

Inferential analysis
Tables 4 and 5 present differences in chatbot usage and attitudes by gender.The chi-square tests indicate statistically significant genderbased differences in familiarity and usage across all chatbots with weak to medium effects (between 0.2 and 0.38).Post-hoc analysis of the corrected standardized residuals shows significant deviations from the expected distribution in all categories with males consistently expressing high familiarity and usage than females.For ChatGPT in particular, it is interesting to note that females were more likely than expected to be familiar with the chatbot but never actually use it (r = 13.7).
Concerning attitudes towards chatbots (see Table 5), chi-square tests also reveal statistically significant gender differences across all attitude statements.The gender effects are generally weak (between 0.1 and 0.3) and manifest in different directions.Female respondents were ostensibly more concerned about the impact of AI on education (r = 5.6), considered the use of chatbots as potentially contrary to the purpose of education (r = 9.3), and viewed the use of chatbots in assignments and exams as cheating (r = 6.1) that should be prohibited (r = 9.6).Males, on the other hand, had an overall more positive attitude towards chatbots (r = 14.8) and perceived them to a greater extent as tools that can improve their learning (r = 13.3) and grades (r = 12.1).
Furthermore, we examined differences in chatbot familiarity, usage and attitudes by academic level (Tables 6 and 7).While the chi-square tests indicate statistically significant differences in the familiarity and usage of chatbots as well as most attitude statements across academic levels, these effect sizes are generally very weak (<0.1), as indicated by Cramer's V.
The corrected standard residuals show that first-cycle students had a greater likelihood of being unfamiliar with chatbots (r between 3.4 and 7.4), particularly with Bing AI and Bard AI.They also reported lower rare (r between − 2.5 and − 3.7) and regular chatbot usage (r between − 2.2 and − 3.8, non-significant for Bard AI).Second-year students generally showed a reversed trend with fewer students than expected being unfamiliar with chatbots across all five chatbots (r between − 3.6 and − 6.7).A higher proportion of second-year students used ChatGPT (r = 4.5) and CoPilot (r = 2.0) on regular base.Notably, third-cycle students, while showing no significant differences for ChatGPT and most other categories, had significant positive residuals for the regular usage of all other chatbots (r between 3.3 and 5.1) indicating a higher reliance on AI technologies beyond the popular ChatGPT among this group.
With regard to attitudes towards chatbots (see Table 7), first-cycle students generally were more negative compared to other students.Not only did we find less agreement towards an overall positive attitude (r = − 3.9), but also regarding the efficacy of chatbots in improving their learning effectiveness (r = − 3.9), language ability (r = − 5.9), and study grades (r = − 4.6).Further, these students had stronger reservations about the role of chatbots in education.This is reflected in the tendencies towards seeing the use of chatbots for completing assignments as cheating (r = 3.4) that should be prohibited (r = 3.7) and that goes against the purpose of education (r = 4.4).
In contrast, second-cycle students displayed overall more positive attitudes towards chatbots (r = 3.9).For example, they were more likely to agree that chatbots enhance their effectiveness as learners (r = 4.0), improve their study grades (r = 4.3) as well as their language ability (r = 4.5).They also expressed more disagreement with ethical concerns such as perceiving the use of chatbots as going against the purpose of education (r = 5.9), as cheating (r = 2.6), or that chatbots should be prohibited (r = 6.0).
Third-cycle students, besides perceiving chatbots as contributing to their language ability to a higher extent (r = 4.6), did not show any significant residuals at all.Table 8 shows the results of a Kruskal-Wallis test examining the effect of field of study on university students' chatbot usage and attitudes.The test value column shows that there were statistically significant differences between at least two groups of students with different fields of study for all items with p < 0.001.Effect sizes are small to medium and tend to be particularly strong for the items related to usage of AI chatbots.The differences between the fields can be seen in the subset columns.Each subset column shows those study fields that build a homogenous subgroup, meaning there were no significant differences between the responses from students from fields listed in the subgroup.Subsets and groups are ordered from low to high, meaning that the group on the left has the lowest usage or agreement and the highest can be found on the right.
For example, for ChatGPT, medicine and healthcare students were least familiar with it, followed by students in the humanities.The differences between the students in these fields were not statistically significant, so they build subgroup 1 with the lowest familiarity of ChatGPT.In the same way, students from the natural sciences and social sciences showed significantly higher usage of ChatGPT than the former two, building subgroup 2. Students of technology and engineering expressed significantly higher familiarity with ChatGPT than all other groups and are therefore alone in subgroup 3. Students from a certain field of study can also belong to several subgroups (as for example for Bard AI).Here, students from the natural sciences showed higher usage than students from medicine, humanities and social sciences, but the difference was only statistically significant when compared to the lowest group, medicine students.Thus, students of humanities and social sciences are both in subgroup 1 (with medicine and healthcare students) and 2 (with natural sciences students).Looking at all the items in the survey, some consistent patterns emerge.Students in technology and engineering clearly stood out from all other groups.They used chatbots to a statistically significantly higher degree and had the least concerns about the ethical aspects of AI usage in education.Notably, for most items, the difference is statistically significant in relation to all other academic fields.Conversely, students from the arts and humanities as well as medicine and health care showed the opposite pattern; students from those fields were significantly less familiar with ChatGPT.They also expressed more concern, an overall less positive attitude towards the use of chatbots in education, and more students from these groups believed that the use of chatbots goes against the purpose of education.They were also more supportive of a prohibition of chatbots in education.

Correlational analysis
Table 9 shows the Spearman's rho correlation coefficients between the usage of ChatGPT and attitudes towards AI in education and chatbot usage.The results show a statistically significant relation for all attitude questions except one.The use of ChatGPT is strongly positively associated with students' belief that the use of chatbots is common among fellow students (rho = 0.406), their positive attitude towards chatbots in education (rho = 0.581), and the belief that chatbots make them more effective learners (rho = 0.644).A weak positive correlation was found between the usage of ChatGPT and the belief that chatbots improve general language ability (rho = 0.197) and study grades (rho = 0.189).Medium and strong inverse correlations were found between the usage of ChatGPT and concern about AI-chatbots' impact on students' learning in the future (rho = − 0.304), between the views that using chatbots contradicts the purpose of education (rho = − 0.557), constitutes cheating (rho = − 0.327), and that chatbots should be prohibited in educational settings (rho = − 0.564).Finally, a weak negative correlation was found between ChatGPT usage and their awareness of rules or guidelines on the responsible use of chatbots (rho = − 0.049).No statistically significant correlation was found between ChatGPT usage and the belief that chatbots generate better results than the respondents could produce on their own.

Discussion
This study attempted to empirically investigate university students' AI chatbot usage and attitudes towards artificial intelligence in education, following the introduction of potentially disruptive large language models, particularly ChatGPT, on a wide scale.Through this exploration, a complex landscape has been unveiled, wherein multiple factors interact to shape students' perceptions and behaviors.Notably, the analysis has provided a more nuanced understanding of the roles that gender, discipline, and academic level play in this new learning context.
First, we were interested in the general usage frequency and familiarity with different chatbots among students.Here, the hype around ChatGPT is reflected in our data since 95% of the respondents were familiar with ChatGPT and more than one third (35.4%) claimed to use it regularly.This was confirmed by a similar proportion of students stating that the use of chatbots is common among their fellow students.With regard to the true representativeness of the total student population, these numbers need to be interpreted with caution, as students already familiar with chatbots might be somewhat more likely to participate in the survey compared to unfamiliar ones.Still, our results indicate an increase in familiarity and usage compared to earlier reports (Lobet et al., 2023;Vogels, 2023;Welding, 2023) that would suffer from a similar bias, thus supporting the claim that ChatGPT, despite its novelty and limitations, has quickly attained more widespread recognition and use among students in higher education.Interestingly though, this popularity appeared to be restricted to ChatGPT in particular and did not expand to other chatbots at the time of the survey.Some chatbots from larger companies, such as Bing AI and Bard, were also familiar to a substantial proportion of university students in Sweden, but most had never used any of the other applications.While this pattern does not apply to AI applications in general (see e.g., Malmström et al., 2023) and might change in the future, it underscores ChatGPT's prominent role as the key driving force behind the ongoing popularization of generative AI in education.It also stresses the urgent need for educators and educational institutions to adapt education to this new situation and find ways to address the potentials and challenges connected to this technology.At the time of the survey, more than four out of five students were not aware of any rules or guidelines from teachers or their universities in this regard.
Examining students' attitudes towards chatbots more specifically, we saw that while over half of the respondents expressed an overall positive attitude towards the use of chatbots in education, almost as many expressed concerns about their impact in the future.This disparity of optimism and concern highlights the complex relationship that students have with this emerging technology.With regard to the effect of chatbots on learning and performance, almost half of the students indicated that chatbots make them more effective as learners, pointing at the potential but also the ease with which ChatGPT seems to be utilized as self-study tool, facilitator, or assistant for learning (Chen et al., 2023;Nisar & Aslam, 2023;Rahman & Watanobe, 2023).
Nevertheless, fewer than one in five of the students felt that chatbots produced better results or improved their grades.This could indicate that most students use chatbots as a complement in the learning process rather than to complete assignments and exams, though the high number of students choosing "don't know/prefer not to say" requires caution when interpreting these results.However, these findings receive some confirmation when questions about academic integrity are considered: more than sixty percent believed that the use of chatbots during examination is cheating, Nonetheless, a majority of students were against the prohibition of AI in education settings and neither thought that chatbots go against the purpose of education.Thus, it appears that many students were aware of the potential of chatbots to support the actual learning process, and more insight is needed on precisely how students use chatbots in practice.Qualitative research could potentially address this aspect.
Regarding the relationship between the familiarity and usage of AI chatbots and student attitudes, our findings provide empirical support for the predicted relationship between students' attitudes towards AI chatbots and their level of familiarity and usage.We found a strong positive correlation between ChatGPT familiarity and usage and positive attitudes towards chatbots as well as perceived benefits from their use.Conversely, experience with ChatGPT was strongly negatively correlated with concerns about the impact of AI on future learning and ethical concerns surrounding chatbot usage in education.These findings are consistent with the broader predictions of UTAUT (Venkatesh et al., 2003) and other empirical research (e.g., Alzahrani, 2023;Qin et al., 2020).They underscore the importance of the relationship between exposure and hands-on experience and beliefs about technological innovations in educational contexts.
To what extent do chatbot usage and attitudes differ for different groups of students?Our results found statistically significant and consistent differences in the responses for all three examined grouping variables (gender, academic level and field of study).Regarding gender, we found particularly strong differences in familiarity with and usage of chatbots.Female students were overall more negative and concerned about the impact of AI on learning and assessment.Thus, as predicted by UTAUT (Venkatesh et al., 2003) and confirmed by recent AIED empirical research (Nouraldeen, 2022), our results indicate that gender is still a relevant factor for AI adoption for the current student generation and in the context of ChatGPT; this may be indicative of underlying societal factors or personal experiences that necessitate further investigation.Moreover, these results prompt an essential reflection on the design and implementation of educational technology.If AIED tools like ChatGPT are to be effectively utilized and integrated across diverse student populations, the identified differences must be acknowledged and addressed.Gender-sensitive approaches, tailored interventions, and inclusive design principles may be required to ensure that AI-powered educational solutions cater to the unique needs and preferences of various student demographics.
The effect of academic level, even though existent, was very weak (the effect sizes were consistently small).Consistent with Abu-Shanab (2011), the post-hoc analysis of the residuals showed that advanced-level students exhibited slightly greater familiarity and usage.Second-cycle students were more likely to be users of ChatGPT, while third-cycle students showed a higher tendency to regularly use other AI chatbots.These effects could be linked to more specialized AI utilization opportunities in advanced, research-based projects, or generally higher self-regulated learning skills among graduate and PhD students.Further, we found more favorable attitudes and more limited concerns about using chatbots in education for second-year students, along with the opposite tendency for first-year students.Interestingly, third-cycle students did not show the same attitude patterns as second-year students, indicating a potential non-linear relation between academic level and attitudes towards chatbots.However, given the paucity of empirical studies on this question, more research is required to draw definitive conclusions.
As for disciplinary differences, our results indicate clear and consistent differences, with students from engineering apparently using chatbots to a much higher degree and expressing stronger optimism towards AI.Conversely, students from the arts and humanities and from medicine and health care used chatbots less and were more skeptical.While the results for humanities and arts confirm prior research (Mercader & Gairín, 2020), the results for students from the medicine and healthcare sectors are surprising, given the fact that it is one of the fields most actively discussing AI adoption (e.g., Zhang et al., 2023).Our findings also contradict the results of prior research reporting that students generally have positive attitudes towards AI in medicine (Santomartino & Yi, 2022).Potentially, our students' skepticism is connected to chatbots in particular or reflects more on the health care sector as a whole than medicine alone.However, this should be considered in the sometimes euphoric discussion about the potential of ChatGPT in medical education (e.g., Lee, 2023).The consistency and decent effect sizes of our results also raise the question of the importance of academic disciplines and differences in disciplinary traditions and practices as explanatory factors in AIED in general.Our findings motivate further examination of this perspective, which is not well theorized and so far largely lacks empirical verification (see Orji, 2010 for an exception).
Generally, our findings also have pedagogical implications for teachers and academic institutions.Students' AI literacy development enabling them to critically evaluate, communicate with, and use AI technologies (Long & Magerko, 2020) are likely to be significantly shaped by the ways in which teachers insert and discuss AI technologies in their teaching practices.Our results suggest that AI-related policies and guidelines in that regard should not be one-size-fits-all; rather, support efforts need adaptation to the student characteristics and teaching methods in specific disciplinary contexts.Thus, solutions to handling AI in education should be developed and implemented "locally" to address the specific needs of the local student population.For example, the widespread enthusiasm and experience with AI chatbots among technology and engineering students can provide an opportunity to integrate AI chatbot tasks more deeply into curricula, while at the same time requiring particular efforts to create awareness of the technology's limitations and ethical concerns.In other fields, different instruction strategies might be required to address a lack of comfort and/or proficiency with AI chatbots, for example through feedback mechanisms, allowing students to share their concerns, experiences, and suggestions about AI chatbot usage.While there will always be variation between individual students that needs to be considered, our results can help teachers adapt their handling of AI tools to their particular teaching contexts.

Conclusions and limitations
Almost 6000 university students in Sweden answered survey questions about their usage and attitudes towards ChatGPT and other AI chatbots.The insights gleaned from this research underscore the importance of understanding student perceptions and experiences of AI chatbots in educational settings.The study has revealed the widespread usage of ChatGPT among university students.Given that we are likely only seeing the beginning of large language model applications, we agree with other educators and AIED researchers to conclude that the use of ChatGPT and other chatbots in education among students is already mainstream and likely to stay (Hajkowicz et al., 2023).While concerns about academic integrity and cheating are valid and justified, many students acknowledge the usefulness of AI for their actual learning, and our efforts should be directed towards supporting these developments.Students still need substantive training and learning, and ChatGPT should be treated as a tool rather than a replacement (Berdanier & Alley, 2023), but both students and teachers need new competencies in integrating AI chatbots in the learning process.
Case-based inspiration and examples of how AI chatbots can productively support learning are currently published on a daily basis (e.g., Santos, 2023).Nevertheless, some students benefit more from these developments than others.Our results show multifaceted and sometimes conflicting views on the role of AI in education, and these views are influenced by gender and academic discipline.Addressing the needs of different student populations will require locally adapted solutions.Given the apparent lack of guidelines and rules, many teachers and decision-makers would appear to be unprepared to this end.
This study also highlights the importance of addressing students' concerns about the potential impact of AI on their future learning.
Ongoing lively discussions about the potential and dangers of AI in education need to be complemented with empirical studies of the kind presented here.In addition, qualitative research is needed to better understand how students use AI tools in practice.Ultimately, the findings from this study contribute to the growing body of literature on the role of AI in education and provide a valuable resource for developers, educators and policymakers as they navigate the emerging landscape of AI chatbots within the educational sector.It is also important to acknowledge that the data underpinning our study was collected in May of 2023, marking a specific snapshot in the rapidly evolving landscape of AI.As such, the reported AI chatbot usage and attitudes are likely to be subject to change as new applications emerge and awareness grows.Thus, the temporal context serves both as a limitation and a springboard for future research highlighting the need for continuous investigation to update and adjust the observed findings.Certain further limitations of the present study must be acknowledged.First, the sample used in this study was not random.The respondents self-selected to participate, and the topic of the survey might make students who already had some degree of exposure to chatbots in their academic settings more likely choose to engage.This potential selection bias could somewhat overstate the familiarity and usage of AI chatbots.Despite these limitations, the large sample size used in this study, encompassing thousands of responses, enhances the statistical power of the analysis and allows for the detection of even small effect sizes.Furthermore, the broad mix of respondents from different academic levels and genders provides a heterogeneous sample that has offered a rich view into the range of student experiences and attitudes towards AI chatbots.This was enabled through the use of single-response items instead of larger attitudinal constructs, which may raise concerns about the reliability of our approach.Thus, while we feel that the use of single items is justified in this particular study to provide a particular, readily interpretable snapshot of certain attitudes and behaviors, we encourage future research to build on this work by employing established or newly developed multidimensional scales for a more comprehensive understanding of the factors that drive students' perceptions and interactions

Table 1
Descriptive statistics of the demographic characteristics of participants.

Table 2
Descriptive analysis of students' usage of different chatbots.

Table 3
Descriptive Analysis of Students' Attitudes towards AI chatbots in Education.

Table 4
Chi-square and corrected standardized residuals for Chatbot Usage by Gender.

Table 5
Differences in chatbot attitudes by gender.

Table 6
Differences in chatbot usage by academic level.

Table 7
Differences in chatbot attitudes by academic level.

Table 8
Differences in chatbot usage and attitudes by field of study (kruskal-wallis test with homogeneous subgroups (p < 0.05), grouping from low to high usage and disagree to agree).Medicine and healthcare, H … Humanities and art, N … Natural sciences, S … Social sciences, T … Technology and Engineering (homogeneous subgroups are based on asymptotic significances with α = 0.05).Correlation between usage of ChatGPT and attitudes towards AI in education and chatbot usage.