Measuring CS Student Attitudes Toward Large Language Models

With the mainstream adoption of Large Language Models (LLMs), members of both academia and the media have raised concerns around their impact on student learning and pedagogy. Many students and educators wonder about the pedagogical fit of this emerging technology. We aim to measure the adoption of and attitudes toward LLMs among the CS student population at an R1 University to determine how students are using these new tools. To this end, we conducted a large survey study targeting two populations participating in computing courses at the university: intro-sequence students (ISS) and experienced students (ES).

In our preliminary results from Spring 2023, we've found several significant differences among the views of over 700 respondents across the two groups. Most students reported LLMs' unparalleled potential for quick information access, yet many harbor concerns about the reliability of the LLM responses, and the impact on academic integrity. Additionally, while ES have rapidly integrated LLMs into their learning, ISS remain cautious of the tools, highlighting a stark contrast in adoption rates between the groups.

LLMs are clearly going to reshape pedagogical approaches and student engagement. Our study hopes to provide insight on the nuanced student attitudes toward LLMs. For example, the notable reservations expressed by ISS illustrate an imperative for careful, informed, and ethical integration to ensure these tools enhance rather than compromise the educational experience. In the future, we plan to continue tracking student attitudes in order to gain further understanding of the changing perceptions of LLMs and their impact.