Are AI detection and plagiarism similarity scores worthwhile in the age of ChatGPT and other Generative AI?

,


Introduction
Just a few years ago scholars were investigating the strengths and weaknesses of whether tablets and mobile phones should be used in the classroom (van't Hooft, 2013;Clarke & Svanaes, 2014).In hindsight, it seemed silly to prohibit the use of such devices, as in today's modern classrooms digital pedagogy is common, and those who do not have access to technology are simply left behind.The traditionalist might argue that digital devices are troublesome, arguing that they reduce the learners' attention as the learners provide the teacher with only pseudo attention when using their devices-a point with which I do not disagree.However, I also know that if my students do not have digital devices, their whole qualification would need to be delivered differently, and they would not be ready for the world of work, which also relies on digital tools.Thus, the use of digital technology has become the status quo in most aspects of daily life, bringing new benefits and challenges as technology becomes increasingly immersive (Kovács, Murray, Rozinaj, Sulema & Rybárová, 2015;Calvet, Bourdin & Prados, 2019).
AI has become useful in many fields and even indispensable in others.GAI is being actively used in webpage design (Calò & De Russis, 2023), writing of code for software designers (White, Hays, Fu, Spencer-Smith & Schmidt, 2023), customer support, cyber security (Kalla & Smith, 2023), virtual assistance, marketing, and human resources (Fui-Hoon Nah, Zheng, Cai, Siau & Chen, 2023).With regard to education, students and academics have a new mode in how knowledge can be sourced and composed.While the benefits of technology are many, there is also another side, that Jacques Ellul (1964:324) warned about: Technique does not confine itself merely to the realm of technical production, but infiltrates every aspect of human existence, and has no time for "inefficiencies" caused by loyalties to family, religion, race, or culture; a society of dumbed-down consumers is absolutely essential to the technological society, which must contain predictable "demographics" in order to ensure the necessary financial returns.The only thing that matters technically is yield...This is the law of technique… Humans were once the only source of knowledge, but that space has become crowded, not with other humans, but with AI.The ease of use that GAI exhibits makes it enticing and it is unsurprising that students worldwide have flocked to using ChatGPT and other similar platforms (Baidoo-Anu & Ansah, 2023).It seems plausible that the increasing reliance on GAI may indeed accelerate what Ellul termed "dumbed-down" humans, particularly if humans regularly rely on AI to "think" for them.
SOTL in the South 8(2): August 2024 Baron Since the purpose of education relies on the acquisition and demonstration of knowledge, both of which AI can do, this new normal brings with it new questions and techniques that educators will need to consider.It thus seems fair to term GAI a paradigm shift in how we think about knowledge (Bozkurt, 2023), especially since AI is now accessible in a manner never seen before (Bughin, Seong, Manyika, Chui & Joshi, 2018;Lin, 2023;Roser, 2023;Yu, 2024).This has far-reaching implications for educational institutions in terms of how knowledge is assessed, since traditional modes of assessing students, such as written assignments, research reports, and essays, can now be generated by AI (Herbold, Hautli-Janisz, Heuer, Kikteva & Trautsch, 2023;Terwiesch, 2023).This challenges a fundamental principle in education: the principle that students must be evaluated based on their own work, or said differently, no student should get an unearned advantage over another student (Mullens, 2000;Hughes & McCabe, 2006).However, achieving this goal is challenging in the world of GAI, as platforms such as ChatGPT can generate plausible content within seconds, and it is difficult to discern where the student's learning started and where it stopped (Lin, 2023).
While some educators may think that they can simply continue with their regular teaching approach in the way they did before, my aim for this article is to demonstrate just how easy it is to overcome the traditional detection mechanism (Turnitin) that is commonplace in thousands of universities and schools.A discussion follows addressing some of the implications of this new normal, including a proposal for how one may detect plagiarism in the future.
To follow, is a brief explanation of different types of GAI and how different forms of software work to detect possible plagiarism.

Generating unique content using AI and the detection thereof
The term GAI refers to a subset of AI techniques focused on creating new and original content.This content includes text, images, audio, and video that are derived from training data (Aldausari, Sowmya, Marcus & Mohammadi, 2022;Feuerriegel, Hartmann, Janiesch & Zschech, 2024).A point worth noting is that while there are many approaches used in machine learning, GAI is not the same as discriminative modelling.While discriminative models focus on learning the boundary between different classes or categories in the data, generative models aim to capture the underlying distribution of the entire dataset.In other words, while discriminative models learn to discriminate between different classes based on features present in the data, generative models learn to generate new data points that resemble the original dataset (Bishop, 2006).This is important since the generated content in generative models has a novel element, which means that if students use ISSN 2523-1154 SOTL in the South 8(2): August 2024 Baron the generated content, it potentially would not be detected as copied or plagiarised.This level of newness in the content is novel in the sense that it is written uniquely, not necessarily new in the sense of it being brand new ideas, although GAI can do that too.
For example, since ChatGPT generates its content by using sources from the internet as part of its learning process, if a student prompted ChatGPT to answer a series of assignment questions on ChatGPT provide each student with the same content?Fergus, Botha andOstovar (2023:1674) found that identical questions asked to ChatGPT from different user accounts did not produce identical answers.When the responses were run through Turnitin, the similarity index provided was between 1 A high percentage similarity report does not necessarily mean the work was plagiarised, and a low similarity percentage might even mask a submission that includes plagiarism.Thus, the correct terminology would be "text-matching software" or a similarity detector (Tauginienė et al., 2019:68).However, the phrase "plagiarism checker" is used in this article as this is how numerous companies that offer this service advertise their product, and the purpose of the software is to alert the user to possible instances of plagiarism.See www.quillbot.com/plagiarism-checker; www.grammarly.com:www.duplichecker.com 2 https://youtu.be/n4HoqiyotB0 3 Quantifying what is considered novel can be a slippery slope.In literary works, novelty might be measured by the use of different words, which can qualify as intellectual property.However, in scientific contexts, novelty is often assessed based on the originality of ideas rather than just the wording.Attempting to mask a copied idea by using different words may still be considered plagiarism.
SOTL in the South 8(2): August 2024 Baron 21-25%, which the authors described as relatively low4 .The implications of these two scenarios are that, firstly, if students copy the responses directly from ChatGPT, since there is a level of newness in the content, they might evade a plagiarism checker.Secondly, if several students submit their individual responses from ChatGPT as their assignment, even though the students all asked the same questions, it is unlikely that they could be identified as copying one another.This weakness suggests that Turnitin focuses on text similarity rather than idea similarity.
One approach to address the aforementioned issue of students relying solely on ChatGPT's responses involves implementing an AI detector to identify whether AI was employed in composing the answers.In this context, an AI detector refers to a tool designed to detect the presence of AI in academic writing.Common AI detection tools include Copyleaks, Originality, and Turnitin, with new tools regularly coming online.In this study, Turnitin was used to detect AI-generated content.
It is noted that AI detection is different from a similarity test.Thus, in this study, two forms of plagiarism are investigated: the first is the use of AI-generated content submitted as someone's own work, and the second is the copying of a scholar's work but then using AI to change/paraphrase the work.

Plagiarism detection
Plagiarism ranks among the most significant challenges faced by academia (Vrbanec & Meštrović, 2017).Plagiarism is the act of using someone else's words, ideas, or creations without giving proper acknowledgment.It can occur in various forms, including copying text directly from a source without quotation marks or proper citation, paraphrasing someone else's work without giving credit, interleaving and/or interweaving various sources together in the work without citation, or even providing proper citation but failing to change the wording sufficiently (Butakov & Scherbinin, 2009;Gregory & Leeman, 2021).
The digitisation of scholarly content has provided a fertile ground for plagiarism owing to the ease of copying and pasting digital content, further facilitated by the introduction of search engines (Park, 2017).One method used to counteract this scourge was the introduction of text analysers which are used in text-matching software, such as Turnitin.
SOTL in the South 8(2): August 2024 Baron Text analysers use text mining algorithms, which are complex and cover a wide variety of methods to scrutinise suspicious text.Grammar-based methods detect plagiarism by using string-matching algorithms such as the Knuth-Morris-Pratt, Boyer-Moore, and Rabin-Karp algorithms to compare two strings of text to identify similarities between them for exact matches (Sri, Bhavsar & Narooka, 2018).Subsequence matching is a technique that enables the identification of matches even when the sequence of words is not maintained, making it valuable for detecting slightly paraphrased passages.Another form of string matching is the shingles algorithm, which involves breaking down text documents into sets of consecutive tokens, known as shingles, and then hashing these strings to create a unique identifier.The hashed shingle sets for each document are then compared to identify similarities (Vashchilin & Kushnir, 2017).
People also plagiarise by copying the original source but trying to hide this by changing certain words using synonyms or other minor changes-often termed syntax-preserving plagiarism.In this form of plagiarism, the sentence still has the same intended meaning as the structure and organisation of a sentence or passage are preserved, while the original words are replaced with synonyms or only slightly paraphrased.One method used to detect this type of plagiarism is semantic analysis.This detection approach involves analysing the meaning and context of text to identify similarities between documents.This representation can be achieved using techniques such as word embeddings, which map words to high-dimensional vectors based on their semantic relationships.If two texts are semantically similar, the texts are interpreted as evidence of plagiarism (Al-Shamery & Gheni, 2016).Semantic models focus on determining if there is similarity even when the words in the sentence are changed but the meaning remains same.Semantic methods are more robust to changes in wording, syntax, and structure and are better suited for detecting paraphrased content (Al-Shamery & Gheni, 2016).

Modern plagiarism
Apart from generative chatbots (such as ChatGPT and Gemini), online AI writing tools, including paraphrasers, summarisers, and co-writers, have become increasingly capable and prevalent.These tools are so popular that the public is often spoilt for choice.Figure 1 shows a non-exhaustive collage of AI writing tools that students and academics might use.Some of these tools are marketed with claims that their products can "humanise" AI-generated text5 .This marketing term, "humanising," is typically followed by assertions that the AI writing tool can produce content that would not be detectable by an AI detector, such as Turnitin.For example, a website called undetectable.aiclaims that they will refund the user's money if their "humaniser" tool is detected as AI-generated text6 .
Figure 1: Examples of online writing tools that can assist users with their writing.Many of these tools also offer the capability to paraphrase AI-generated text.

The point of no return: AI-generated content creation
Ellul's law of technique (1964) posits that technique does not tolerate inefficiencies.In my own life, ChatGPT has saved me time on academic tasks and even assists in writing responses to users who comment on my webpages.It would be challenging to stop using ChatGPT, as it significantly enhances my work efficiency.This trend is becoming evident across many disciplines (Calò & De Russis, 2023;Fui-Hoon Nah et al., 2023;Kalla & Smith, 2023;White et al., 2023).AI tools are also beneficial for improving grammar, which is particularly advantageous for students and academics who are not native English speakers.As an educator and scholar, I thus understand the appeal of using ChatGPT and other GAI tools as valuable sources of knowledge.
For students, ChatGPT has become essential for home learning, serving both as a knowledge source and a tool to streamline routine tasks.However, using ChatGPT to aid one's learning is fundamentally different from using it as a substitute for learning.In the first scenario, ChatGPT acts as a tutor, whereas in the second scenario, it would be akin to expecting one's tutor to complete all the work.This situation can be compared to an extreme form of plagiarism known as "contract cheating," specifically ghostwriting, which involves paying someone else to complete the work (Gregory & Leeman, 2021).Consequently, some individuals may rely heavily on GAI to complete all their educational tasks and then attempt to conceal the fact that they are not the original creators of the work.
In this article, I report on two categories of tests conducted using Turnitin.The first category involves AI detection-identifying AI-generated content.The second category examines whether Turnitin can identify plagiarised published human-written content that has been paraphrased or rewritten by AI.

Methodology and findings
In this case study, three tests were conducted to evaluate Turnitin's capability in detecting submissions of AI-generated content (Tests 1 & 2), as well as content plagiarised from a scholarly The numeric results were compared between the ChatGPT essay that was paraphrased by AI (Test 2a) and Turnitin, as well as the control (ChatGPT essay with no paraphrasing) and Turnitin.
For Test 3, the aim was to evaluate Turnitin's similarity index rather than its AI detection capabilities.
In this last test, a published journal article was deliberately plagiarised and then paraphrased by To follow are further descriptions of each test and their results:

Test 1: GAI versus Turnitin AI detection
Test 1-a: Six essays were tested using Turnitin's AI detection tool.The prompts provided to ChatGPT were as follows: • [Essay 1] Write a scholarly essay on second order cybernetics.
• [Essay 2] Write an essay on positivist research methodology.
• [Essay 3] Compose an essay detailing Carl Jung's theory of the unconscious.
• [Essay 4] Explain what teachback is and how it works.
• [Essay 5] Write an essay explaining how conversation theory developed.
• [Essay 6] Write a scholarly essay on the increasing prevalence of solar energy.
Five essays were generated solely using ChatGPT (100% ChatGPT text) which were then individually submitted to Turnitin.The sixth essay was different; this essay was interleaved 7 with human and AI text.In the sixth essay, some complete paragraphs were copied directly from ChatGPT while others were written by a human.Figure 2 shows the snapshot of the submissions.
7 It is worth noting that there is a difference between interleaving text and interweaving text.In this study, interleaving is defined as copying one complete paragraph from an AI source and pasting it after a complete paragraph from a human source.Interweaving text implies the use of AI-generated text and human-written text within the same paragraph.The results for Test 1-a showed that the five essays copied verbatim from ChatGPT were successfully detected as AI-generated by Turnitin's AI detector.Figure 3 illustrates an example of this positive result.The blue highlighting indicates the AI-generated content, which, for this 765-word essay, was detected with 100% accuracy.A summary of the results is provided in Table 1.Overall, for this first test, Turnitin was very effective in detecting AI-generated text even after considering the reduced sensitivity of the interleaved test (essay 6).It should be noted that all tests were conducted in English, and the human text for essay #6 was provided by a native English speaker.Liang, Yuksekgonul, Mao, Wu and Zou (2023) have highlighted that AI detectors may be more likely to label text written by non-native English speakers as AI-generated.
Test 1-b: Since ChatGPT is not the only platform for GAI, a test was also conducted using Google's Gemini.For this test, Turnitin was 100% accurate in detecting an essay submitted that was entirely generated by Gemini.
Test 1-c: "Write like a human…" ChatGPT vs. Turnitin (Additional trials): since the earlier tests were published on YouTube, public comments were posted under these videos.Many people asked the author to perform tests with different prompts to ChatGPT.For example, one user asked for the test to be conducted again, but this time the prompt should be adjusted to one that asks ChatGPT to compose the work in an informal manner.Given the numerous public requests of this nature, additional trials were undertaken whereby the prompts to ChatGPT were "write like a human" and "write in an informal manner".
One example involved asking ChatGPT to write a 2000-word essay on poverty and crime.This essay was then copied and pasted into a new chat with the prompt "rewrite the text like a human."The output from ChatGPT was subsequently run through Turnitin, which detected 91% of the text as AI- The user may also shorten and simplify the copied text which oftentimes provides a complete rephrasing of the whole paragraph.Although the specific AI detection algorithms used by Turnitin are not publicly disclosed, the initial findings of this study align with Sadasivan et al. (2023), who have noted that AI detection becomes problematic when AI-generated text is paraphrased.
Since Tests 1 and 2 focused on evaluating Turnitin's AI detection capabilities, no similarity scores were analysed for these tests.The next test category will address the similarity index.

Test 3: Paraphrased plagiarised human text versus Turnitin
This test aimed to determine if Turnitin could identify that text was plagiarised but paraphrased by an AI paraphraser.
Method: Pages from a published journal article were copied and pasted into QuillBot's paraphraser.
The plagiarised but paraphrased text was then run through Turnitin's similarity checker (Test 3-a).A control test was undertaken whereby the copied pages from the journal article were also run through Turnitin, but these pages never had any AI paraphrasing; thus, the control test was a completely plagiarised submission to Turnitin (Test 3-b).
Figure 6 shows the summary of this test and the sample results for both the AI-paraphrased text and the control.The control test revealed that Turnitin successfully detected the submission as wholly plagiarised.Turnitin was highly accurate in identifying that each line and paragraph was a direct copy from a published source.The similarity score was 99%, as indicated by the green circle for Test 3-b in Figure 6.Additionally, Turnitin correctly identified the source of the plagiarised text.The AI detection score for this submission was zero, which was accurate given that the plagiarised article was written by a human.
The plagiarised but AI paraphrased submission, however, had a completely different result.The similarity score dropped from 99% to 31%.Turnitin's results page displayed minimal and discontinuous red highlighting, with the source list including references unrelated to the original article.This implied that there was minimal semantic analysis, or possibly, if there was, it was not working well.
SOTL in the South 8(2): August 2024 Baron Although Test 3 was designed to assess similarity checking, Turnitin also provided an AI detection score.Since the plagiarised text had been run through an AI paraphraser, it was interesting to note whether Turnitin would detect that.Turnitin was unable to detect any AI writing and thus gave it a 0% score, even though it had been paraphrased by AI.Therefore, after the initial paraphrasing of the copied journal text, I took that paraphrased version and paraphrased it again, resulting in two layers of paraphrasing of the original text.The outcome was that after the second paraphrasing by QuillBot, Turnitin's similarity index dropped to 1%, a significant decrease from the initial 31%.Turnitin no longer identified the original source from which the text was copied.The AI detection score remained at 0%.The results for this twice paraphrased but plagiarised text are shown in Figure 7, with a summary provided in Table 2.

Textual analysis of QuillBot's paraphrasing quality
One might question whether AI paraphrasing tools like QuillBot produce useful output, specifically in terms of preserving the original ideas and maintaining grammatical correctness.As the original coauthor of the article that was deliberately plagiarised for this test, I was able to assess whether the AI paraphrasing retained the core ideas and whether the resulting sentences were coherent.
SOTL in the South 8(2): August 2024 Baron Upon reviewing the output from QuillBot's first attempt at paraphrasing, I found that it successfully preserved the meaning of the paragraphs.If this paraphrased text were submitted for assessment, I would not have concerns regarding grammar.However, the second attempt at paraphrasing yielded sentences that were occasionally confusing and lost some of the original meaning, though this was more the exception than the rule.The second round of paraphrasing introduced some significant changes.However, it is important to note that substantial changes can also occur with the first paraphrasing attempt, depending on the paraphrasing settings chosen by the user, as illustrated in universities to align with decolonisation goals, there currently does not exist a decolonisation and transformation strategy for private HE institutions (PHEIs).The private sector's role in education is becoming increasingly important, with a higher growth rate than that of the public sector.

First paraphrase:
The topic of decolonizing HE has garnered significant attention and endorsement inside public institutions in South Africa.Although public universities are already implementing curricular and other adjustments to fit with decolonization aims, PHEIs currently lack a decolonization and transformation plan.The private sector's involvement in education is progressively gaining significance, exhibiting a greater pace of expansion compared to the public sector.

Second paraphrase:
There has been much focus and support among public institutions in South Africa around the subject of decolonizing HE.PHEIs now lack a decolonization and transformation strategy, despite the fact that public universities are already adopting curricular and other reforms to align with decolonization objectives.The role of the private sector in education is becoming more significant, demonstrating a faster rate of growth in comparison to the public sector.
When reading the three versions of this short paragraph, the author's points are still clear.

Undetected flagrant plagiarism
The key finding from this study highlights a concerning issue: it is alarmingly easy to plagiarise scholarly work without detection by Turnitin.It seems that as AI technology evolves, detecting and differentiating between human and AI-generated text may become increasingly difficult.This represents a significant challenge for universities and academia, where plagiarism has already been a major concern (Vrbanec & Meštrović, 2017).With the emergence of new techniques to bypass established plagiarism checkers, academic misconduct may increase to unprecedented levels.For example, according to the European Network for Academic Integrity (Tauginienė et al., 2019:80), similarity checking software is "highly recommended" and should be incorporated into the institutional plagiarism policy.Mphahlele andMcKenna (2019:1080) claim that "most universities around the world use text matching software", of which Turnitin is one of the most popular tools (Bruton & Childers, 2016;Halgamuge, 2017:896) as an approach in mitigating plagiarism.Some have reported that in India, a similarity index exceeding 60% can lead to the cancellation of registration, though such rigid rules are subject to controversy (Tauginienė et al., 2019).
Having served as the chair of an academic disciplinary committee, I observed numerous cases where students were expelled based on Turnitin reports.However, with the advent of AI paraphrasing tools, the distinction between expulsion and evading detection can often come down to a mere 30second copy-and-paste paraphrasing activity.This troubling situation is untenable for at least four reasons: Firstly, widespread cheating undermines the credibility of academic institutions (Iqbal et al., 2021).
Secondly, if students perceive that their peers are cheating with AI writing tools, this behaviour may become normalised, potentially creating a "depressive environment" for those unwilling to cheat (Iqbal et al., 2021:605).Thirdly, evidence suggests that students who engage in academic dishonesty may continue to exhibit unethical behaviour in their professional lives (Nonis & Swift, 2001;Carpenter, Harding, Finelli & Passow, 2004).Lastly, while there is no universally accepted policy on plagiarism penalties (Iqbal et al., 2021), assessing the extent of the plagiarism and the student's intent9 are two common criteria for determining a sanction, neither of which is an easy task to conduct.For example, historically, some students fooled Turnitin by plagiarising text but converting it to images, which Turnitin did not detect.Other students hid white characters placed between the black adjoining words, which caused the words to run into one another but the sentence still looked acceptable to the reader.In these two cases, the intention to cheat was clear, and disciplinary actions were swiftly applied without needing to analyse the Turnitin similarity index.
However, if a student copies paragraphs verbatim from a published source or ChatGPT and then runs them through an AI paraphraser, the penalty may be less severe, as it is difficult to determine the extent of the plagiarism.For the parts that can be identified, the student could claim they simply forgot to cite that section of the work.The onus is now on the lecturer to demonstrate that the student had the intention to cheat, which is a much more onerous and time-consuming task.
Previously, in cases of a student copying work from other sources, the Turnitin report would be the main part of the lecturer's evidence, with the onus on the student to defend themself.However, SOTL in the South 8(2): August 2024 Baron when the plagiarised text is rephrased, as demonstrated in Test 3, the difficult part is locating the original source article(s).Thus, to bring academic misconduct charges against a student who paraphrased and/or summarised without citations (or even with incorrect citations), the lecturer would need to spend hours trying to piece together a jigsaw puzzle10 .
If students use ChatGPT's responses in place of their own synthesis and expression of their knowledge, Turnitin should ideally be able to detect this, as advertised11 .However, the findings from this study indicate that Turnitin accurately detects AI-generated text only when it is copied directly from ChatGPT or Google's Gemini without any additional paraphrasing.When AI-generated text is mixed with human content, Turnitin's accuracy in detecting the AI-generated portions declines.If students interleave their own content with text generated by ChatGPT, Turnitin seems to struggle with distinguishing between the AI-generated and human text.
This issue has significant implications for educators who rely on assignments and other text-based submissions for grading.It also raises concerns for scholarly work regarding the attribution of new ideas and the synthesis of academic content.The involvement of AI introduces a grey area in defining intellectual property rights (Lin, 2023).Would the intellectual property be shared, and by whom and how would this even be determined?

Lack of awareness
After conducting these AI detection and plagiarism checking tests, I was alarmed by the weaknesses I discovered in Turnitin and decided to share the results on YouTube12 .I also shared the videos with educators, but it became clear that the people who watched the videos were mainly students since the comments and discussions under the videos were mostly about other ways to trick Turnitin.
Recognising that my videos were having the unintended effect of promoting ways to trick Turnitin, I removed many of them, leaving only those I deemed most useful for educators.However, SOTL in the South 8(2): August 2024 Baron ultimately, it did not matter as there are now plenty of videos on this topic with openly candid titles like "How to Bypass Turnitin AI Writing Detection" or "How to Trick Turnitin".What was surprising was that it seems as if many educators are largely unaware of this scourge.Thus, educators have a critical role to play in this problem.

Recommendations
Relying solely on Turnitin and other detectors gives a false sense of safeguards Postgraduate work, while also meant to show evidence of learning, is still wholly text based.
Academics publish most of their research in journals and books, and many academics are under immense pressure to publish.It is thus unsurprising that plagiarism is not confined to students (Santoso & Cahaya, 2019), and copying someone else's work might seem like an option for those who cannot meet their academic commitments.One could copy a published article, make some changes (paraphrasing), and still get published.No human can track the hundreds of publications in each field, so editors must rely on software to find similarities.If this software-based plagiarism checking does not reliably detect plagiarism, the task becomes increasingly difficult for the editor.
However, not all students and academics use AI writing tools, and Turnitin still provides some identification of copied work.Thus, similarity checking retains its place as a first line of defence, although deeper analysis of suspected plagiarised work is still required.
Plagiarism detection software might also improve, but until then, as Ellul (1964) stated, technology waits for no one.If students refrain from using AI assistance, they will fall behind the curve as AI has much to offer in terms of increasing the capacity for expanded work packages.Attempting to block AI is like blocking a student's tutor.In turn, many educators have embraced the presence of GAI by adapting their teaching approach to align with AI (Trust, Whalen & Mouza, 2023;Elbanna & Armstrong, 2024;Yu, 2024).Some universities 13 have even disabled AI detectors for fear of false accusations being raised against students and have warned lecturers not to use AI detector results as grounds for charges of academic misconduct.Thus, if educators are still solely relying on AI detectors and similarity scores (in their current form), they are fighting a losing battle and have probably framed their courses in a manner that may inadvertently support cheating.The original goals of SOTL in the South 8(2): August 2024 Baron education have not changed-we still aim for proof of learning-however, many popular assessment methods are now inept.Educators could rethink their approach and design their curricula with a focus on in-person demonstrative or evidence-based evaluations.
Should we reconsider what constitutes original work in a world of chatbots and large language models (LLMs)?
It seems probable that we are not far off from LLMs indexing scholarly publications and then competing for customers, as companies are already using generative pre-trained transformers (GPTs) to scan proprietary databases.For example, Morgan Stanley14 uses GPT-4 to scan hundreds of thousands of pages of their financial reports, commentaries, and market research data.Their chatbot is then used internally as a point of reference.According to the head of their analytics division, the chatbot is like the most knowledgeable "person" in wealth management 15 .
At the time of writing, the free version of ChatGPT does not provide responses with in-text citations.
However, having used Microsoft's Copilot 16 , I found that it was able to provide some scholarly sources for certain searches in its conversational responses.This suggests that ChatGPT could potentially provide in-text citations if it was allowed to, but it appears to be programmed not to.
Academia, like any other profession, is monetised.If a company used a GPT to specifically scan and learn from scholarly sources, these companies could market their products to students, academics, and universities.It is likely that such a system could provide a literature review or even scholarly research outputs on a chosen topic with citations.If this is indeed the case, there are a few questions worth asking: 1. Would a literature review generated by AI constitute intellectual property if it is only a cited synthesis and summary 17 ?2. Would it be considered plagiarism if AI was used to rewrite (paraphrase) a human's literature review, while keeping the citations intact, considering the original literature review was only a summary and synthesis of other people's work in the first place?
SOTL in the South 8(2): August 2024 Baron 3. If AI can learn from scholarly articles, would a meta-analysis or systematic review of previous publications be accepted for scholarly publication if they were generated by AI? a.If this same AI copied this published meta-analysis to use elsewhere, would this be classified as self-plagiarism?4. When AI can perform in-depth scholarly inquiries with modelling and results similar to humanauthored articles, will such articles be published alongside those written by humans?a.Will the AI-generated knowledge be owned by the tech company who provides the AI?
These questions underscore the need for updated academic policies and ethical guidelines that address the unique challenges and opportunities posed by AI in research and education.However, the reason for posing the above questions is based on a deeper question: what is going to be considered original work in a world where AI can assist humans in composing written works, music, art, movies, and so forth?While a journal should not accept any manuscript with a copied literature review, it will not be long before AI can generate a literature review in a matter of seconds, making summaries and synthesis a menial task.There is currently no system in place that can accurately determine what portion of work is human-generated and what is AI-generated.Humans are already using AI to generate music, and some publish this as their own work, in a sense plagiarising from AI.
The reverse is also possible; students and academics may use AI to reformulate creative works that have already been published, but owing to the changes to the original, these reformulations go unnoticed.It is incredibly difficult to determine what the AI part was unless AI platforms log and track every single output from the start of their content generation.Even if platforms like ChatGPT store every output, accessing such data to accuse the student would infringe on their privacy.
It is also worth noting that GAI uses existing knowledge as the basis for its learning.Thus, using AIgenerated content does not necessarily guarantee enough newness to be considered original.As there is an increasing amount of AI-generated content on the internet, the secondary source might be another AI, while the primary source is a human.A possible solution to this problem might rely on the very thing that is giving rise to the problem in the first place.To aid in tracking the generation of knowledge, I am proposing the use of LLMs as described next.

Idea protection using LLMs
Ideas that introduce novel contributions need to be protected.A patent, for example, confirms a novel aspect of a process.A patent can only be registered after it has been verified that there is enough distinction between the proposed new process or method that implements the idea to what SOTL in the South 8(2): August 2024 Baron already exists.Only once the patent office has approved the patent can it be granted.Scholarly articles, however, can be published irrespective of the uniqueness of the contribution.The uniqueness rests on the knowledge of the authors, reviewers, and editors.There are thousands of journals, and there is just no way to track who did what without hours of research, and even after that, sources could still be missed.I am not suggesting that academic work first be "approved" by a central authority prior to publication as that could interfere with academic freedom.What I am suggesting is that by using LLMs, the system could provide a uniqueness score.Uniqueness in the sense of the ideas put forward, not uniqueness in the sense of how the text differs from what was already published.Currently, plagiarism checkers are good at detecting character-preserving plagiarism (verbatim copying) and syntax-preserving plagiarism (synonym substitution), but they have trouble with semantics-preserving plagiarism.
Since LLMs can "understand" text, it is proposed that such a system could also be used to assist in plagiarism checking.Currently, ChatGPT 3.5 does not directly access specific journals or proprietary databases (OpenAI, 2022).Turnitin, however, does have access to many proprietary databases, which is specifically why educators use it.Thus, what is proposed here is a method to protect the ideas of original works by providing a uniqueness score that demarcates who has already proposed similar ideas, including the date and place of these publications.Since humans are incapable of exhaustively interrogating such a large data source, LLMs would handle this task.Companies like Turnitin could use LLMs to scan the published works on the internet, to which they already have access, and instead of only focusing on text similarity, they should focus on the "understanding" of the text to discern the uniqueness of the content's ideas, arguments, methods, etc.Thus, student submissions and new research articles would be scanned not only for text similarity but also for idea similarity.I am proposing an idea similarity score or idea matrix, which is provided by Turnitin and other plagiarism checkers.This means that once a new submission is sent through Turnitin, the results page will also have a list of authors who have already been published on this topic with a synopsis of those authors' contributions.This also makes it easier for educators to track any plagiarism as they could be provided with authors who have worked on similar topics.A weakness of this approach of using LLMs for idea mining is that once the GAI has full unfettered access to scholarly works, it might also start publishing … ISSN 2523-1154 SOTL in the South 8(2): August 2024 Baron

Concluding remarks
The deep integration of AI into academia seems unavoidable as AI is already on track to explain much of the internet to students (Baron, 2018:337).It is thus important to revise academic policies and rethink curriculum outcomes and assessment practices.This will be a significant challenge for schools and universities; however, that may not be the biggest hurdle yet.Institutions of higher learning are traditionally one of the main places where knowledge is generated.However, popular AI platforms are offered by tech companies, not universities.Running AI models such as ChatGPT requires powerful hardware, software, data, and expertise, which are beyond the scope of a university.Even universities that are leaders in AI research cannot compete with the infrastructure and network requirements that Big Tech has in place.Big Tech already has many GAI chatbots running as part of their product offerings, and it seems likely that GAI will increasingly become part of Big Tech's product portfolio, potentially at odds with educational institutions.
The capacity for GAI to improve is significant and imminent.When GAI gains access to scholarly publishers such as Elsevier, Emerald, Springer, Wiley, and so forth, its level of reasoning and scope could become unprecedented.Compiling a literature review traditionally takes a human many hours or even days, but AI could accomplish this in seconds, including the simulation of complex physical systems, chemical reactions, or biological processes.This would make much of academic research and review a rudimentary task for powerful AI.As GAI evolves, learning from scholarly databases and other proprietary corporate databases, much of the new knowledge produced might be owned by Big Tech, which is generally not inclined to share its knowledge.Tech firms compete in a domain of product development, market share, and intellectual property, all of which underscore Ellul's Law of Technique (1964).This law posits that technology evolves according to its own logic and principles of efficiency, suggesting that technological advancements are driven by the demand for efficiency and effectiveness rather than by social or ethical considerations.Consequently, the future of knowledge production might be dominated by technology companies rather than universities.This has dystopian consequences for academia.It might be time to take Heidegger's (1977:4) point about technology more seriously: Everywhere we remain unfree and chained to technology, whether we passionately affirm or deny it.But we are delivered over to it in the worst possible way when we regard it as something neutral; for this conception of it, to which today we particularly like to do homage, makes us utterly blind to the essence of technology.
human source but paraphrased by an AI writing tool (Test 3).The tests were designed to evaluate two of Turnitin's features: AI writing detection, as well as text similarity; however, these features were assessed separately.ISSN 2523-1154 SOTL in the South 8(2): August 2024 Baron In the first test (Test 1-a), Turnitin's AI detection was assessed by submitting five ChatGPT-generated essays through Turnitin, which were then numerically evaluated based on Turnitin's AI detection percentage.The results were tabulated and quantitatively analysed.A smaller version of this test (Test 1-b) was then conducted using an AI-generated essay obtained from Google's Gemini, rather than ChatGPT.This test involved only one essay.Thereafter, a third attempt to test Turnitin's AI detector, which relied on data points from the public (Test 1-c), was undertaken.Variations in prompts provided to ChatGPT, based on comments from YouTube on how to generate more "human-like" responses, were used to produce additional essays.Thus, an additional five essays were run through Turnitin's AI detector based on the qualitative data extracted from YouTube comments with the aim of "humanising" ChatGPT's responses.In total, 12 essays were submitted through Turnitin's AI detector across Tests 1-a, 1-b, and 1-c, each with variations in the texts used.For Test 2, an AI writing tool (QuillBot) was used to paraphrase AI-generated text from ChatGPT.The aim of this test was to determine if Turnitin could still detect AI writing if it had first been paraphrased by an AI writing tool prior to being submitted through Turnitin (Test 2-a).This means that the ChatGPT essay was paraphrased by another AI writing tool (QuillBot) prior to submitting the essay on Turnitin.For Test 2, one trial was conducted, which included an associated control.In the control part (Test 2-b), the same ChatGPT essay was submitted to Turnitin without any paraphrasing.
QuillBot before being submitted to Turnitin.A control was also used, where the same text was submitted to Turnitin without any paraphrasing.The outcome of Test 3 was quantitatively analysed based on Turnitin's similarity percentage.Additionally, a qualitative assessment was performed by reviewing the Turnitin report to determine whether Turnitin could still identify text similarity after AI paraphrasing.The three tests were thus separated into two categories.The first category aimed to determine whether Turnitin could detect if the text was AI-generated.The second category assessed if Turnitin could locate similarity when the text was copied from a human source but paraphrased by AI.The tests are as follows: ISSN 2523-1154 SOTL in the South 8(2): August 2024 Baron AI Detection Category: Test 1: GAI versus Turnitin AI detection Question: Can Turnitin detect text generated by ChatGPT and Google Gemini?a) ChatGPT vs. Turnitin (six essays) b) Google Gemini vs. Turnitin (one essay) c) "Write like a human…" ChatGPT vs. Turnitin (five essays) Test 2: GAI text paraphrased by an AI paraphraser versus Turnitin's AI detection Question: Can Turnitin detect if text generated by ChatGPT has been paraphrased by an AI writing tool?a) ChatGPT + AI Writing Tool (QuillBot) vs. Turnitin b) ChatGPT vs. Turnitin (Control test) Similarity Checking (Plagiarism) Category: Test 3: Paraphrased plagiarised human text versus Turnitin Question: Can Turnitin identify plagiarism if text is copied directly from a journal article but has been paraphrased by an AI paraphraser?a) Plagiarised human writing + AI writing tool (QuillBot) vs. Turnitin b) Plagiarised human writing vs. Turnitin (Control test) Figure 2: Screenshot of the six essays that were submitted to Turnitin via the Blackboard platform

Figure 3 :
Figure 3: Screenshot showing the successful AI detection (100%) by Turnitin for the first AIgenerated essay

Figure 6 :
Figure 6: A symbolic representation showing the results of the two tests: a paraphrased plagiarised submission and the associated control test

Figure 7 :
Figure 7: The similarity and AI detection results for a plagiarised article that was paraphrased twice

Figure 4 .
Below is an example of the paraphrasing process: first, the original text is provided, followed by the first paraphrased version, and then the second paraphrased version.Original: Decolonisation of HE is an important subject which has gained much support in public universities in South Africa.While curricular and other changes are currently being implemented in public SOTL in the South 8(2): August 2024 Baron institutions have even prescribed Turnitin's use (Sol Plaatjie University (SPU), 2020; University of Cape Town (UCT), 2023:17; UJ, 2021:4; University of Pretoria (UP), 2019:2) and coded this into their plagiarism and academic misconduct policies.Additionally, Turnitin similarity reports are frequently used as evidence in disciplinary cases (North-West University (NWU), 2021:3).Some universities SOTL in the South 8(2): August 2024 Baron even establish strict maximum allowable similarity percentages.For instance, Chandere et al. (2021) Sigmund Freud's psychodynamic model, would Turnitin be able to detect the original sources of ChatGPT's responses?In other words, does ChatGPT provide answers that are directly copied from original sources?If a student then copied ChatGPT's responses verbatim, would a plagiarism checker 1 detect this?The answer to these questions is based on the researcher's own tests-as published on YouTube 2 in 2023, which found that ChatGPT does not provide copied text in its responses in the sense that it does not provide verbatim regurgitated text from its original sources.This speaks to the point that ChatGPT's model is generative, and in so doing, it formulates its responses in a unique manner.While it may still be plagiarism if ChatGPT's response do not have citations, the output is novel 3 in the sense that ChatGPT formulates and provides somewhat unique text.A follow-up question: what happens if other students insert the same prompts into ChatGPT with the aim of answering the same questions?Would ChatGPT provide the same responses to the different students?For example, if several students were all given the same assignment on Freud's psychodynamic model and each student prompted ChatGPT with the same questions, would Turnitin or use Turnitin in some capacity.For example, iThenticate is owned by Turnitin LLC, and upon reviewing the websites of PlagScan and Unicheck, both show affiliation with Turnitin under their logos.While new online tools emerge from time to time, it appears that Turnitin is one of the oldest and most widely used tools for plagiarism checking in HE capabilities, and user interfaces.They are predominantly used in higher education (HE), secondary schools, and for academic publishing.It is worth noting that some of these online tools are either

Table 1 :
Results for the six essays tested with Turnitin's AI detector in Test 1-a