A study conducted by the makers of the leading plagiarism detection service, Turnitin, revealed that over the past year, students submitted more than 22 million essays written with the help of authoring tools such as ChatGPT, Gemini and others as prototypes their work. A trend that many are now saying means the end of education and exams as we’ve known them for decades.
A year ago, Turnitin released an AI writing detection tool that was trained on the multitude of student-written assignments as well as other AI-generated texts.
Since then, more than 200 million documents have been reviewed by the crawler, mostly written by high school and college students. Turnitin found that 11% may contain AI-powered written language in 20 percent of their content, with 3% of total documents reviewed flagged as having 80% or more AI writing. Turnitin says its detector has a false positive rate of less than 1% when analyzing full papers.
The chatbot can synthesize information and deliver it almost instantly—but that doesn’t mean it always gets it right. Generative AI is known to be… hallucinating, creating its own facts and quoting academic references that don’t actually exist.
The AI chatbots being created have also been caught spewing biased text about gender and race. Despite these flaws, students have used chatbots for research, organizing ideas, and writing assignments.
Traces of chatbots have even been found in doctoral theses and journal articles.
Teachers understandably want to hold students responsible for plagiarism, but this requires a reliable way to prove that AI was used in a given task. Experts have from time to time tried to come up with their own solutions to detect artificial intelligence. They haven’t succeeded.
Detecting the use of Gen AI is difficult. It is not as easy as flagging plagiarism because the text generated is still original text. Additionally, there is a difference in how students use gen AI. Some may ask chatbots to write tasks for them (long chunks or whole ones), while others may use the tools as a simple aid.
Students are also not only tempted by ChatGPT and similar models. So-called word spinners are another type of AI software that rewrites text and can make it less obvious to a teacher that the work has been plagiarized or created by AI. Turnitin’s AI detector has also been updated to detect a percentage of such attempts.
Detection tools themselves have a risk of failure. English language learners are more likely to screw them up. A 2023 study found a false positive rate of 61.3% when evaluating Test of English as a Foreign Language (TOEFL) exams with seven different AI detectors. The study did not examine the version of Turnitin.
The company says it has trained its handwriting detector on students who are learning English as well as native English speakers. A study published in October found that Turnitin was among the most accurate of 16 AI language detectors in a test that had the tool examine undergraduate papers and AI-generated papers.
Colleges and universities using Turnitin with access to AI detection software for a free pilot period that ended earlier this year have opted to purchase AI detection.
However, the risks of false positives against English learners have led some universities to abandon these tools for now. Montclair State University in New Jersey announced in November that it would stop using the AI probe. Vanderbilt University and Northwestern University did the same last summer after concerns about potentially biased results from AI detectors, as well as the fact that the tools can’t provide confirmation like they can with plagiarism.
Additionally, Montclair State does not want to impose a blanket ban on artificial intelligence, which will have a place in academia. With time and more confidence in the tools, policies could change.
Academics consistently point out that the use of ChatGPT to produce completed assignments raises ethical and academic problems as it does not promote authentic student creativity and learning. It is important to encourage students to develop their own skills and use information sources responsibly and with respect to the rules of academic good practice.
But who can control AI anymore?
