By Linda Cheung
As a trial test, New Hyde Park junior Guranaad Kaur inserted a personal essay she had written into GPTZero. The result: her writing was flagged as 50-60% likely AI written. Kaur’s experience reflects the concern of many students throughout NHP because the fast-paced development of AI language models present opportunities for inaccuracy found across numerous AI detectors.
“I don’t think that AI detection technology is very accurate,” Kaur said. “I would put my essays in just for fun, to test them out. It still flagged my writing as AI written, when in reality I hadn’t used ChatGPT at all because those were some very personal aspects I had to write about.”
Source by Linda Cheung
Junior Fiona O'Reilly tests the capabilities of ChatGPT in writing a human-like essay.
Launched last November, ChatGPT is an AI Chatbot trained using Reinforcement Learning from Human Feedback (RLHF), a technique meant to optimize a language model using human input. AI written text is notoriously difficult to detect (and frequently inaccurate) because it is meant to generate fluent and human-seeming text based on the preferences of trainers regardless of incorrect answers.
Although AI detectors can be a helpful tool for teachers in deciphering authentic versus AI- written text, AI detection technology should be treated as a singular data point and an indication of AI generated text rather than an accusation against a student. To ensure the authenticity of student assignments, solutions should be implemented that judge assignments holistically in a way that places trust in the teacher-student relationship.
Introduced to the English department last December, AI language models have presented a valid and growing issue among both students and teachers. Due to the rising popularity of AI services, English teachers have begun using AI detectors including GPTZero to ensure that students are producing authentic work.
“There’s no AI checker that will 100% for sure tell you that it was produced by AI, but there are certain patterns in the language models that wind up being produced,” English department chairperson Mr. Otton said.
While AI detection technology can be useful in many instances for teachers, various students express concern regarding the accuracy of AI detection tools.
“I think there’s almost definitely issues with AI detection,” senior Paul Wang said. “You need to be really careful with it because even if there’s a small percentage of cases that are false negatives, false positives, that could still translate to a lot of cases. There’s just so many instances of AI being used. It’s bad if someone gets away with it, but I feel like it’s horrible if an AI detector falsely detects a student using AI.”
Additionally, analysts of AI language models have observed that tools such as these can discourage independent thought, originality and creativity. Students may become reliant on AI language models to write entire essays or assignments rather than use the tool as a starting point in the creative thought process. Given the existence of AI language models, teachers question how personal aspects of the creative writing process can be encouraged.
“When a teacher gives you a writing assignment, the goal is to engage with those ideas, externalize your thinking, draft it, revise it,” Mr. Otton said. “This is certainly going to shift that. We’re also trying to figure out how to give you the opportunities to develop into the type of person who’s going to be able to self-actualize, to enjoy their lives, to adapt to changing circumstances, not just in the job market. In terms of society, in terms of the way you feel about yourself, in terms of what you do for fun, interests and how you engage with everything, the world.”
Moreover, others state that the issue may lie within the nature of the assignment itself, rather than the use of an AI language model.
“I think AI is a useful tool, but I don’t think it can completely replace human thought or creative thought,” Wang said. “For example, for a Social Studies class, if the assignment that you’re giving could be written by an AI, that might say something about the assignment more than it does about a student using the AI. The AI doesn’t have the capability to actually think critically, so if there’s an assignment where you can copy-paste something from the internet, and that’s a response that would earn a good score on the essay, that might be a sign that it’s not an assignment that’s worthwhile if it’s just listing a bunch of things you found off the internet, because that’s essentially what the AI is doing.”
Both students and teachers alike present a variety of solutions regarding the use of AI detection software. Some assert that handwritten assignments are the sole remedy to AI generated text.
“The only thing you can do is to have handwritten essays,” English teacher Mr. Stencel said. “I know it’s more work for the teacher, it’s more work for the student, but this has proven to us that there are limitations to our Google plagiarism detector.”
However, others state that a more technologically savvy approach using a combination of AI detection tools and other data points could indicate the presence or absence of AI written text.
“When a student types an essay up in Google Docs, the teacher can go back in the revision history and see exactly what a student added, at what time they added it at, when the document was created and how long they’ve been editing the document for,” Kaur said. “A better solution would use Google Docs, where you can see the revision history actively so you can see whether a student was writing something on their own accord or not; if they went back and made revisions versus if they just copied and pasted something. Teachers just need to make it very clear in their classrooms that this can be a very helpful tool or a very harmful tool depending on how you look at it.”
“The best AI detection tool is the relationship that a teacher has with the student,” Mr. Otton said. “Speaking as an English teacher, the tool is not the first line of defense. They are very explicit about this on the tools themselves; these aren’t perfect. All writers have a particular voice, and you can tell that the voice of the work being submitted is not their own voice. That to me seems like one very narrow piece of this. For teachers, it’s really important for us to make sure that what students are submitting is their authentic work. It will require adjustments, it will require changes.”
Ultimately, AI detection software is one small aspect in a broader attempt to ensure the authenticity of student assignments. Multiple factors could be used to draw these conclusions, including AI detection software, handwritten assignments and a Google Docs version history. The use of AI generated text among students may lie within the nature of the assignment itself, because writing is meant to promote critical thinking, originality and creativity. Although AI can be used as a starting point in the creative thought process, relying entirely on an AI language model for assignments only serves as a detriment to the user.