AI text detectors aren’t working. Is regulation the answer?

Times Higher Education

Tom Williams

August 9, 2023

More regulation could make the job of detecting whether academic writing has been generated by artificial intelligence easier, amid concerns that tools created for this purpose are suffering from low accuracy rates and inbuilt biases.

Universities worldwide have embraced the use of AI detectors to combat the rising concern that the likes of ChatGPT and its successor GPT-4 can help students cheat on assignments, although many remain wary as an increasing body of evidence shows that they struggle in real-world scenarios.

In a paper published in June, researchers based across European universities concluded that “the available detection tools are neither accurate nor reliable and have a main bias towards classifying the output as human-written rather than detecting AI-generated text”. This followed another paper that showed that students whose second language was English were being disproportionately penalised because their vocabularies were more limited than native English speakers’.

A third study from academics at the University of Maryland confirmed inaccuracy concerns and found that detectors could be easily outwitted by students using paraphrasing tools to rewrite text initially generated by large language models (LLMs).

Continue Reading