The AI detector that its own maker switched off

When ChatGPT arrived in late 2022, schools and employers wanted a way to tell human writing from AI writing. Detector tools appeared quickly. On 31 January 2023 OpenAI, the company behind ChatGPT, released its own AI Text Classifier, and said it was not fully reliable.

In OpenAI's own tests on English text, the classifier correctly flagged only 26% of AI-written text as likely AI-written. It wrongly flagged 9% of human-written text as AI. On 20 July 2023 OpenAI switched it off, citing its "low rate of accuracy".

In 2023 Stanford researchers led by Weixin Liang tested seven popular detectors. On 88 essays by US eighth-graders, the detectors were nearly always right. On 91 essays written by non-native English speakers for the TOEFL English test, they wrongly labelled 61% as AI-written on average, and 89 of the 91 were flagged by at least one detector. When the researchers asked ChatGPT to rewrite AI essays with fancier language, detection dropped sharply.

Many detectors work partly by measuring how predictable the wording is, since AI tends to choose likely words. But plain, simple wording is also how many people write in a second language. The study's authors linked the false alarms to exactly this: less varied vocabulary looks "predictable". Meanwhile, a writer using AI can make text less predictable with one extra prompt. So the tool penalises honest writers and misses careful cheaters.

The AI angle: a detector is itself an AI system that has learned patterns, and it reports a confident-looking score that is often wrong. The myth is that a number from a tool settles the question. It does not. When OpenAI launched its classifier, it said itself that reliably detecting all AI-written text is impossible.

The idea behind this caseLesson: How machines learn →

1. 1Never treat a detector score as proof. At most it is a reason to have a conversation.
2. 2Judge work by what the person can explain: ask them to talk through their ideas, sources and drafts.
3. 3Be extra careful with second-language writers, whose plain style is more likely to be wrongly flagged.
4. 4Set clear rules in advance about when AI help is allowed, so the question becomes whether the rules were followed, not what a detector guessed.
5. 5If a chatbot tells you a detector is accurate, ask for independent tests, including the false-positive rate on human writing.