Can AI detectors spot ChatGPT text? How accurate they really are
Short answer Sometimes, but not reliably. Most detectors guess from word patterns, so they miss edited AI text and flag some human writing. Their makers say so: Turnitin tells users to start a conversation, not draw a conclusion. Blackboard's SafeAssign doesn't check for AI at all. Treat a score as a hint, never as proof.
Want help applying this to your business? Talk it through on a free 30-minute call.
Book a callCan AI detectors really tell if ChatGPT wrote something?
Not with certainty. A detector reads the text and estimates how likely it is that a machine wrote it. It never sees how the text was made, so the result is a probability, and it can be wrong in both directions:
- It misses AI text. OpenAI's own detector, in its launch tests, caught 26% of AI-written text. OpenAI withdrew it on July 20, 2023, citing "its low rate of accuracy" (OpenAI).
- It flags human text. The same OpenAI tool wrongly labeled human writing as AI 9% of the time (OpenAI). Turnitin says some sentences it highlights were written by people (Turnitin).
The tool's makers say how to use the result. Turnitin's chief product officer:
"Use the information to initiate a conversation, not to draw a conclusion." (Turnitin, June 2023)
That's the rule for a business too. A score can tell you where to look. It can't tell you who wrote something.
How do AI detectors work?
There are three kinds of checks, and only one gives a sure answer.
| Check | What it does | Can it prove ChatGPT wrote it? |
|---|---|---|
| Pattern guessing (most "AI detectors") | A model trained on human and AI samples scores how predictable the wording is | No. It's an estimate |
| Text matching (plagiarism checkers) | Compares the text with pages and papers already stored | No. New AI text matches nothing |
| Watermark check | Looks for a hidden signal the AI tool put in its own output | Only for that tool's output, and edits weaken it |
Pattern guessing is what free online detectors and most paid ones do. AI models tend to pick the most likely next word, so their text is smooth and predictable. Plain, simple human writing can look just as predictable, which the Stanford study below links to false flags.
Watermarks are the stronger method, but limited. Google adds a hidden watermark called SynthID to "text generated by the Gemini app and web experience" (Google DeepMind). Its own write-up says confidence "can be greatly reduced when an AI-generated text is thoroughly rewritten" or translated, and that the watermark is "less effective on factual responses" (Google DeepMind and Hugging Face, 2024). A Gemini watermark also says nothing about text from ChatGPT or any other tool.
Can Blackboard or SafeAssign detect ChatGPT?
Not by itself. SafeAssign, Blackboard's built-in checker, uses a "text matching algorithm capable of detecting exact and inexact matching between a paper and source material" (Anthology Blackboard Help). Its help page says nothing about spotting AI writing. Fresh ChatGPT text usually matches no stored source, so it passes.
A school can add other tools. "Turnitin can be integrated with Blackboard LMS," and instructors turn it on per assignment when the school has it (Anthology Blackboard Help). Then the AI result comes from Turnitin, with Turnitin's limits above. Instructors also judge by eye: a paper that doesn't sound like the student's earlier work gets a closer look.
How often do AI detectors get it wrong?
More often than their sales pages suggest, and not evenly.
- Sentence level. Turnitin says its sentence-level false positive rate is "around 4%," and that 54% of those wrongly flagged sentences sit right next to real AI text (Turnitin). A person who edits one AI paragraph may see their own sentences flagged too.
- Whole documents. Turnitin puts its document false positive rate under 1%, but only "for documents with 20% or more AI writing" (Turnitin). That's the vendor's own test, not an independent one.
- Non-native English writers. Stanford researchers found seven popular detectors "consistently misclassify non-native English writing samples as AI-generated" (Liang et al., Patterns, 2023). The same study found "simple prompting strategies" got AI text past them.
So the people most likely to be wrongly flagged are those who write simply. That includes many good business writers, because plain sentences read best on a phone.
For what detector scores mean for Google rankings, and the FTC case against a detector that overstated its accuracy, see AI detectors and SEO.
Should a business owner use AI detectors at all?
Use one, if at all, as a reason to ask a question. Never as the decision.
| Situation | Don't | Do instead |
|---|---|---|
| A freelancer's blog post scores "90% AI" | Refuse to pay on the score alone | Ask for their notes or draft history, and check the facts in the post |
| Your contract bans AI writing | Rely on a detector to enforce it | Spell out what's allowed and require sources for every figure |
| A client says your page "is AI" | Rewrite it until a tool says "human" | Show what only you know: your prices, photos, local details |
| You publish AI-assisted pages | Chase a low AI score | Fact-check every claim; Google judges quality, not origin (see Google's policy) |
What decides whether content works is not who typed it. It's whether the facts are right and whether it shows real experience. Human input in AI content covers the parts worth adding.
Example
A roofer vetting a new writer
The writer's first post scores 85% AI on a free detector. Instead of firing her, the owner checks it: two shingle prices are wrong and nothing mentions local weather. He sends it back with that list. The second draft is accurate and specific. The score never mattered; the facts did.
A bookkeeper accused by a client
A client runs the bookkeeper's welcome guide through a checker and gets "60% AI." She wrote it herself in plain English. She shows the client her dated draft in her documents folder, and the matter ends there.
A cleaning company's blog
The owner drafts posts with ChatGPT, then adds his own checklists and photos. He stops running posts through a detector and spends that half hour answering one more customer question instead.
FAQ
Can ChatGPT tell if it wrote something?
No. OpenAI withdrew its own detector in 2023 for low accuracy (OpenAI). Asking ChatGPT "did you write this?" gets a guess, not a record.
Do AI detectors work on short text?
Worse. OpenAI called its classifier unreliable under 1,000 characters (OpenAI). Most ad copy and product blurbs are shorter than that.
Does editing AI text get it past detectors?
Often. Simple prompting beat the detectors in the Stanford study, and heavy rewriting weakens even Google's watermark (Liang et al.; Google DeepMind and Hugging Face).
Which AI detector is the most accurate?
No independent body certifies any of them, and vendors test on their own data. Turnitin's figures above are its own measurements.
Different for you?
- Worried a detector score hurts your Google ranking: AI detectors and SEO
- Publishing AI-drafted pages: Does Google penalize AI content?
- Your pages lack trust signals: E-E-A-T for a small business
- You want a team to write and check pages: SEO agent