ChatGPT Could Not Spot Its Own Sora Videos
In 92.5 Percent Of Tries.

The sales pitch for a modern AI assistant is that it can help you make sense of the flood. Show it a photo, a document, a screen grab, a video, and it will read the thing and tell you what you are looking at. So the media watchdog NewsGuard ran the obvious test. It gathered twenty videos generated with Sora, OpenAI's text-to-video tool, each one built to advance a provably false claim drawn from NewsGuard's own database of tracked misinformation. Then it fed those videos to the three leading chatbots that let users upload clips and asked them, in plain language, whether the footage was real and whether it was made by AI.

The results, published on January 22, 2026, read like a stress test that every product flunked. When the videos carried no watermark, xAI's Grok called them authentic 95 percent of the time. ChatGPT missed 92.5 percent of them. Google's Gemini, the least awful of the three, still failed on 78 percent. These are not the numbers of a tool that occasionally slips. These are the numbers of a tool that does not have the capability at all, dressed up as one that does.

The House Could Not Recognize Its Own Handwriting

Sit with the ChatGPT figure for a second, because it is the part that turns an ordinary study into an indictment. OpenAI owns Sora. OpenAI owns ChatGPT. The video and the detector came out of the same building, from the same research culture, trained by the same people. If any assistant on earth should be able to recognize a Sora clip, it is the one made by Sora's parent. And it whiffed on 37 of 40 tries. The company that manufactures the fake cannot reliably identify the fake, which tells you the detection was never a real feature. It was an assumption everyone made because the chatbot sounds confident about everything else.

This is the recurring theme in every failure documented on this site. The machine does not know what it does not know. Asked whether a video is genuine, it does not pull up provenance data or check a cryptographic signature. It generates the most plausible-sounding answer, and the most plausible-sounding answer to "is this real footage of a real event" is almost always yes, because most footage in its training was real footage. The model is not lying to you on purpose. It is doing exactly what it always does, which is produce fluent text that pattern-matches to a reasonable reply, whether or not that reply is true.

The Watermark Is Not The Safety Net You Think

Here is the reassurance the industry likes to offer. Sora videos ship with a visible watermark, so you can always tell. NewsGuard tested that claim too, and it does not hold. Using a free third-party tool, EZremove.ai, researchers stripped the watermark off the clips in seconds, and a small ecosystem of similar services has grown up specifically to do this. The watermark is a speed bump, not a wall, and it disappears the moment anyone with bad intentions wants it gone.

Even the watermarked versions were not handled cleanly. With the watermark still visible, Grok failed to flag the videos as AI-generated 30 percent of the time, and ChatGPT missed 7.5 percent. Only Gemini caught every watermarked clip. So the defense collapses on both ends. Strip the watermark and the detection rate craters. Leave the watermark on and two of the three assistants still trip over a label that is sitting right there in the frame. The safeguard depends on a mark that is trivial to remove and that the chatbots do not consistently read even when it stays.

Silence Is Its Own Failure

A responsible tool that cannot answer a question should say so. That is the humble, honest response. Instead the chatbots mostly just committed to being wrong. According to the audit, ChatGPT disclosed that it could not reliably detect AI content in only 2.5 percent of its responses. Gemini did so 10 percent of the time, Grok 13 percent. The rest of the time the models simply rendered a verdict, usually the wrong one, with no hedge and no warning that this was outside their competence. A user asking in good faith is not told "I cannot verify this." They are told, in effect, "looks real to me," and they carry that false confidence back into the world.

That gap between capability and confidence is the throughline connecting this study to nearly everything else we track. It is the same mechanism that produces fabricated legal citations that get lawyers sanctioned, and it sits directly upstream of the harm in the deepfake lawsuits now piling up against image and video generators. The models generate confidently and verify poorly, and when the thing being verified is whether a piece of media is even real, that weakness stops being an inconvenience and becomes a public-information crisis.

Why This One Matters More Than A Bad Chatbot Reply

Most AI failures hurt the person holding the phone. This one hurts everyone downstream. Picture the flow of a viral hoax. A fabricated clip of a politician, a disaster, a celebrity, a war crime spreads across a feed. The natural, media-literate instinct in 2026 is to check it, and increasingly people check it by asking a chatbot. If that chatbot returns a false all-clear four times out of five, it is not a neutral bystander to the misinformation. It is an amplifier wearing the badge of a fact-checker. It launders the fake through the authority of a tool people have been told to trust.

The uncomfortable takeaway is that AI-generated video has outrun AI-powered detection, and it is not close. The same industry racing to make synthetic video indistinguishable from reality has not built anything remotely capable of telling the two apart, and in some cases cannot even recognize its own output. Until that changes, the only safe assumption is that a chatbot's judgment about whether a video is real is worth nothing, and that a green light from ChatGPT, Grok or Gemini is not evidence of anything. For the wider pattern of these tools failing exactly where people rely on them most, see our running timeline of AI failures and the full catalog of documented ChatGPT problems.