AI photo detectors compared: what we measured
The short answer
An AI photo detector is a model that learned to separate AI pictures from photographs, and how well it does depends on the pictures you test it on. We ran five public ones on the same 249 pictures: four scored between 0.63 and 0.72 on a standard measure where 0.5 is a coin toss, and the best scored 0.90. Even that one called 25% of real pictures AI and all five real anime pictures AI. Compare any detector by testing it on pictures it has not seen, and look at which kinds it gets wrong.
Detector models are easy to find and hard to judge. Each model card gives a high accuracy, measured on pictures like the ones it trained on. This guide shows what happened when we ran several on one fresh set, so that you can ask the same questions of any detector, including ours in the AI photo detector.
How we compared them
We collected 249 pictures from Wikimedia Commons: 142 made by 18 AI services and generators (ChatGPT, Midjourney, Stable Diffusion, Gemini, Firefly and others) and 107 real ones (phone and camera photos, CC0 photos, paintings, anime art). Each detector scored every picture; we report AUC, the chance that it ranks a random AI picture above a random real one (0.5 is a coin toss, 1.0 perfect). That ignores where you set the cut-off, which makes comparison fair.
| Detector (public, open license) | AUC on our pictures |
|---|---|
| umm-maybe AI-image-detector | 0.63 |
| prithivMLmods AI-vs-Deepfake-vs-Real (SigLIP 2) | 0.63 |
| An anime-focused ViT detector | 0.64 |
| A deepfake-detection ONNX model | 0.69 to 0.72 |
| Ateeqq ai-vs-human-image-detector (SigLIP 2) | 0.90 (0.88 with 8-bit weights, 0.90 with 4-bit) |
The one that did best is the one we use. Its card claims 99.2% accuracy on its own test pictures and notes that some users reported overfitting; that is the gap between a detector's own numbers and a fresh set.
Where the best one fails
- Drawings. It called all five real anime pictures AI and half of four paintings. Detectors learn the smooth look of generated art; real flat-color art shares it. We show no number for such pictures.
- Photo-real services. Firefly (a third caught) and DALL·E (half) are the ones that fool it most.
- Phone photos. 18% flagged: heavily processed phone pictures can look smooth.
What a detector cannot see
It sees the picture squeezed to 224 × 224 pixels. Fine detail is gone, and it knows nothing of the file. That is why we read the file first: a prompt or Content Credentials beats any score, and camera records push the other way. Our blend adds the picture's size and its noise, which helped the score on reposted copies less than on originals (held-out AUC 0.90 against 0.85).
How to compare any detector
- Ask what it was tested on. The maker's own pictures are the best case.
- Test it on a set you know, including real drawings and processed phone photos.
- Look at the error on real pictures, not only the catch rate: a detector that flags everything catches everything.
- Check whether it reads the file. If not, it ignores the strongest evidence.
Run the detector on photos: it reads the file for AI statements and camera records, then scores how AI-made the picture looks. It runs in your browser. Open the AI Photo Detector.
Frequently asked questions
Which AI photo detector is the most accurate?
Of the five public models we ran on the same pictures, the SigLIP 2 fine-tune by Ateeqq, at AUC 0.90. Accuracy on your pictures will differ, and every detector fails on some kinds of picture.
What is AUC?
The chance that a detector gives a higher AI score to a random AI picture than to a random real one. 0.5 means no better than guessing; 1.0 means perfect ranking. It does not depend on the cut-off.
Why do detectors fail on anime and paintings?
Generated art tends to be smooth and clean, and real drawings often are too. A detector trained on photographs and generated images mistakes one for the other.
Can a detector be fooled on purpose?
Yes. Re-saving, resizing, adding noise or editing a picture can change a score. That is another reason not to use a score alone.