How to find pictures by describing them
- Add your pictures. Drop a folder or many pictures anywhere on the page, or click the box, or use Choose a whole folder instead. Subfolders are included, and files that are not pictures are skipped. JPEG, PNG, WebP and iPhone HEIC photos work.
- Wait while they are read. The pictures appear at once and are read one after another. The first time, the page also downloads the model (see below). You can search as soon as the first pictures are read, and the results update as more arrive.
- Describe what you are looking for and press Enter, or press one of the example buttons. The best matches come first, numbered #1, #2 and so on.
- Choose how many to take. The slider Select the best picks the top few (12 to start with). Click a picture to add it to the selection or take it off.
- Use the selection. Download selected (ZIP) saves the original files, unchanged. Copy the file names puts their paths on your clipboard, one per line, best first. Open under a picture shows the original in a new tab.
Nothing is moved or deleted: the page cannot touch your folder. It makes a ZIP or a list for you to use.
What to type
Short descriptions of what the picture shows work best, in English: “a dog”, “a red car”, “a person smiling”, “food on a table”, “a city street at night”, “a black and white photo”, “a drawing”, “a snowy mountain”. A single word is fine; in our test one word found the right kind of picture about as well as a full sentence did.
- Things it does well: the main subject, the scene, the colors, and the kind of picture (photo, drawing, painting, screenshot).
- Things it does badly: counting (“three dogs” finds dogs, not three), reading text in a picture (use Image to Text), telling close breeds or models apart, and anything small or at the edge of the picture.
- Names of people: do not expect it to find a particular person. It is not made for that, and the page does not try.
Find similar
Under every picture there is a Find similar button. It searches with that picture instead of words: the picture itself comes first, then the pictures that look most like it. This is a quick way to collect all the pictures of one scene or one subject once you have found one of them. It is not a duplicate finder: for exact copies and resized copies see the duplicate finder, and to sort a whole folder into groups use Group Similar Images.
How well does it work?
We tested it on 80 photographs from Wikimedia Commons: eight each of cats, dogs, beaches, mountains, cars, flowers, pizza, bridges, birds and trains. We typed each kind as one word into the page and looked at the eight best matches:
- The top result was the right kind of picture for all 10 words.
- On average 90 of every 100 pictures in the top eight were right. Cars, flowers, pizza and trains were perfect (eight of eight). Cats, dogs, beaches and birds got seven of eight, and bridges and mountains were weakest at six of eight.
- Find similar is less sharp than words. Of the seven nearest pictures to each of the 80, on average 4.7 were of the same kind. The most common mix-ups were dogs and cats, bridges and mountains, and cars and trains: pictures that look alike but are different things.
These are our numbers on a small, tidy test set of ordinary photos. A folder of your own pictures will behave differently, so look at the results before you rely on them. The model looks at the middle square of each picture, cut from the center, so something at the far edge of a wide photograph is not seen.
Speed, the model and the download
The page uses CLIP (the ViT-B/32 model), made by OpenAI, in a compact 8-bit form of the files converted by the transformers.js project. It turns a picture, or a piece of text, into 512 numbers, so that a sentence and a picture that mean the same thing end up close together. It runs in this tab through transformers.js and ONNX Runtime. OpenAI's CLIP repository is under the MIT license.
- Download: about 146 MB the first time (85 MB for the picture side, 61 MB for the text side), kept by your browser.
- Speed: about a quarter of a second per picture on a laptop (80 pictures took about 20 seconds on an Apple M1 Max laptop that was busy with other work). A folder of 1,000 takes a few minutes. Leave the tab open.
- Size: up to 6,000 pictures at a time. The compact files are what keeps the download small. We compared them with the full-size files on the same 80 photos: they found the right kind about equally well (91 against 88 of every 100 in the top eight for one-word searches).
What to keep in mind
OpenAI's own model card for CLIP says that it struggles with fine-grained classification and with counting objects, that its researchers found significant disparities with respect to race and gender, and that the model was not purposely trained or evaluated on languages other than English. It also says that deployed use of the model is out of scope without thorough testing. We take that to mean: use this page to sort and find your own pictures, check what it returns, and do not use it to judge or identify people.
Privacy
The searching happens in this tab. For what the site collects, see our privacy policy.
Frequently asked questions
How can it search pictures that have no tags or names?
The model has learned to turn pictures and sentences into the same kind of numbers, so a sentence can be compared with a picture directly. Nothing has to be written about the pictures first.
Does it work in other languages?
Write in English. The model was trained on mostly English text, and OpenAI says its use should be limited to English.
Why did a search for a dog bring up cats?
Pets look alike to the model. Of the seven pictures nearest to a dog in our test set, about two or three were cats. Making the description more specific (“a dog on a leash”) helps a little, and the slider lets you take fewer pictures.
How many pictures can I search?
Up to 6,000 at a time. They are read at about four a second on a laptop, so a large folder takes a few minutes.
Will it move or delete my pictures?
No. A web page cannot change your folder. It shows the best matches, and you can save them as a ZIP of copies or copy their file names.
Does it read the text in a picture?
No. It sees pictures the way a quick glance does, so it cannot read a sign or a screenshot reliably. Use Image to Text for that.