Duplicate image finder

Quick start

1. Add your images. Drag images or a whole folder anywhere onto the page, or click the drop box to choose files. Subfolders are included. Grouping starts right away.
2. Look over the groups. Each numbered group holds images the tool considers alike. Everything else goes into "Unique Images" at the bottom.
3. Pick how alike. Choose Exact copies, Resized or re-saved (the default) or Similar-looking. The groups update by themselves. Do this before you start choosing, because regrouping re-checks every box.
4. Uncheck the copies you don't want. Every image starts checked. Click a thumbnail to toggle it.
5. Download Selected. The checked images go into deduplicated_images.zip, unchanged and under their original names.

Building a training set? Remove duplicates before you color-correct or tag. Our dataset cleaning workflow explains why.

The three "How alike?" choices

These three use the same method and a 32 px comparison. Anything finer lives under Advanced settings, which is closed by default and is described next.

Advanced settings: what each one does

First, the tool computes a SHA-256 hash of every file. Byte-for-byte identical files are dropped and only the first copy is kept. The rest are compared visually with one of three methods. Our guide to how perceptual hashing works covers the mechanics. This is the practical version.

Comparison method

All three methods look at a grayscale thumbnail squeezed into a square. They judge the arrangement of light and dark, not color or exact pixels. DCT + Euclidean copes best with resizing and recompression, and we suggest it for most jobs. Average Hash + Hamming is the classic bit hash, but simple images such as plain gradients match by chance more often.

Likeness threshold

Images are grouped when their distance is at or below the threshold, so lower is stricter. In the Euclidean modes, both lists of numbers are normalized first, which turns the distance into a measure of correlation. In DCT mode the default of 3 asks for a correlation of about 0.93 across the 63 coefficients; the perceptual hashing guide tabulates the other values.

That default is deliberately strict: it groups resized, re-saved and recompressed copies but leaves merely similar pictures apart. Raise it one step at a time if you also want light edits grouped. From about 4 or 5 upward, unrelated images with a similar layout start to match. In Hamming mode the threshold counts differing bits, and it doesn't scale with resolution. 5 bits is about 8% of the hash at 8 px but under 2% at 16 px.

Hash resolution

This is the thumbnail size in pixels. For the average-hash methods it sets how many values are compared: 64 at 8 px, 256 at 16 px. More values catch finer differences, and in Average Hash + Euclidean they also stretch the distance scale, so the same threshold is much stricter at 16 px. DCT mode always keeps the same 63 coefficients, so resolution matters far less there. The default of 32 works well.

Recommended starting values

These are starting points, not rules. Check a few groups and adjust one step at a time.

A change in a small part of the frame, like a different facial expression on the same drawing, barely moves the hash. Expect such alternates to group together even at strict settings, and choose between them by eye.

Reading the groups and choosing keepers

Grouping is a single greedy pass. The first image not yet in a group becomes an anchor, and every later image within the threshold of that anchor joins it. So:

• The first image in a group is just the earliest file, not necessarily the best copy.
• Matches don't chain. Two members are each close to the anchor but may differ more from each other. A near-copy of a member can land in another group or in "Unique Images", and a different file order can give different groups.
• An image joins the first anchor it matches, not the closest one.

Each thumbnail shows the filename, pixel size and file size. When choosing a keeper:

• Prefer the largest pixel size, but look too: an upscaled copy is big without more detail.
• At the same size and format, a larger JPEG was usually saved at higher quality.
• Pass over copies with watermarks, borders or tight crops.
• For training sets, keep a near-duplicate only if it adds something, like a new pose. Near-identical images make one picture count several times.

Fastest method: clear a group's header checkbox to uncheck the whole group, then click the one thumbnail you want.

What it can't catch

Every method compares layouts of light and dark, so some kinds of copy need a different approach:

Large folders and performance

The work runs in three stages, one image at a time, in the page itself:

1. A SHA-256 hash of every file, which reads every byte.
2. Decoding each image and computing its hash.
3. Comparing. Each anchor is checked against every later ungrouped image. With few duplicates that approaches n × (n − 1) ÷ 2 comparisons: about 500,000 for 1,000 images, about 50 million for 10,000.

Each comparison is cheap, but the count grows with the square of the batch. Memory is often the tighter limit, because every image stays decoded at full size, even when it's shown as a thumbnail.

For big collections, work in batches, for example by source. Four batches need about a quarter of the comparisons. Then run a final pass over the combined keepers to catch duplicates across batches. Test settings on a small sample first, since Reprocess repeats stages 2 and 3. In DCT mode, a resolution above 32 adds work but rarely changes the result.

Privacy

Everything happens in this tab. Hashes come from the browser's built-in Web Crypto API, comparisons run on a canvas, and the ZIP is built on your device. Nothing is uploaded, and the files on your disk are never changed. Only your method and slider values are saved, in local storage. Read more about how browser-only tools keep files private or see our privacy policy.