WD14 tagger models compared: ConvNeXt V2 vs SwinV2 vs ViT

The ImgTagger auto-tagger offers three models in its Model menu: ConvNeXt V2 (the default), Swin V2 and ViT. All three are WD14 taggers published by SmilingWolf on Hugging Face. They take an image and score every tag in a fixed Danbooru vocabulary. They differ in network design, file size and, most importantly, in which tags they know.

This guide compares them using what we could verify: the files themselves, their tag lists and the published model cards. We don't publish speed benchmarks, because tagging speed depends heavily on your device.

The three models at a glance

Download sizes are for each model's model.onnx file. The tag list adds about 254 KB for the two v2 models and 174 KB for ViT. The network names come from the model files: layer names inside the ConvNeXt V2 file read ConvNextBV2, the ViT file's read ViT-B16, and SwinV2's config describes a SwinV2-Base network adapted to 448-pixel input. The validation figures come from the ConvNeXt V2 model card and the SwinV2 model card.

What the network names mean

All three are "Base"-sized versions of well-known image recognition designs. You don't need the details to use them, but the names make more sense with a one-line summary of each:

• ConvNeXt V2 is a convolutional network. It scans the image with small filters and builds up from local detail, such as edges and textures, to whole objects. ConvNeXt modernized this classic design with ideas borrowed from transformers. V2 is a 2023 revision.
• Swin Transformer V2 is a transformer that compares image regions within small windows, and the windows shift from layer to layer. Like a convolutional network, it works from fine detail to coarse structure.
• ViT (Vision Transformer) cuts the image into 16×16-pixel patches and lets every patch attend to every other. At 448×448, that is 784 patches.

On paper these are quite different designs. On this task, the published results for the two v2 models end up almost the same, so the architecture matters less than the tag list and the download size.

Same data, same tags: the two v2 models

ConvNeXt V2 and SwinV2 were trained on the same dataset, and their model cards describe it identically:

• Danbooru images up to post ID 5,944,504.
• Training and validation images were split by post ID number, so validation images were not used for training.
• Images with fewer than 10 general tags were left out.
• Tags with fewer than 600 images were left out. The smallest count in the tag list is exactly 600.

Their selected_tags.csv files are byte-for-byte identical: 9,083 tags in the same order, with the same post counts. Any tag one model can output, the other can too. When they disagree on an image, it is because each network scores the same tag a little differently, not because one knows a tag the other doesn't.

The SwinV2 card also notes a later re-export (v2.1) to work around an ONNX Runtime bug. According to the card, the trained weights did not change, so its published figure still applies. Both cards advise pinning a release rather than tracking the latest files, and ImgTagger does: each model loads from a fixed commit (for SwinV2, the v2.1 export), so the files can't change without a site update.

What the published accuracy numbers mean

Each v2 card reports a single result: the confidence threshold at which precision equals recall on the validation set, and the F1 score at that point. When precision and recall are equal, F1 equals both. So for ConvNeXt V2, at a threshold of about 0.37, roughly 69% of the tags it wrote matched the image's Danbooru tags, and it found roughly 69% of the tags Danbooru had.

Keep three caveats in mind:

• The reference tags are incomplete. Human taggers miss things. A correct prediction that Danbooru never tagged counts as an error, which pulls the score down.
• It is an average over thousands of tags. It says little about how well any single tag works on your images.
• The gap is tiny. 0.6862 versus 0.6854 is a difference of less than a tenth of a percentage point. Treat the two models as equally accurate.

The ViT repository has no model card, so there are no published figures for it. (SmilingWolf later released a separate wd-v1-4-vit-tagger-v2 with its own card. ImgTagger's ViT option is the original model, so those numbers don't apply.) ImgTagger's default threshold of 0.35 sits just below the published balance points. Our confidence threshold guide explains what moving it does.

How the ViT tag list differs

The ViT model predates the v2 pair, and its vocabulary shows it. It has 6,549 tags: the four ratings and 6,545 general tags. Compared with the v2 list:

• ViT lacks 2,560 tags that v2 has: all 2,132 character tags plus 428 general tags.
• ViT has 26 tags that v2 lacks. Several are older names for tags the v2 list spells differently.
• For 6,493 of the 6,519 general tags the two lists share, ViT's post count is the same or lower, about 95% of the v2 figure on average. That fits a list built from an earlier snapshot of Danbooru.

The practical point: don't mix ViT output and v2 output in one dataset without checking for pairs like these. Otherwise one concept ends up split across two caption tags. For a closer look at the everyday tags all three models share, see our booru tag glossary.

Input size and white padding

All three models take a 448×448-pixel image. ImgTagger prepares every image the same way. It centers the image on a white square, resizes it to 448×448, converts the color channels to BGR order and passes the raw 0–255 pixel values. That matches the preprocessing in SmilingWolf's own demo, except that the browser's canvas does the resizing. Three consequences are worth knowing:

• Large images lose fine detail. A 4,000-pixel image is judged at 448 pixels. Small accessories, distant faces and eye colors in wide shots may be too small to read. If those details matter, crop the image into sections in an image editor before tagging.
• Very wide or tall images shrink further. A panorama is scaled until its long side fits, then padded with white above and below. The picture fills only a band of the square.
• Transparency becomes white. Cut-out PNGs are tagged as if they sat on a white background.

Download, caching and memory

When you open the tagger, it starts loading the selected model right away, which is ConvNeXt V2 by default. The first time, that means downloading 365–455 MB from Hugging Face, depending on the model. The files are then stored in your browser's IndexedDB, so later visits load them from disk. You don't have to wait for it: if you click Auto Tag while the model is still loading, the run starts as soon as the model is ready. Choosing another model in the menu loads it immediately, and it is cached the same way. Caching all three takes about 1.2 GB. The "Delete downloaded models" button (under Advanced in the Auto-tag panel) removes them. In a private window, or if the browser blocks site storage, the tagger still works but downloads the model again next visit. Our guide to how in-browser AI works covers the memory and storage costs.

Tagging runs on your CPU through WebAssembly, using a pool of background workers (80% of your CPU threads, from 4 up to 20). Each worker holds its own copy of the model, so a larger model multiplies memory use. SwinV2, the largest file, asks the most of your machine. Larger models also generally take longer per image. If the tab struggles on a laptop or an older computer, ConvNeXt V2 or ViT is the lighter choice.

Which model should you pick?

• ConvNeXt V2, for almost everything. It has the full vocabulary with character tags, the higher published score (by a hair) and a mid-sized download. It is the sensible default for LoRA training captions and for organizing a collection.
• SwinV2, as a second opinion. It has the same tags and essentially the same published accuracy, but a larger download and more memory. Because the two networks make different mistakes, it can be worth comparing them on a sample of your own images. If you run it on images that already have tags, ImgTagger keeps the existing tags and adds only new ones. The result is the union of both models' output, so you get more coverage but also more to prune.
• ViT, when you want no character names. Blacklisting 2,132 character tags one by one isn't practical, and ViT simply never outputs them. That helps with original characters, or when you will add your own trigger word with the Prepend box. The trade-offs are an older, smaller vocabulary and no published accuracy.

Whichever you choose, use one model per dataset. Each model scores tags a little differently, and ViT names some tags differently, so captions stay more consistent when one model produced all of them.

What about the newer v3 models?

SmilingWolf has since released a v3 series of taggers trained on a newer dataset, including wd-vit-tagger-v3, wd-swinv2-tagger-v3 and wd-eva02-large-tagger-v3. ImgTagger doesn't offer them yet. SmilingWolf's demo Space on Hugging Face lets you try them, but it processes your image on a server rather than in your browser.