Booru tags vs natural-language captions: which should you use?

There are two main ways to caption a training image. The first is a comma-separated list of booru tags, the output of WD14 taggers like the one in ImgTagger's auto-tagger. The second is one or more plain English sentences, the output of captioning models such as BLIP or Florence-2. Both work, but they are good at different things, and the right choice depends more on the model you are training than on taste. This guide compares them on one example image and ends with a decision table.

The same image, captioned three ways

Imagine an illustration of two girls on a park bench under cherry blossoms. The girl on the left has short black hair and a red jacket and is reading a book. The girl on the right has long blonde hair and a blue dress and is eating ice cream. The captions below were written by hand to show the typical shape of each format. They are not the output of a particular model.

Booru tags, in the comma-separated format ImgTagger exports:

2girls, multiple girls, sitting, bench, short hair, black hair, red jacket, long hair, blonde hair, blue dress, book, reading, ice cream, holding, cherry blossoms, tree, outdoors, day

A short caption, typical of BLIP, whose captioning models were fine-tuned on the one-sentence captions of the COCO dataset:

two girls sitting on a bench in a park

A detailed caption, the kind you get from a model with a "detailed caption" mode, such as Florence-2's <MORE_DETAILED_CAPTION> task:

Two girls sit side by side on a wooden park bench under cherry blossom trees. The girl on the left has short black hair and a red jacket and is reading a book. The girl on the right has long blonde hair, wears a blue dress and is eating an ice cream cone.

Read the tag list again and try to work out who is wearing the red jacket. You can't. The tags record every ingredient but not how they fit together. The short caption has the opposite problem: it is accurate but leaves out almost everything. The detailed caption has both the ingredients and the structure, at the cost of length.

Where booru tags are stronger

Consistency

A WD14 tagger can only choose from a fixed list. The default model in ImgTagger has 9,083 possible labels. A red jacket is always red_jacket, never "crimson coat" in one file and "red outerwear" in the next. Caption models write freely, so the same thing can be described several ways across a dataset. A consistent vocabulary makes it easier for a model to tie one word to one visual idea.

Counting

Booru tags have dedicated count labels: 1girl, 2girls, solo, no_humans and so on. They are among the most common tags in the whole vocabulary. In the WD14 v2 tag list, 2girls appears on 729,721 posts and 1girl on more than four million, so anime models have seen them constantly. A sentence can say "two girls" just as well, but caption models sometimes miscount, and nothing flags the error.

Editability

Tags are separate items, so you can edit a whole dataset at once. You can remove blurry_background from 400 files, add a trigger word to the front of every caption, or filter to all images tagged outdoors. ImgTagger's tag editor does all of this in bulk. Fixing the same mistake in 400 hand-written sentences means editing each one. The tag pruning guide shows how far bulk cleanup can go.

Per-tag control

A tagger gives every tag its own confidence score, so you can tune what makes it into the caption. Raise the confidence threshold and weak guesses drop out. A caption model gives you finished prose with no score for each claim. If it invents a detail, the mistake reads just as confidently as the true parts.

Training tricks

Many trainers shuffle tag order each epoch so the model doesn't learn that the first tag always matters most. In kohya-ss sd-scripts, --shuffle_caption shuffles the comma-separated parts of a caption and --keep_tokens holds the first few in place. That only works when each part makes sense on its own, which is true of tags but not of clauses in a sentence.

Where sentences are stronger

Attribute binding

"Attribute binding" means attaching each property to the right object: the red jacket to the girl on the left, the blue dress to the girl on the right. Tags can't do this when two subjects share an image. They work best when there is only one character, where every attribute obviously belongs to the same person.

Relationships and layout

Danbooru does have some relationship tags, such as holding_hands, looking_at_another and hand_on_another's_shoulder. They record that something is happening, but not who is doing it. Left and right, in front of and behind, and "the cat on the girl's lap" are all easy in a sentence and mostly impossible in tags.

Things outside the vocabulary

The booru vocabulary grew up around anime illustration. It has very detailed coverage of hairstyles and outfits, but nothing for many everyday ideas. A caption model can name a specific object or describe a mood. A tagger can only use the labels on its list. If your dataset is photos of products, buildings or food, sentences usually describe it better.

Token length: what fits in 75 tokens

CLIP text encoders read at most 77 tokens, two of which are start and end markers. That leaves 75 for your caption. Trainers such as sd-scripts can extend this to 150 or 225 with --max_token_length by encoding the caption in chunks. We counted the example captions above with the original CLIP tokenizer:

Two details are worth knowing. First, every comma is a token, so 17 of the 45 tokens in the tag caption are commas. Second, underscores are tokens too. looking_at_viewer costs 5 tokens and looking at viewer costs 3. That is a point in favor of writing tags with spaces, as ImgTagger does. The caption .txt file guide covers that choice in more detail.

Tags are not automatically shorter than sentences. A sentence spends tokens on words like "the", "is" and "on the left", which carry its structure. Models with a T5 encoder have far more room. The diffusers FLUX pipeline, for example, uses a default maximum of 512 T5 tokens.

Match the captions to the text encoder

A LoRA or fine-tune trains on top of a base model and reuses what it already knows. If your captions use the same "language" as the base model's original training captions, the new concept attaches to words the model already understands. If they don't, part of your training is spent teaching the model a new caption style.

The Animagine XL 3.1 model card is explicit: it says the model is optimized for Danbooru-style tags rather than natural-language prompts. Illustrious XL was fine-tuned on the Danbooru2023 dataset. On the other side, the Stable Diffusion 3 paper explains that it mixed in synthetic captions because human-written captions focus on the main subject and leave out the background and composition. OpenAI's DALL·E 3 paper found that training on highly descriptive generated captions substantially improved prompt following. That is why newer general-purpose models lean toward sentences.

Model cards change between versions, and some newer anime releases also claim better natural-language understanding. Before you caption a large dataset, check the model card for the exact checkpoint you are training on.

Hybrid captions: a sentence plus tags

A hybrid caption puts a short sentence first, for the relationships, and then the tags, for coverage and consistency:

Two girls sit on a park bench under cherry blossoms; the one on the left reads a book, the one on the right eats ice cream. 2girls, sitting, bench, short hair, black hair, red jacket, long hair, blonde hair, blue dress, book, ice cream, cherry blossoms, outdoors

Most of the caption stays easy to edit, and the sentence records what tags can't. This suits models that understand both styles, and multi-character images in an otherwise tag-captioned dataset.

Watch out for caption shuffling. --shuffle_caption splits on commas, so a sentence that contains commas gets cut into pieces and scrambled. sd-scripts solves this with --keep_tokens_separator. Everything before the separator stays fixed, and only the part after it is shuffled:

Two girls sit on a park bench, one reading and one eating ice cream ||| 2girls, sitting, bench, book, ice cream, cherry blossoms

Set keep_tokens_separator to |||, or whatever marker you chose, in your training config. The sd-scripts dataset config documentation shows more examples.

Decision table

The last row is a different job entirely, covered in image keywords for SEO.

What ImgTagger does today

ImgTagger runs WD14 taggers, so it produces booru tags and nothing else. It does not write sentences. If you want sentence or hybrid captions, tag and clean your images here first, export the .txt files, then add the sentences with a captioning model or by hand. Tagging first still helps, because the tag list is a checklist of what is visible. For the full tag-based workflow, see captioning a LoRA dataset.