How to caption a LoRA training dataset with tags

A LoRA learns from pairs: an image and the text that describes it. The caption decides which parts of each image get linked to your trigger word and which parts get linked to ordinary tags you can still change at prompt time. Captions are why one character LoRA can wear any outfit while another always draws the same jacket.

This guide walks through the whole job, from a folder of images to a ZIP of matching caption files, using the ImgTagger auto-tagger in your browser. Trainer settings have their own section near the end, because people use different trainers.

What a captioned dataset looks like

Almost every LoRA trainer uses the same convention. Each image has a sidecar text file with the same base name, and the trainer pairs them by name. A small character dataset looks like this:

mira_001.png
mira_001.txt
mira_002.jpg
mira_002.txt
mira_003.webp
mira_003.txt

Each .txt file holds one line of tags separated by a comma and a space:

miraoc, 1girl, solo, smile, looking at viewer, school uniform, upper body, outdoors, cherry blossoms

The file rules (matching names, encoding, separators) have their own page: caption .txt sidecar files. If your images aren't final yet, clear out duplicates and near-duplicates first with the duplicate image finder. A repeated image gets extra weight in training.

Step 1: Pick a trigger word

The trigger word is what you will type in prompts to call up your concept. It works best when the base model has no strong existing meaning for it, so a made-up word such as miraoc is safer than a real one. Avoid ordinary words and existing booru tags such as red or scarf, which already mean something to the model.

Put the trigger first in every caption, spelled exactly the same way each time. A fixed position lets trainers keep it in place when they shuffle tags (see the trainer section), and it makes a missing trigger easy to spot.

In ImgTagger, open the Auto-tag panel and type the trigger into Always add these tags first before you run the tagger. Prepend tags go at the very start of every caption, ahead of anything the image already had, in the order you type them. To add the trigger to images that are already tagged, use the Add Tags panel with the mode set to Prepend (Add to start). It also keeps the order you type, but it skips images that already have the tag anywhere in their list, so it won't move an existing copy to the front. Remove the tag with the Remove Tags panel first, then prepend it again.

Step 2: Auto-tag the batch

Drop the images onto the tagger. Pick a model (ConvNeXt V2 is the default, and our WD14 model comparison explains the others) and leave the confidence threshold at 0.35 for a first pass. The confidence threshold guide covers when to raise or lower it.

Fill in the blacklist with tags you already know you don't want, such as the four rating tags if you won't use them. The blacklist matches whole tags and ignores case and underscores, so simple_background and simple background both work. Then click Auto-tag all.

Each caption comes out in this order: prepend tags, then any tags the image already had, then new model tags from most to least confident. When the run finishes, a summary lists every added tag with a count, and each tag's remove button takes it back off the images that just received it. That is a quick way to drop obvious misfires.

Step 3: Decide what to tag and what to leave out

This is the step that shapes your LoRA. The usual rule of thumb is: caption what should stay variable, and leave out what should be baked into the concept.

During training, the model tries to explain each image using its caption. A feature named by a tag becomes linked to that tag, so you can still switch it on and off by prompting. A feature that appears in every image but is never tagged has nothing else to attach to, so it gets absorbed by the trigger word, the one tag that is always there. Leaving a feature out is how you tell the LoRA "this is part of the concept".

Example: a character LoRA

Say your original character always has grey hair, green eyes and a hairclip, and you have drawn her in different outfits, poses and places. One image comes back from the tagger as:

miraoc, general, 1girl, solo, grey hair, green eyes, hairclip, hair ornament, smile, school uniform, jacket, upper body, outdoors, tree

The pruned caption becomes miraoc, 1girl, solo, smile, school uniform, jacket, upper body, outdoors, tree. If she only ever wears one outfit, you could cut those tags too, at the cost of making the outfit hard to change later.

Example: a style LoRA

A style LoRA flips the logic. The subjects change from image to image, and the look stays the same. Say you are training a soft ink-and-watercolor illustration style across portraits, animals and landscapes, and one image comes back as:

inkwash, general, no humans, cat, flower, outdoors, traditional media, watercolor (medium), limited palette

Here you cut traditional media, watercolor (medium) and limited palette, because they describe the style you want the trigger to learn. You keep no humans, cat, flower and outdoors, because the point of a style LoRA is to apply the look to any subject. Cut the subject tags instead, and the LoRA starts to tie the style to cats and flowers.

Make these cuts across the batch, not image by image. The tag pruning guide covers redundant tags, misleading ones and starter blacklists for both kinds of LoRA.

Step 4: Review the batch with the Filter panel

Tagger errors are spread thinly across a batch, so reading one caption at a time misses them. Review by tag instead: the Filter panel lists every tag in the batch with the number of images that carry it.

• Sort by count, high to low. Your trigger should be at the top, with a count equal to the number of images. Other tags near the top are on almost every image. Decide whether each is part of the concept (cut) or a real variable (keep).
• Sort by count, low to high. Tags on only one or two images are often misfires. Check each against its image.
• Any mode. Check a tag to show only the images that have it, then step through them with the left and right arrow keys.
• All mode. Check two tags that shouldn't appear together, such as solo and multiple girls, or long hair and short hair. Any image left on screen has a contradiction to fix.
• None mode. Check your trigger word. Any image still showing is missing it, which often happens to images added after the first run.

When some images have no tags at all, the list also shows a ∅ untagged entry; check it in Any mode to see them. To fix a problem in bulk, filter down to the images you want, click Select All (it selects only the visible images), then use the Add Tags or Remove Tags panel with its "Selected" button. Selected-image actions ignore anything the current filter hides. The trash button next to a tag in the Filter panel is different: it removes that tag from every image in the batch.

Step 5: Keep captions consistent

Consistency matters more than any single tag. A trainer can't tell that long hair and long_hair mean the same thing, and ImgTagger's Filter panel lists them as two separate tags. So keep to:

• One spelling style. ImgTagger writes spaces, so type spaces in hand-added tags too, and convert older underscore captions before you mix them in.
• One trigger, one place. Always first, always spelled the same.
• One set of rules. Apply the same keep-and-cut decisions to every image, rather than hand-describing a few in far more detail than the rest.

Hand-added tags work best when they are real booru tags the base model already knows. The booru tag glossary lists common ones by topic.

Step 6: Export the captions

Click Download in the tagger's bottom bar. Tick .txt tag files, and also Image files if you want a self-contained copy, then choose All Items or Selected Only. ImgTagger builds a ZIP such as images_tags.zip. Each caption is saved as a .txt named after its image, with tags joined by a comma and a space. Images go in exactly as you dropped them, and the ZIP has no subfolders.

Before you export, check the ∅ untagged entry one last time. An image with no tags is exported with an empty .txt file, and some trainers refuse to start when they find one.

Trainer settings: kohya-ss sd-scripts

This section applies to kohya-ss sd-scripts and tools built on it, and was checked against its source and docs. Other trainers read the same files but name their options differently.

• --caption_extension=".txt". The default is .caption, so without this the trainer won't find ImgTagger's files. In a dataset config file, the same setting is caption_extension = ".txt".
• Folder naming. --train_data_dir points at a parent folder, and each subfolder inside it is named <repeats>_<name>, such as 10_miraoc. The number sets how many times each image is repeated per epoch. The text after the underscore is used as the caption only for images that have no caption file. Folders that don't start with a number are skipped. If you use a --dataset_config file instead, folder names are not parsed, and you set num_repeats in the file.
• --shuffle_caption shuffles the comma-separated tags each time an image is used, so the model doesn't come to depend on tag order.
• --keep_tokens=1 keeps the first tag (your trigger) in place while the rest are shuffled. Use a higher number if you put more than one fixed tag at the front. (--keep_tokens_separator is an alternative.)
• File rules. sd-scripts reads captions as UTF-8, uses only the first line of each file unless --enable_wildcard is on, and stops with a "caption file is empty" error if it meets an empty one.

A typical layout and the matching flags:

train/
  10_miraoc/
    mira_001.png
    mira_001.txt
    ...

--train_data_dir="train" --caption_extension=".txt" --shuffle_caption --keep_tokens=1