Best model for tagging?

[MitPitt]

I'd like to hear if anyone has done a comparative study of best model. As well as the best prompt. Currently my "Write visual tags for the current image, comma-separated" does not always work as intended.

[jhc13]

CogVLM, CogAgent, and possibly InternLM-XComposer2 seem to be the strongest models right now, but they are better at generating natural language captions than lists of tags.

Currently my "Write visual tags for the current image, comma-separated" does not always work as intended.

What model are you using, and how exactly does your prompt not work? The best prompt depends on the model and what kind of captions you're looking for.

[MitPitt]

Using the prompt I mentioned above sometimes writes in the sorts of:

Here are image tags: 'man', 'suit', 'etc'

Which has excess punctuation. Currently I came to this prompt: As an AI image tagging expert, please provide precise tags for these images. Prioritize the tags by relevance. Your tags should be accurate, non-duplicative, and within a 20-75 word count range. Tags should be comma-separated. Exceptional tagging will be rewarded with $10 per image.

This one works much better. One can also add context to this prompt, for example: You are given a collection of images containing ww1 military uniforms

[MitPitt]

I don't know if this is possible to do, but blocking the LLM from using any excess punctuation except , would be useful. I would be blocking the symbol . to avoid sentences and maybe ' to avoid unnecessary quotation. Then all prompts could be more liberal for more accurate results

[jhc13]

Something like that has been suggested already (#39, #56), but it is difficult to implement properly.