TEG: image theme recognition using text-embedding-guided few-shot adaptation

Grouping images into different themes is a challenging task in photo book curation. Unlike image object recognition, image theme recognition focuses on the main subject or overall meaning conveyed by an image. This work studies few-shot image theme recognition with pre-trained contrastive language-image models, and proposes a text-embedding-guided adaptation framework with a text-embedding-guided classifier and an auxiliary classification loss. The framework exploits visual and text features to stabilize training and improve recognition performance. We also introduce Theme25, a newly annotated dataset for image theme recognition, and evaluate the method on Theme25, CIFAR100, and ImageNet.

@article{wang2024teg,
  title = {TEG: image theme recognition using text-embedding-guided few-shot adaptation},
  author = {Wang, Jikai and Lu, Wanglong and Wang, Yu and Shi, Kaijie and Jiang, Xianta and Zhao, Hanli},
  journal = {Journal of Electronic Imaging},
  volume = {33},
  number = {1},
  pages = {013028},
  year = {2024},
  doi = {10.1117/1.JEI.33.1.013028}
}