
- Cet évènement est passé
Séminaire Image : « Text-Aided Domain Adaptation for Adapting CLIP-like Models to Novel Domains », Louis Hémadou
27 novembre 2024 / 14:00 - 15:00
Nous aurons le plaisir d’écouter Louis Hémadou, doctorant CIFRE SAFRAN.
Il donnera un séminaire IMAGE le mercredi 27 novembre à 14h00 en salle de séminaire F-200.
Titre : « Text-Aided Domain Adaptation for Adapting CLIP-like Models to Novel Domains »
Résumé :
Pretrained text-image models like CLIP demonstrate impressive zero-shot classification abilities across various tasks. However, fine-tuning the vision model on specific training images is often necessary to move beyond zero-shot capabilities. Yet, when there is a domain shift between the training and test images, fine-tuning can sometimes degrade model performance on the test set. To address this, we propose using the textual descriptions of the test image domain to adjust the source images, reducing the domain gap. These adjustments are performed in CLIP space, where text and image modalities are semantically aligned. We demonstrate that this approach enhances the performance of several fine-tuning methods on test images.