TY  - CONF
AU  - Raul Gomez
AU  - Lluis Gomez
AU  - Jaume Gibert
AU  - Dimosthenis Karatzas
A2  - ECCVW
PY  - 2018//
TI  - Learning to Learn from Web Data through Deep Semantic Embeddings
T2  - LNCS
BT  - 15th European Conference on Computer Vision Workshops
SP  - 514
EP  - 529
VL  - 11134
N2  - In this paper we propose to learn a multimodal image and text embedding from Web and Social Media data, aiming to leverage the semantic knowledge learnt in the text domain and transfer it to a visual model for semantic image retrieval. We demonstrate that the pipeline can learn from images with associated text without supervision and perform a thourough analysis of five different text embeddings in three different benchmarks. We show that the embeddings learnt with Web and Social Media data have competitive performances over supervised methods in the text based image retrieval task, and we clearly outperform state of the art in the MIRFlickr dataset when training in the target data. Further we demonstrate how semantic multimodal image retrieval can be performed using the learnt embeddings, going beyond classical instance-level retrieval problems. Finally, we present a new dataset, InstaCities1M, composed by Instagram images and their associated texts that can be used for fair comparison of image-text embeddings.
UR  - https://link.springer.com/chapter/10.1007/978-3-030-11024-6_40
L1  - http://refbase.cvc.uab.es/files/GGG2018a.pdf
N1  - DAG; 600.129; 601.338; 600.121
ID  - Raul Gomez2018
ER  -