Vision-and-Language Pretrained Models: A Survey

Vision-and-Language Pretrained Models: A Survey

Siqu Long, Feiqi Cao, Soyeon Caren Han, Haiqin Yang

Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence
Survey Track. Pages 5530-5537. https://doi.org/10.24963/ijcai.2022/773

Pretrained models have produced great success in both Computer Vision (CV) and Natural Language Processing (NLP). This progress leads to learning joint representations of vision and language pretraining by feeding visual and linguistic contents into a multi-layer transformer, Visual-Language Pretrained Models (VLPMs). In this paper, we present an overview of the major advances achieved in VLPMs for producing joint representations of vision and language. As the preliminaries, we briefly describe the general task definition and genetic architecture of VLPMs. We first discuss the language and vision data encoding methods and then present the mainstream VLPM structure as the core content. We further summarise several essential pretraining and fine-tuning strategies. Finally, we highlight three future directions for both CV and NLP researchers to provide insightful guidance.
Keywords:
Survey Track: Multidisciplinary Topics and Applications
Survey Track: Computer Vision
Survey Track: Natural Language Processing