Vision-language model

Common card · No. 281901

A vision–language model (VLM) is a type of artificial intelligence system that can jointly interpret and generate information from both images and text, extending the capabilities of large language models (LLMs), which are limited to text. It is an example of multimodal learning. Many widely used commercial applications now rely on this ability. OpenAI introduced computer vision capabilities to its GPT-4V variant of the GPT-4 model, enabling users to incorporate uploaded photographs or diagrams into their discussions with ChatGPT. It has since become an integral part of ChatGPT's standard offering. Similar capabilities were added to Google’s Gemini, Anthropic’s Claude 3 Opus, and Microsoft’s Copilot with Vision. Alongside these models, several open-source vision–language models—such as LLaVA, InstructBLIP, and MiniGPT-4—have been released by the research community, offering smaller-scale alternatives for experimentation and academic study.

Read on Wikipedia