Vision-Language Models 2026: Gemma, DINOv3 and 8 More Compared

3D illustration of a camera lens sending glowing image tiles through a connector into a neural network that forms blank speech bubbles
AI-generated illustration

Ask a modern AI model to read a scanned invoice, find the dent in a photo of a car door, or tell you when the forklift first appears in a two-hour warehouse video, and there is a good chance it will manage. That everyday trick is the work of vision-language models, or VLMs: systems that take pictures and words in together and answer in words. By late 2026 the field has split into a crowded open-weight camp and a handful of frontier cloud services, with specialist vision encoders quietly powering much of both. This guide explains how VLMs work, compares ten leading options in one capability matrix, and offers a practical way to choose.

What a VLM is, and how it works

Most VLMs are built from three parts. A vision encoder chops an image into small patches and turns each patch into a vector of numbers. A projector, often a small multilayer perceptron (MLP), translates those vectors into the same “language” the text model uses. Then a large language model reads the projected image tokens alongside your prompt and writes the answer. The InternVL team names this recipe directly: its InternVL3.5 models “follow the ViT, MLP, LLM paradigm,” pairing InternViT encoders with Qwen3 and GPT-OSS language models, according to the InternVL3.5 model card.

Variations are multiplying. Google’s PaliGemma 2 joins a SigLIP-So400m image encoder to Gemma 2 decoders. Meta describes Llama 4 as natively multimodal with “early fusion,” so image and text tokens share one mixture-of-experts backbone from the start. And Google’s newest twist removes the encoder entirely: the Gemma 4 model card says the 12B “Unified” model projects raw image patches and audio straight into the language model “through lightweight linear layers,” which the company says cuts multimodal latency and lets the whole model be fine-tuned in one pass.

Image resolution is the hidden cost lever. Gemma 4 lets developers set a visual token budget of 70, 140, 280, 560 or 1,120 tokens per image, recommending low budgets for captioning and video and high budgets for OCR and small text. InternVL3.5 represents each image tile with 1,024 encoder tokens, compresses them to 256 for the language model, and its Flash variants route simple patches down to 64 tokens, cutting visual tokens by half while keeping “nearly 100%” of performance, per OpenGVLab.

Where DINO fits: the encoder, not the chatbot

Meta’s DINO family belongs in this conversation, but in a different seat. DINOv3, released in August 2025, is a self-supervised vision backbone trained on 1.7 billion images without labels and scaled to a 7-billion-parameter model. It produces dense, high-resolution features rather than sentences. Meta’s model card lists 12 checkpoints, from a 21M-parameter ViT-S to the 6.7B ViT-7B plus four ConvNeXt models, and says simple linear layers on top of the frozen features are enough for classification, segmentation, depth estimation and video tracking.

Two details make DINOv3 relevant to VLM builders. First, its features are strong at the pixel level, which is exactly where caption-trained encoders tend to blur. Meta reports DINOv3 matches or beats SigLIP 2 and Perception Encoder on many classification benchmarks while “drastically” widening the gap on dense tasks. Second, the DINOv3 repository ships dino.txt, a text encoder aligned to a distilled ViT-L for zero-shot, open-vocabulary segmentation. In practice, teams use DINO as the eyes of a system and attach a lightweight head, an MLP projector, or a separate language model when they need words out.

Key capabilities and use cases

  • Documents and OCR: reading forms, tables, charts and handwriting. Gemma 4, Qwen3.8 and InternVL3.5 all list document parsing as a core skill.
  • Grounding and detection: pointing to or boxing an object a user describes. Ai2’s Molmo 2 extends pointing and tracking to multi-image and video.
  • Video understanding: Qwen3.8-27B advertises hour-scale video; Google says Gemini 4 Argon scores 91.7% on the LVBench long-video test, per its launch post.
  • Screen and GUI agents: reading a desktop or phone screen and acting on it. Qwen reports 84.3 on OSWorld-Verified for Qwen3.8-27B, and OpenAI’s GPT-6.2 Sol supports a computer use tool.
  • Dense visual features: segmentation, depth and change detection, the home turf of DINOv3.

The capability matrix

The table below compares ten widely used options as of November 2026. Every cell comes from the developer’s own model card, documentation or announcement, linked in the model name. Figures describe what vendors publish, not independent testing.

ModelDeveloperTypeParameter sizesOpen weights / licenseInputsKey strengthsContext or resolution noteBest fit
Gemma 4Google DeepMindFull VLM (12B is encoder-free)E2B, E4B, 12B, 26B A4B (MoE), 31BYes, Apache 2.0Image, interleaved multi-image, video as frames (up to 60 s); audio on E2B, E4B, 12BOCR, document and PDF parsing, charts, UI, object detection, pointing, reasoning128K (small) or 256K (medium) tokens; image budgets of 70 to 1,120 tokensOn-device and self-hosted assistants
PaliGemma 2GoogleFull VLM (fine-tune base)3B, 10B, 28B (SigLIP plus Gemma 2)Yes, Gemma Terms of UseImage, short video captioningCaptioning, VQA, text reading, detection, segmentation224 and 448 px variants benchmarked; single-turn, not a chatbotNarrow tasks you plan to fine-tune
DINOv3Meta AI (FAIR)Vision encoder (self-supervised backbone)ViT 21M to 6.7B; ConvNeXt 29M to 198MYes, DINOv3 License (commercial use, gated download)Image; video via frame featuresDense features: segmentation, depth, detection, tracking, retrieval; dino.txt adds zero-shot text alignmentPatch size 16; any image size in multiples of 16; generates no textBackbone for custom vision pipelines, satellite and specialist imagery
Llama 4 (Scout, Maverick)MetaFull VLM (native early fusion, MoE)17B active: 109B total (Scout), 400B total (Maverick)Yes, Llama 4 Community LicenseText and image; tested up to 5 imagesVisual recognition, image reasoning, captioning, charts and documents10M tokens (Scout), 1M (Maverick)Long-context self-hosting on large GPUs
Qwen3.8-27BAlibaba QwenFull VLM (LM with vision encoder)27B denseYes, Apache 2.0Image, video (hour-scale)Documents, STEM diagrams, computer and GUI agents, reasoning262,144 tokens native, extensible to 1MOpen agentic multimodal work on one server
InternVL3.5OpenGVLabFull VLM (ViT, MLP, LLM)1B to 241B (MoE, 28B active)Yes, Apache 2.0Image, multi-image, videoOCR, charts, documents, grounding, GUI and embodied tasksDynamic high resolution; 256 visual tokens per tile (64 in Flash)Choosing an exact size for your hardware
Molmo 2Ai2Full VLM4B, 8B, 7B (fully open Olmo variant)Yes, Apache 2.0 (research and education use; data terms apply)Image, multi-image, videoPointing, counting, object tracking, video groundingOpen training data and codeResearch and transparent video grounding
Gemini 4 (Neon, Hydrogen, Carbon)GoogleFull multimodal modelNot disclosedNo, API onlyText, images, audio, video, PDFsLong video, PDFs and documents, computer use, Search grounding1M unified tokensManaged enterprise multimodal workloads
GPT-6.2 SolOpenAIFull VLM (image input, text output)Not disclosedNo, API onlyText and image (no audio or video)Reasoning over images, tool use including computer use1,050,000-token context; 128K max outputAPI agents that read screenshots and documents
Claude Opus 5.5AnthropicFull VLM (image input, text output)Not disclosedNo, API onlyText and imageVision with reasoning and tool use for knowledge work1M tokens; 128K max outputDocument-heavy knowledge work via API

How to choose

Open or closed. Open weights let you run the model in your own data center, keep images on premises and fine-tune on your data. Closed APIs such as Gemini 4, GPT-6.2 Sol and Claude Opus 5.5 trade that control for frontier capability and zero hosting work; none of the three vendors discloses parameter counts.

Size and the edge. If the model must live on a phone, laptop or camera, look at Gemma 4 E2B and E4B, InternVL3.5 at 1B to 4B, Molmo 2 at 4B, or a small DINOv3 ConvNeXt backbone. Mixture-of-experts models such as Gemma 4 26B A4B activate only about 3.8B parameters per token, which Google says lets it run “almost as fast as a 4B-parameter model.”

Licensing. Read the license before the benchmark table. Apache 2.0 (Gemma 4, Qwen3.8-27B, InternVL3.5) is the most permissive. The Llama 4 Community License requires a separate license from Meta for companies above 700 million monthly active users and asks for “Built with Llama” attribution. DINOv3 uses Meta’s own commercial license with gated downloads. Molmo 2’s weights are Apache 2.0, but Ai2 notes it was trained on third-party datasets subject to academic and non-commercial terms, per the Molmo 2 card.

Generate or perceive. If you need answers in words, choose a full VLM. If you need masks, measurements or embeddings at scale, a frozen encoder like DINOv3 plus a small head is usually cheaper and easier to validate.

What it means for business, IT and healthcare

For business leaders, VLMs turn image-heavy back-office work, from claims photos to shipping labels, into structured data without bespoke computer-vision projects. For IT teams, the decisive questions are where images are processed, how many visual tokens each request burns, and whether the license allows the intended use. InternVL3.5’s decoupled deployment, which runs the vision encoder and language model on separate GPUs, shows how serving architecture is now part of the cost conversation, per its technical report.

Healthcare organizations are watching closely, and vendors are publishing medical benchmarks. Google reports Gemma 4 31B at 61.3% on the multimodal MedXPertQA test, and PaliGemma 2’s card includes results on MIMIC-CXR chest X-ray report generation. Meta notes that DINOv2 already underpins research in histology, endoscopy and medical imaging. These are research benchmarks, not clinical validation or regulatory clearance, so any deployment still requires local evaluation, privacy review and clinical governance.

Conclusion

The VLM market no longer has one right answer. Frontier APIs lead on long video and complex reasoning, open families like Gemma 4, Qwen3.8 and InternVL3.5 bring strong document and agent skills under permissive licenses, Molmo 2 offers unusual transparency, and DINOv3 remains the go-to set of eyes for teams building their own pipelines. Start with the task, the data rules and the license, then let the matrix narrow the shortlist to two or three models worth testing on your own images.

Sources