Machine Learning
Intermediate
Pro
Multimodal LLMs and Vision
How GPT-4V/4o-style vision-language models see images — patches, attention, grounding, and why they're not object detectors
25 min read
6 views
Learn how multimodal LLMs actually process images: patch tokenization, vision-encoder pretraining, and how the model distinguishes between multiple objects in a scene through attention rather than bounding-box regression. Covers the practical gap between vision-language models and classical object detectors, common failure modes (counting, fine-grained discrimination, spatial relations), and prompting techniques (referring expressions, crops, set-of-mark) that make multi-object questions reliable in production.