// ◉ Multimodal & Vision
Visual Instruction Tuning (LLaVA)
Connects a vision encoder to an LLM via a projector, tuned on visual instructions.
视觉指令微调（LLaVA） 2023 Intermediate core
IN PLAIN WORDS
CLIP features pass through a simple projector into visual tokens fed with text to the LLM, tuned on GPT-4-generated visual conversations. This encoder–projector–LLM recipe is used by many open VLMs.
Used in: LLaVA, Qwen-VL, InternVL and other open VLMs
TIMELINE · prerequisites → this → what builds on it
01Original paper
Visual Instruction Tuning
Haotian Liu, Chunyuan Li, Qingyang Wu, et al. · 2023
arXiv ↗ ✓ Verified · arXiv API
02Where it sits
Learn the left column first; the right column builds on this one.
Then you can learn
- Nothing in the atlas builds on it yet
Related:Flamingo
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
04Where the course teaches it
On learning paths:Generative Models & Frontier Research
For AI / data
- Verification: verified, checked 2026-10-21.
- JSON: /api/learn/algorithms.json (id:
llava)