// ⛨ Safety, Evaluation & Security
LLM-as-a-Judge
Strong LLMs grade or compare other models' answers.
大模型当评委 2023 Foundational core
IN PLAIN WORDS
On MT-Bench and Chatbot Arena, GPT-4 judges agree with humans over 80%, but show position, verbosity and self-preference biases that need mitigation.
Used in: Model evaluation, automatic RAG/agent evals
01Original paper
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, et al. · 2023
arXiv ↗ ✓ Verified · arXiv API
02Where it sits
Learn the left column first; the right column builds on this one.
Then you can learn
- Nothing in the atlas builds on it yet
Related:Automated Red TeamingReward Modeling (Preference Learning)
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
Reference code
04Where the course teaches it
On learning paths:AI Application Engineer
For AI / data
- Verification: verified, checked 2026-10-21.
- JSON: /api/learn/algorithms.json (id:
llm-judge)