Evaluating LLM Judges for Personalized Writing Style Assessment

Performance correlates weakly with general LLM ability (MMLU ρ ≤ 0.2) and varies by task and evaluation criteria. Pairwise comparisons yield more consistent results than independent scoring. Stronger models do not best replicate human judgments, possibly due to over-sensitivity to subtle details ignored by humans and weaker models.

Summarised from the original article by an open-source model, then checked automatically: every number and English name must appear in the source. The original article is authoritative.