// AI news · 2026-11-08 · Dev.to LLM
LLM Judge Evolves from Laptop to Under ₹1 per Chat
An LLM judge was built to evaluate thousands of AI support conversations daily, scoring full dialogues against rules and context.
Published 2026-11-08 01:49 (Beijing time)Source Dev.to LLMAuto-summarised, fact-checked against source
01Summary
Initially costing ~3 rupees per chat, optimization via batch APIs and caching reduced costs to under ₹1 per conversation. The judge evaluates correctness, usefulness, and tone using a compliance file defining acceptable answer shapes.
Summarised from the original article by an open-source model, then checked automatically: every number and English name must appear in the source. The original article is authoritative.
02Key points
- Judge scores entire conversations, not individual replies.
- Compliance file defines 'good' answers as shape-based ranges, not exact matches.
- Evaluation includes accuracy, usability, and tone.
- Cost reduced to under ₹1 per chat via batch API and caching.
03Source
The Evolution of an LLM Judge: From “It Works on My Laptop” to Under a Rupee a Conversation
Dev.to LLM · published 2026-11-08 01:49 Beijing time
05Related news
- Same source · osFoundry optimizes Claude Code session caching2026-11-08 · Dev.to LLM
- Same source · Local replication of weight-space edit2026-11-08 · Dev.to LLM
- Same topic · Read public Google Sheets as JSON without OAuth or API key2026-11-08 · Dev.to AI
- Same topic · Google Messages Homepage Redesign2026-11-08 · 9to5Google
- Same day · Agent Workspace Is a Cache, Not Disk2026-11-08 · Dev.to AI
From AIMap AI news: collected from 1,100+ official and media sources. Summaries only restate the original article; follow the link for the full story. All stories of 2026-11-08 · Archive by date · All AI news · Today