Lesson 4: LLM as Judge: Rubrics, Formats, and What Not to Let It Judge

Learning goals:

• Write a multi-dimensional rubric for free-form text outputs, make each dimension's question clear, and understand that one success criterion often needs several rubrics to fully evaluate
• Tighten judge output into a shape programs can handle: reason first then score, 0.0–1.0 plus pass/fail, and explain why a single call is often more stable than multiple judges evaluating separate aspects
• Recognize three failure modes of judges—grading their own work, being asked to find problems so they always do, trusting only the agent's self-report—and provide actionable mitigations for each

Prerequisites: Lessons 1–3, understanding "end-state first" evaluation and the prioritization of deterministic verifiers | Previous: << Lesson 3 | Next: Lesson 5 >>

Where Lesson 3 hits its ceiling, and where LLM judges fit

You've given your research agent a task: summarize how domestic EV charging subsidies changed over the past three years, produce a two-page brief. It ran for fifteen minutes, called search tools, fetched PDFs, wrote 1,800 words. Reads plausible.

Now you want to verify it. None of Lesson 3's deterministic checks apply here: no test suite to run, no build exit code to read, no golden_answer for output == golden_answer because you can't write one—put two human analysts on the same brief and they won't produce identical text.

This isn't a failure of imagination. This is what these outputs look like. When Anthropic reflected on their multi-agent research system, they were direct: research outputs are free-form text, rarely have a single correct answer, so they're difficult to evaluate programmatically—LLMs are a natural fit for grading outputs like this¹.

But before you casually toss the brief to another model and ask "how's this look," understand where LLM judges sit in the official ranking. The principle for choosing a grading method is "fastest, most reliable, most scalable"²: code-based grading is fastest and most reliable, extremely scalable, but lacks nuance for complex judgments requiring less rule-based rigidity²; LLM-based grading is fast, flexible, scalable, and suitable for complex judgment—and the official guidance pairs that with a prerequisite in the same sentence: test to ensure reliability first, then scale²; human grading is most flexible and high quality, but slow and expensive, so avoid if possible².

Treat the prerequisite in that middle option as a hard requirement, not a disclaimer. An LLM judge isn't "a smarter check," it's a model call: it can make mistakes, it costs money, it might score the same input differently on successive runs. Its value is singular—it reaches things deterministic checks can't.

So this lesson isn't about "how to get a model to score your output." That's trivial—any prompt gets you a number. It's about making that number worth trusting: how to structure rubrics, how to constrain output, who runs the judge, how it breaks, and when you shouldn't ask it at all.

Rubrics: breaking "good" into separately answerable questions

A rubric sounds formal. In plain terms, it's a scorecard: break vague "is this good" into specific questions, answer each separately, score each separately.

Why break it down? Ask "is this brief good" and the model gives you vague praise or vague criticism. Ask "does every claim in the brief appear in its cited sources" and the model can check claim by claim. Most use cases need multidimensional evaluation along several success criteria².

Anthropic's research agent used a five-dimension rubric¹, each dimension behind a real failure mode:

• Factual accuracy—do claims match sources? The model says "subsidies dropped 30% in 2023," does the cited document actually contain that number? This dimension catches hallucination.
• Citation accuracy—do the cited sources match the claims? Opposite direction from the previous dimension; many people conflate them. The first asks "is this statement backed up," this one asks "does the link attached to this sentence actually discuss this topic?" A common agent failure: the claim is fine, but it's tagged with a source whose title merely looked relevant.
• Completeness—are all requested aspects covered? You said "past three years" and it only wrote about last year, you said "changes" and it only wrote current state. Check each requirement, don't evaluate how pretty the prose is.
• Source quality—primary sources or low-quality secondary? This dimension catches the sneakiest problems. Anthropic's human testers found: early agents consistently chose SEO-optimized content farms over authoritative but less highly-ranked sources like academic PDFs or personal blogs¹. The output reads completely normal, but the foundation is rotten.
• Tool efficiency—right tools, reasonable call count? Getting the same answer with three searches versus thirty searches differs by an order of magnitude in cost. This dimension evaluates process economics, not "did it follow my prescribed steps"—the latter doesn't work, as Lesson 2 explained.

These five dimensions are for research tasks. Different tasks need different sets. What you can take is the decomposition method: start from "if this output breaks, how does it break," derive dimensions from that, one failure mode per dimension. Also don't miss: a given use case, or even a specific success criterion for that use case, might require several rubrics for holistic evaluation². Don't expect one scorecard to cover everything.

Output format: one call, one score, one pass/fail

First, the shape of the verdict. Anthropic tried multiple judges evaluating different components—one judge per component. The result: a single LLM call with a single prompt outputting scores from 0.0-1.0 and a pass-fail grade was the most consistent and aligned with human judgments¹.

Fair statement: breaking evaluation into multiple LLM calls, each call evaluating one aspect, is itself an automating-evals pattern the official guidance has described³. Neither approach is wrong. The difference is "single-call is better" comes from Anthropic's comparison on their own real system¹. Recommended approach: start with a single call, and only split if you have evidence that splitting improves accuracy.

Next, how tightly to constrain output. Official guidance for judge prompts is "empirical or specific": for example, instruct the LLM to output only 'correct' or 'incorrect', or to judge from a scale of 1–5; purely qualitative evaluations are hard to assess quickly and at scale². In plain terms: if the judge returns "overall quality acceptable, but some details handled somewhat roughly," you have nothing—you can't average 200 such responses, can't tell if today's better than yesterday's, and you still have to read each one. So why ask the judge?

The critical technique: reason first, score second, then discard the reasoning. Official guidance explicitly states this: ask the LLM to reason first before producing an evaluation score, and then discard the reasoning—this increases evaluation performance, particularly for tasks requiring complex judgment². "Discard" doesn't mean don't look at it. The reasoning is scaffolding for the judge—it needs to write "paragraph 3's claim maps to which transcript segment" before it can judge accurately. But that text shouldn't enter your downstream statistics. The report wants scores and pass/fail; reasoning goes to logs, waiting for you when a score looks suspicious.

What about subjective dimensions? Some dimensions can't be binary, like "is this brief's tone appropriate for sending to clients?" Official guidance provides a tool: the LLM-based Likert scale, which uses an LLM to judge subjective attitudes or perceptions²—a Likert scale is those fixed-increment questionnaire formats like "strongly disagree / disagree / neutral / agree / strongly agree" (the number of points is a survey-design convention, not an official mandate). The key principle still holds: give it fixed increments, don't let it free-form.

Putting it together, judge output looks roughly like this (example shows only first three dimensions; source_quality and tool_efficiency have the same shape, omitted for brevity). Programs read score and verdict; reasoning gets persisted for review:

The one doing the work can't be the judge

At this point, a tempting shortcut appears: have the agent grade itself at the end. It knows best what it just did.

Suppress that temptation. Two reasons. First, official experience: implementing guardrails where one model instance processes user queries while another screens them for inappropriate content or requests tends to perform better than having the same LLM call handle both guardrails and the core response³—that experience originally described content moderation (one instance responds, another screens), but the principle "don't let the same call wear both hats" applies here. Note it says "same call"—the problem is shared context, not insufficient intelligence. Second, the mechanism: Claude Code official documentation describes a practice where a reviewer running in a fresh subagent context sees only the diff and the criteria you give it, not the reasoning that produced the change, so it evaluates the result on its own terms⁴.

Read it backward to see why self-grading fails: the chain of reasoning that produced the output is still hanging in context. The model just convinced itself "writing it this way is correct," you immediately follow with "is this correct," and it will likely repeat the same reasoning. What you get isn't an independent judgment, it's self-repetition.

If you're worried the judge is too lenient or too strict, official guidance provides a knob: evaluating whether a given piece of content is inappropriate, with multiple prompts evaluating different aspects or requiring different vote thresholds to balance false positives and negatives³. In a verification scenario, a false positive is "flagging something fine as broken"—you'll be annoyed to death by false alarms; a false negative is "letting something broken through"—bad work ships. Three judges with two needing to say fail before it counts as fail is more lenient than one judge saying fail counts as fail—which setting you pick depends on which kind of error costs more in your scenario.

How judges break

You've switched to a different instance, you've written a rubric, and the judge still breaks. It breaks in predictable ways.

One: reviewers asked to find problems will always find problems. Most counterintuitive, most expensive. Official documentation is clear: a reviewer prompted to find gaps will usually report some, even when the work is sound, because that is what it was asked to do⁴. The consequence isn't just "a few useless suggestions": the same passage notes that chasing every finding leads to over-engineering—extra abstraction layers, defensive code, and tests for cases that can't happen⁴. Your agent enters a self-escalation loop: reviewer suggests three things, you fix them, re-review suggests three more, code gets thicker, actual problems remain unsolved.

You've probably seen these reports: "suggest adding empty-string defense to parseDate"—but upstream already guarantees non-empty; "suggest extracting these three constants into config"—but they haven't changed in three years; "suggest adding retry tests for network timeout"—but this path has no network calls. All three aren't wrong, all three aren't worth doing, but mixed into a review report they look identical to real issues.

Official guidance provides the mitigation: tell the reviewer to flag only gaps that affect correctness or the stated requirements, and treat the rest as optional⁴. Write that verbatim into the judge prompt, and give "other suggestions" a place to go—add an optional_notes field and explicitly state it doesn't participate in scoring. Somewhere to write means it doesn't have to cram style preferences into deduction justifications.

Two: agent self-reports aren't evidence. Many people take a shortcut and only feed the agent's final summary to the judge: "I retrieved three authoritative sources and cross-checked before confirming the subsidy rate." Judge reads it, thinks the process sounds solid, gives a high score. But this statement isn't evidence. Official guidance on tool evaluation makes the point: what agents omit in their feedback and responses can often be more important than what they include—LLMs don't always say what they mean⁵. It says "cross-checked three sources" but might have only called search once; the failed retry it doesn't mention, the moment it got empty results and kept fabricating, that's what you most need to see. The solution is also direct: review the raw transcripts (including tool calls and tool responses) to catch any behavior not explicitly described in the agent's chain-of-thought⁵. Translated to a solution: judge input must include raw logs, not just self-report. The "tool efficiency" dimension especially needs this—it evaluates actual call count and whether calls were correct; that information only exists in logs.

Three: the question itself is ambiguous. Strictly speaking this isn't the judge breaking, it's that you gave it a broken question. Official guidance on eval design names a category to avoid: ambiguous test cases where even humans would find it hard to reach an assessment consensus². You'll recognize these when validating the judge: you and a colleague judge the same output and get opposite results. Don't rush to fix the judge prompt—the judge is "inaccurate" because this question has no accurate answer. Either refine the criteria until humans can agree, or remove this case. Using it to compute the judge's consistency rate only produces a self-deceptive number.

When to build a review-revise loop

Does the judge finish and you're done, or do you feed its feedback to the agent for a revision, then judge again? This loop is tempting, and it easily becomes a token-burning perpetual motion machine.

Official guidance provides two signals for when this workflow truly fits³: first, that LLM responses can be demonstrably improved when a human articulates their feedback; and second, that the LLM can provide such feedback. The same passage adds a prerequisite: this workflow is particularly effective when we have clear evaluation criteria, and when iterative refinement provides measurable value³.

Use those two conditions as admission tests. The approach is straightforward: you be the judge first, write feedback to the agent, see if it actually improves after revision. If human-written feedback doesn't move it, LLM-written feedback will move it even less. At that point what needs fixing is the prompt or the tools, not adding a review layer. Conversely, if human feedback clearly works, check the second condition—have an LLM write feedback against the rubric, compare it to yours. If both pass, the loop is worth building.

Proportionality: a judge is also a model call

Return to Lesson 3's ranking. The principle for choosing a grading method is fastest, most reliable, most scalable²; code-based grading ranks ahead of LLM-based grading on all three². So judges go where deterministic checks can't reach, not as their replacement. For the same brief, the correct layering looks like this:

Layer 1 (deterministic, milliseconds, zero cost)  Valid JSON / Markdown? Length in required range?  Citation links reachable (HTTP status)? Citation count >= 3?  Any item fails → immediate fail, don't invoke judge
Layer 2 (LLM judge, seconds, billed by token)  Five-dimension rubric scoring + pass/fail  Input: task requirements + output + raw logs (tool calls & responses)
Layer 3 (human, slow and expensive, spot-check only)  Judge said fail but you're suspicious  Random sample of judge-said-pass to guard against systematic leniency

Don't send Layer 1 catches to Layer 2 to pay—output that isn't even valid format doesn't need a model call to tell you it's unacceptable. Layer 3 can't be skipped: people testing agents find edge cases that evals miss¹; the earlier "consistently preferring content farms" bias was caught by human testing¹.

This lesson's boundary: how to organize eval sets, how many cases to run this judge against—that's Lesson 5. Wiring the judge into a repeatable eval harness that produces reports—that's Lesson 6. This lesson only solves how to make a single judgment trustworthy.

💻 Exercises

Recap

• Free-form text outputs have no single correct answer and are difficult to evaluate programmatically; LLMs are a natural fit for grading outputs like this¹; but the official positioning for LLM-based grading is "fast, flexible, scalable, suitable for complex judgment—prerequisite: test to ensure reliability first then scale"; human grading is most flexible and high quality but slow and expensive, avoid if possible².
• A rubric breaks "good" into separately answerable questions. Research tasks' ready-made decomposition is five dimensions: factual accuracy, citation accuracy, completeness, source quality, tool efficiency¹; a given use case, or even one success criterion within it, might require several rubrics for holistic evaluation².
• For output shape, a single LLM call with a single prompt outputting 0.0-1.0 scores and pass-fail was the most consistent and aligned with human judgments in actual measurement¹; instruct the LLM to output 'correct'/'incorrect' or 1–5; purely qualitative evaluations are hard to assess quickly and at scale². Ask the LLM to reason first before producing an evaluation score and then discard the reasoning—increases evaluation performance, particularly for complex judgment²; subjective dimensions can use LLM-based Likert scales².
• The doer can't be the judge: implementing guardrails where one model instance processes queries while another screens tends to perform better than having the same call handle both³; a reviewer in a fresh context only sees the output and your criteria, can't see the reasoning that produced the output, so judges the result on its own terms⁴. Multi-prompt evaluation with vote thresholds balances false positives and false negatives³.
• A reviewer prompted to find gaps will usually report some even when work is sound, and chasing every finding leads to over-engineering; fix is to tell it to flag only gaps affecting correctness or stated requirements, treat the rest as optional⁴.
• Don't only feed the agent's self-report to the judge: what it omits is often more important than what it includes, LLMs don't always say what they mean⁵; review raw transcripts (tool calls and tool responses) to catch behavior not in the self-description⁵.
• Whether a review-revise loop is worth building depends on two signals: LLM responses can be demonstrably improved when a human articulates feedback, and the LLM can itself provide such feedback³.
• A judge is also a model call, with cost and noise. The principle for choosing grading methods is fastest, most reliable, most scalable; code-based grading ranks ahead on all three²—deterministic checks that can catch something shouldn't send it to a judge; beyond automated evals still need human spot-checks, people testing agents find edge cases that evals miss¹.

>> Lesson 5: Eval Sets: Start with 20 Real Tasks