Rubrics: writing criteria and grading against them

A rubric turns “is this a good answer?” into a list of things a response to one particular prompt must do, or must not do (Rubrics as Rewards paper, Scale AI). The same rubric can grade a response and serve as the reward a model is trained on (ComplexConstraints rubrics paper, Surge AI). This lesson covers how published rubrics are built, how their criteria are tested, and how to grade against one.

What a rubric looks like

OpenAI’s HealthBench pairs each of 5,000 health conversations with criteria written for that conversation by physicians: 48,562 unique criteria from 262 physicians across 26 specialties who have practiced in 60 countries (HealthBench paper, OpenAI).

The median conversation has 11 criteria, and the range runs from 2 to 48 (HealthBench paper, OpenAI).

Each criterion carries a point value from -10 to 10, with negative values for things a response should not do. A criterion is either met, earning its full points, or not met, earning none (HealthBench paper, OpenAI).

A response’s score is the points it earns divided by the maximum possible, so a response that meets enough negative criteria can score below zero (HealthBench paper, OpenAI).

Surge AI’s ComplexConstraints set uses 10 to 40 criteria per prompt, with a median of 19. Each criterion is labeled explicit or implicit, and objective or subjective; an implicit criterion is a requirement the prompt does not state but a good answer still has to meet (ComplexConstraints rubrics paper, Surge AI).

Four things a criterion needs

The Rubrics as Rewards paper from Scale AI sets out four properties. Criteria should be grounded in expertise, capturing the facts, reasoning steps and conclusions a correct answer needs (Rubrics as Rewards paper, Scale AI).

They should cover the dimensions of quality that matter, such as accuracy, coherence, completeness, style and safety, and include negative criteria, called pitfalls, for frequent or high-risk errors (Rubrics as Rewards paper, Scale AI).

They should be weighted, because some things matter more than others: factual correctness, for example, should outweigh stylistic clarity. The paper used four categories, Essential, Important, Optional and Pitfall (Rubrics as Rewards paper, Scale AI).

And each criterion should be self-contained, so a human or a judge model can assess it on its own, without outside context (Rubrics as Rewards paper, Scale AI).

Who writes the criteria differs. The Rubrics as Rewards authors generated theirs with a model, working from expert reference answers, because human-written rubrics were scarce in their domains (Rubrics as Rewards paper, Scale AI). The ComplexConstraints authors chose expert authors instead, to capture the user intent they say model-written rubrics tend to miss (ComplexConstraints rubrics paper, Surge AI).

Writing criteria that measure the right thing

The usual advice is one thing per criterion. The ComplexConstraints authors argue that, applied literally, this degrades the signal, and aim instead for the smallest unit that is still meaningful (ComplexConstraints rubrics paper, Surge AI).

They also write for intent, not the literal words. Their example is a user who asks for help improving their Spanish while fluently summarizing an economics article in Spanish: a rubric that takes the request at face value rewards a beginner’s vocabulary list. Their authors read several model responses and restate what the user wants, and why, in their own words before writing any criteria (ComplexConstraints rubrics paper, Surge AI).

Two more rules from the same paper: put the answer key into a factual criterion, so the judge checks it rather than recalling it, and split a criterion too broad to grade reliably into concrete, checkable parts (ComplexConstraints rubrics paper, Surge AI).

Testing a criterion before it counts

ComplexConstraints tests every criterion against a judge model before use. The author drafts the criterion, grades a reference response by hand and checks that the judge agrees. If it does not, the author decides whether the wording is ambiguous and revises it (ComplexConstraints rubrics paper, Surge AI).

Then the author edits the response so the correct verdict flips, and confirms the judge’s verdict flips with it. In one case, a criterion asking a poem to avoid alliteration failed when the judge flagged “his heart”; defining poetic alliteration as the repetition of initial stressed consonant sounds fixed it (ComplexConstraints rubrics paper, Surge AI).

This matters most in training, because a noisy judge gives a noisy reward that training will exploit. The authors watched for responses that claim to meet criteria without the substance behind them, and for responses that play to a judge’s known biases, such as a preference for length (ComplexConstraints rubrics paper, Surge AI).

That is now paid work. Outlier advertises expert roles writing rubrics that separate a genuine solution from an agent gaming the score or hard-coding its way to a high one (Outlier ML/AI expert page).

Grading against a rubric

In HealthBench, a negative criterion is worded so that its penalty applies when it is met, which means every criterion is graded by the same question: did the response do this (HealthBench paper, OpenAI)?

Turing’s terms for one evaluation project state that scores must follow the rubric’s definitions, not personal preference or style (Turing CUA evaluation project terms).

Prompt: Draft a two-sentence reply to a customer whose order arrived damaged. Offer a replacement, and keep it under 50 words.

Rubric:

1. Essential, +5 (explicit, objective): offers a replacement.
2. Important, +3 (explicit, objective): stays under 50 words.
3. Important, +3 (explicit, objective): is exactly two sentences.
4. Important, +3 (implicit, subjective): apologizes for the damage.
5. Optional, +1 (implicit, objective): tells the customer what to do with the damaged item.
6. Pitfall, -5 (implicit, objective): asks the customer to pay for the replacement or for return shipping.

Response: We’re so sorry your order arrived damaged. A replacement ships today at no cost to you. You can keep or recycle the damaged item.

Grading, one criterion at a time: 1 met (+5). 2 met, 24 words (+3). 3 not met, three sentences (0). 4 met (+3). 5 met (+1). 6 not triggered (0). That is 12 of a possible 15 points, or 0.8.