Sulba
ReviewSign inAccount
Theme
Editor theme
Light
Dark

Intermediate · Evals

Grading with a model

Grade the help desk's replies with Granite Guardian 3.1 2B on your machine: its prompt rendered from its own template and its verdict read from its likeliest first tokens, its agreement with people's labels, a check on the help desk's own pages, a review of an assistant's grading run, and a report of how many of a release's replies keep to their passages.

You build
grader.py, then agreement.py, then known.py, then judge.py, then grounding.py
Stages
3 Build, 1 Catch, 1 Ship
Time
About 3 hours
Runs on
Your machine

Loading the workspace…