Sample - OpenAI API
POST/fine_tuning/alpha/graders/run

Run a grader

Runs a grader against a supplied dataset item and model sample. Provide a supported grader, the input item, and the model_sample to evaluate; the response includes the reward, grader metadata, sub-rewards, and token usage.

  • RetriesRetries up to 2×, 500ms backoff, 30s timeout.

3 body fields

A grader definition, the dataset item used as input, and the model output sample to evaluate.

graderobjectrequired
The grader used for the fine-tuning job.
itemobjectoptional
The dataset item provided to the grader. This will be used to populate the `item` namespace. See [the guide](https://developers.openai.com/api/docs/guides/graders) for more details.
model_samplestringrequired
The model sample to be evaluated. This value will be used to populate the `sample` namespace. See [the guide](https://developers.openai.com/api/docs/guides/graders) for more details. The `output_json` variable will be populated if the model sample is a valid JSON string.

1 status code
200Returns the overall reward, grader metadata, sub-rewards, and model grader token usage.
rewardnumberrequired
metadataobjectrequired
sub_rewardsobjectrequired
model_grader_token_usage_per_modelobjectrequired

Error handling

grader and model_sample are required. grader must match one of the supported grader variants, such as a string-check grader with type, name, input, reference, and operation, a text-similarity grader with evaluation_metric, or a Python grader with source.