Sample - OpenAI API
POST/evals/{eval_id}/runs

Create an evaluation run

Creates and starts a run for a specified evaluation using a selected data source and model configuration. Supply data_source to identify the run input and, where applicable, the model sampling configuration used to test the evaluation. A 201 response returns the newly created run with its initial status and run identifier.

  • RetriesRetries up to 2×, 500ms backoff, 30s timeout.

1 parameter · 3 body fields
eval_idstringrequired
The unique identifier of the evaluation for which to create a run.

Evaluation run configuration containing the data source and optional run metadata.

namestringoptional
The name of the run.
metadataobjectoptional
Optional key-value metadata attached to the run. Use at most 16 entries, with keys of no more than 64 characters and string values, or set it to null.
data_sourceobjectrequired
Details about the run's data source.

3 status codes
201Returns the created evaluation run with its identifier, associated evaluation identifier, status, model, creation timestamp, report URL, result counts, usage details, data source, metadata, and error information.
objectstringrequired
The type of the object. Always "eval.run".
Allowed:eval.runDefault:eval.run
idstringrequired
Unique identifier for the evaluation run.
eval_idstringrequired
The identifier of the associated evaluation.
statusstringrequired
The status of the evaluation run.
modelstringrequired
The model that is evaluated, if applicable.
namestringrequired
The name of the evaluation run.
created_atintegerrequired
Unix timestamp (in seconds) when the evaluation run was created.
report_urlstringrequired
The URL to the rendered evaluation run report on the UI dashboard.
result_countsobjectrequired
Counters summarizing the outcomes of the evaluation run.
per_model_usagearray<object>required
Usage statistics for each model during the evaluation run.
per_testing_criteria_resultsarray<object>required
Results per testing criteria applied during the evaluation run.
data_sourceobjectrequired
Information about the run's data source.
metadataobjectrequired
Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.
errorobjectrequired
An object representing an error response from the Eval API.
400Returned when the request is invalid, such as when the required evaluation run data source is missing or does not match the evaluation schema.
codestringrequired
messagestringrequired
paramstringrequired
typestringrequired
misalignmentobjectoptional
429Returned when the request is rejected because the rate limit has been exceeded.
errorobjectrequired

Error handling

data_source is required and must match one of the supported JSONL, completions, or responses data source configurations. A 400 is returned for an invalid request, such as a missing evaluation object or data source. A 429 is returned when the request is rejected because the rate limit has been exceeded.