Compare

Compare lets you send the same prompts to multiple model lanes and inspect outputs side-by-side.

Create a run

POST /v1/compare/runs

{
  "name": "Support answer benchmark",
  "prompts": [
    "Summarize our refund policy in one paragraph.",
    "Write a friendly onboarding email."
  ],
  "lanes": [
    {"model": "gemini-3-flash"},
    {"model": "gpt-5.4"}
  ]
}

Run endpoints

Endpoint	Purpose
POST /v1/compare/runs	Create a batch comparison run
GET /v1/compare/runs	List runs
GET /v1/compare/runs/{run_id}	Run status
GET /v1/compare/runs/{run_id}/results	Results
GET /v1/compare/runs/{run_id}/export	Export JSON or CSV
POST /v1/compare/runs/{run_id}/cancel	Cancel a run
POST /v1/compare/runs/{run_id}/resume	Resume a run
POST /v1/compare/runs/{run_id}/lanes/{lane_id}/winner	Mark a winning lane

Datasets and conversations

Endpoint	Purpose
POST /v1/compare/datasets	Create reusable prompt packs
GET /v1/compare/datasets	List prompt packs
POST /v1/compare/conversations	Create multi-turn comparison
POST /v1/compare/conversations/{id}/turns	Add a turn