Upsert Test Cases | Vellum

import requests
url = "https://api.vellum.ai/v1/test-suites/id/test-cases"
payload = {
    "input_values": [
        {
            "type": "STRING",
            "value": "What are your favorite colors?",
            "name": "var_1"
        }
    ],
    "evaluation_values": [
        {
            "type": "ARRAY",
            "value": [
                {
                    "type": "STRING",
                    "value": "Red"
                },
                {
                    "type": "STRING",
                    "value": "Green"
                },
                {
                    "type": "STRING",
                    "value": "Blue"
                }
            ],
            "name": "var_2"
        }
    ],
    "label": "Test Case 1"
}
headers = {
    "X-API-KEY": "<apiKey>",
    "Content-Type": "application/json"
}
response = requests.post(url, json=payload, headers=headers)
print(response.json())

{
  "input_values": [
    {
      "variable_id": "string",
      "name": "string",
      "type": "STRING",
      "value": "string"
    }
  ],
  "evaluation_values": [
    {
      "variable_id": "string",
      "name": "string",
      "type": "STRING",
      "value": "string"
    }
  ],
  "id": "string",
  "external_id": "string",
  "label": "string"
}

Upserts a new test case for a test suite, keying off of the optionally provided test case id.

If an id is provided and has a match, the test case will be updated. If no id is provided or no match is found, a new test case will be appended to the end.

Note that a full replacement of the test case is performed, so any fields not provided will be removed or overwritten with default values.

Authentication

X-API-KEYstring

API Key authentication via header

Path parameters

idstringRequired

Either the Test Suites' ID or its unique name

Request

This endpoint expects an object.

input_valueslist of objectsRequired

Values for each of the Test Case's input variables

evaluation_valueslist of objectsRequired

Values for each of the Test Case's evaluation variables

idstringOptionalformat: "uuid"

The Vellum-generated ID of an existing Test Case whose data you'd like to replace. If specified and no Test Case exists with this ID, a 404 will be returned.

external_idstringOptional>=1 character

An ID external to Vellum that uniquely identifies the Test Case that you'd like to create/update. If there's a match on a Test Case that was previously created with the same external_id, it will be updated. Otherwise, a new Test Case will be created with this value as its external_id. If no external_id is specified, then a new Test Case will always be created.

labelstring or nullOptional

A human-readable label used to convey the intention of this Test Case

Response

This endpoint returns an object.

input_valueslist of objects

evaluation_valueslist of objects

idstringOptional

external_idstring or nullOptional

labelstring or nullOptional

Authentication

Path parameters

Headers

Request

Response