For AI agents: a documentation index is available at the root level at /llms.txt and /llms-full.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
BlogLog InRequest Demo
HomeProductDevelopersSelf-HostingChangelog
HomeProductDevelopersSelf-HostingChangelog
  • Getting Started
    • Overview
  • Workflows SDK
    • Introduction
    • Installation
    • Core Concepts
    • Defining Control Flow
    • Configuration
    • Custom Docker Images
  • Client SDK
    • Introduction
    • Authentication
    • API Versioning
        • GETList Test Cases
        • POSTUpsert Test Cases
        • STREAMBulk Update Test Cases
        • DELDelete Test Case
LogoLogo
BlogLog InRequest Demo
Client SDKTest SuitesTest Cases

Upsert Test Cases

Beta
POST
https://api.vellum.ai/v1/test-suites/:id/test-cases
POST
/v1/test-suites/:id/test-cases
1import requests
2
3url = "https://api.vellum.ai/v1/test-suites/id/test-cases"
4
5payload = {
6    "input_values": [
7        {
8            "type": "STRING",
9            "value": "What are your favorite colors?",
10            "name": "var_1"
11        }
12    ],
13    "evaluation_values": [
14        {
15            "type": "ARRAY",
16            "value": [
17                {
18                    "type": "STRING",
19                    "value": "Red"
20                },
21                {
22                    "type": "STRING",
23                    "value": "Green"
24                },
25                {
26                    "type": "STRING",
27                    "value": "Blue"
28                }
29            ],
30            "name": "var_2"
31        }
32    ],
33    "label": "Test Case 1"
34}
35headers = {
36    "X-API-KEY": "<apiKey>",
37    "Content-Type": "application/json"
38}
39
40response = requests.post(url, json=payload, headers=headers)
41
42print(response.json())
200Successful
1{
2  "input_values": [
3    {
4      "variable_id": "string",
5      "name": "string",
6      "type": "STRING",
7      "value": "string"
8    }
9  ],
10  "evaluation_values": [
11    {
12      "variable_id": "string",
13      "name": "string",
14      "type": "STRING",
15      "value": "string"
16    }
17  ],
18  "id": "string",
19  "external_id": "string",
20  "label": "string"
21}
Upserts a new test case for a test suite, keying off of the optionally provided test case id. If an id is provided and has a match, the test case will be updated. If no id is provided or no match is found, a new test case will be appended to the end. Note that a full replacement of the test case is performed, so any fields not provided will be removed or overwritten with default values.
Was this page helpful?
Previous

Bulk Update Test Cases

Next
Built with

Authentication

X-API-KEYstring
API Key authentication via header

Path parameters

idstringRequired
Either the Test Suites' ID or its unique name

Headers

X-API-VersionstringOptional

Request

This endpoint expects an object.
input_valueslist of objectsRequired
Values for each of the Test Case's input variables
evaluation_valueslist of objectsRequired
Values for each of the Test Case's evaluation variables
idstringOptionalformat: "uuid"

The Vellum-generated ID of an existing Test Case whose data you’d like to replace. If specified and no Test Case exists with this ID, a 404 will be returned.

external_idstringOptional>=1 character

An ID external to Vellum that uniquely identifies the Test Case that you’d like to create/update. If there’s a match on a Test Case that was previously created with the same external_id, it will be updated. Otherwise, a new Test Case will be created with this value as its external_id. If no external_id is specified, then a new Test Case will always be created.

labelstring or nullOptional

A human-readable label used to convey the intention of this Test Case

Response

input_valueslist of objects
evaluation_valueslist of objects
idstring
external_idstring or null
labelstring or null