Create Test Suite Run — Vellum

Trigger a Test Suite and create a new Test Suite Run

Request

This endpoint expects an object.

test_suite_idstringRequired

The ID of the Test Suite to run

exec_configobjectRequired

Configuration that defines how the Test Suite should be run

Response

This endpoint returns an object

idstring

createddatetime

test_suiteobject

stateenum

Allowed values: QUEUEDRUNNINGCOMPLETEFAILEDCANCELLED

The current state of this run

QUEUED - Queued
RUNNING - Running
COMPLETE - Complete
FAILED - Failed
CANCELLED - Cancelled

exec_configobjectOptional

Configuration that defines how the Test Suite should be run

1 curl -X POST https://api.vellum.ai/v1/test-suite-runs \ 2 -H "X_API_KEY: <apiKey>" \ 3 -H "Content-Type: application/json" \ 4 -d '{ 5 "test_suite_id": "test_suite_id", 6 "exec_config": { 7 "type": "EXTERNAL", 8 "data": { 9 "executions": [ 10 { 11 "outputs": [ 12 { 13 "type": "ARRAY", 14 "name": "outputs" 15 } 16 ], 17 "test_case_id": "test_case_id" 18 } 19 ] 20 } 21 } 22 }'

1 { 2 "id": "id", 3 "created": "2024-01-15T09:30:00Z", 4 "test_suite": { 5 "id": "id", 6 "history_item_id": "history_item_id", 7 "label": "label" 8 }, 9 "state": "QUEUED", 10 "exec_config": { 11 "type": "EXTERNAL", 12 "data": { 13 "executions": [ 14 { 15 "outputs": [ 16 { 17 "type": "ARRAY", 18 "name": "outputs" 19 } 20 ], 21 "test_case_id": "test_case_id" 22 } 23 ] 24 }, 25 "test_case_ids": [ 26 "exec_config" 27 ] 28 } 29 }

1	curl -X POST https://api.vellum.ai/v1/test-suite-runs \
2	-H "X_API_KEY: <apiKey>" \
3	-H "Content-Type: application/json" \
4	-d '{
5	"test_suite_id": "test_suite_id",
6	"exec_config": {
7	"type": "EXTERNAL",
8	"data": {
9	"executions": [
10	{
11	"outputs": [
12	{
13	"type": "ARRAY",
14	"name": "outputs"
15	}
16	],
17	"test_case_id": "test_case_id"
18	}
19	]
20	}
21	}
22	}'

1	{
2	"id": "id",
3	"created": "2024-01-15T09:30:00Z",
4	"test_suite": {
5	"id": "id",
6	"history_item_id": "history_item_id",
7	"label": "label"
8	},
9	"state": "QUEUED",
10	"exec_config": {
11	"type": "EXTERNAL",
12	"data": {
13	"executions": [
14	{
15	"outputs": [
16	{
17	"type": "ARRAY",
18	"name": "outputs"
19	}
20	],
21	"test_case_id": "test_case_id"
22	}
23	]
24	},
25	"test_case_ids": [
26	"exec_config"
27	]
28	}
29	}