Datasets Overview

vellum.workflows.inputs.DatasetRow

Datasets allow you to define test scenarios and sample data for local workflow development. They are stored in the sandbox.py file within your workflow directory and are included when you push or pull your workflow artifact using the Vellum CLI.

DatasetRow

The DatasetRow class represents a single test scenario with a label, inputs, optional trigger, and optional mocks.

Attributes

label

strRequired

A descriptive label for the test scenario. This helps identify the scenario in the Vellum UI and logs.

inputs

Union[BaseInputs, Dict[str, Any]]Required

The input data for the workflow. Can be either a BaseInputs instance or a dictionary of input values.

workflow_trigger

Optional[BaseTrigger]

Optional trigger instance for this scenario. Use this to test workflows that are triggered by schedules or integrations.

mocks

Optional[Sequence[Union[BaseOutputs, MockNodeExecution]]]

Optional sequence of node output mocks for testing scenarios. Allows you to override node outputs during local execution.

Basic Usage

The sandbox.py file defines your dataset and creates a WorkflowSandboxRunner to execute your workflow locally.

1 from vellum.workflows.inputs import DatasetRow
2 from vellum.workflows.sandbox import WorkflowSandboxRunner
3 
4 from .inputs import Inputs
5 from .workflow import Workflow
6 
7 dataset = [
8     DatasetRow(label="Scenario 1", inputs=Inputs(user_message="Hello")),
9     DatasetRow(label="Scenario 2", inputs=Inputs(user_message="How are you?")),
10 ]
11 
12 runner = WorkflowSandboxRunner(workflow=Workflow(), dataset=dataset)
13 
14 if __name__ == "__main__":
15     runner.run()

You can run a specific scenario by passing an index to the run() method:

1 runner.run(index=1)  # Runs "Scenario 2"

Inputs

Inputs can be provided as either a typed BaseInputs instance or a dictionary. Using typed inputs provides better IDE support and validation.

Using Typed Inputs

The Inputs class is defined in your workflow’s ./inputs.py file:

1 # ./inputs.py
2 from vellum.workflows.inputs import BaseInputs
3 
4 class Inputs(BaseInputs):
5     user_message: str
6     temperature: float = 0.7

Then reference it in your sandbox.py:

1 # ./sandbox.py
2 from vellum.workflows.inputs import DatasetRow
3 
4 from .inputs import Inputs
5 
6 dataset = [
7     DatasetRow(
8         label="With typed inputs",
9         inputs=Inputs(user_message="Hello", temperature=0.5),
10     ),
11 ]

Using Dictionary Inputs

1 from vellum.workflows.inputs import DatasetRow
2 
3 dataset = [
4     DatasetRow(
5         label="With dict inputs",
6         inputs={"user_message": "Hello", "temperature": 0.5},
7     ),
8 ]

Triggers

Triggers allow you to test workflows that are activated by schedules or external integrations. The workflow_trigger attribute accepts any trigger type that extends BaseTrigger.

When using workflow_trigger, you should not define inputs as the trigger provides its own input context.

Available Trigger Types

Trigger	Description
`ScheduleTrigger`	For workflows triggered on a schedule (cron-based)
`IntegrationTrigger`	For workflows triggered by external integrations
`ManualTrigger`	For workflows triggered manually

Using Schedule Triggers

The trigger class is defined in your workflow’s ./triggers/scheduled.py file:

1 # ./triggers/scheduled.py
2 from vellum.workflows.triggers import ScheduleTrigger
3 
4 class MySchedule(ScheduleTrigger):
5     pass

Then reference it in your sandbox.py:

1 # ./sandbox.py
2 from datetime import datetime
3 from vellum.workflows.inputs import DatasetRow
4 
5 from .triggers.scheduled import MySchedule
6 
7 dataset = [
8     DatasetRow(
9         label="Scheduled execution",
10         workflow_trigger=MySchedule(
11             current_run_at=datetime.now(),
12             next_run_at=datetime.now(),
13         ),
14     ),
15 ]

vellum.workflows.inputs.DatasetRow

DatasetRow

The DatasetRow class represents a single test scenario with a label, inputs, optional trigger, and optional mocks.

Attributes

label

strRequired

A descriptive label for the test scenario. This helps identify the scenario in the Vellum UI and logs.

inputs

Union[BaseInputs, Dict[str, Any]]Required

The input data for the workflow. Can be either a BaseInputs instance or a dictionary of input values.

workflow_trigger

Optional[BaseTrigger]

Optional trigger instance for this scenario. Use this to test workflows that are triggered by schedules or integrations.

mocks

Optional[Sequence[Union[BaseOutputs, MockNodeExecution]]]

Optional sequence of node output mocks for testing scenarios. Allows you to override node outputs during local execution.

Basic Usage

The sandbox.py file defines your dataset and creates a WorkflowSandboxRunner to execute your workflow locally.

1 from vellum.workflows.inputs import DatasetRow
2 from vellum.workflows.sandbox import WorkflowSandboxRunner
3 
4 from .inputs import Inputs
5 from .workflow import Workflow
6 
7 dataset = [
8     DatasetRow(label="Scenario 1", inputs=Inputs(user_message="Hello")),
9     DatasetRow(label="Scenario 2", inputs=Inputs(user_message="How are you?")),
10 ]
11 
12 runner = WorkflowSandboxRunner(workflow=Workflow(), dataset=dataset)
13 
14 if __name__ == "__main__":
15     runner.run()

You can run a specific scenario by passing an index to the run() method:

1 runner.run(index=1)  # Runs "Scenario 2"

Inputs

Inputs can be provided as either a typed BaseInputs instance or a dictionary. Using typed inputs provides better IDE support and validation.

Using Typed Inputs

The Inputs class is defined in your workflow’s ./inputs.py file:

1 # ./inputs.py
2 from vellum.workflows.inputs import BaseInputs
3 
4 class Inputs(BaseInputs):
5     user_message: str
6     temperature: float = 0.7

Then reference it in your sandbox.py:

1 # ./sandbox.py
2 from vellum.workflows.inputs import DatasetRow
3 
4 from .inputs import Inputs
5 
6 dataset = [
7     DatasetRow(
8         label="With typed inputs",
9         inputs=Inputs(user_message="Hello", temperature=0.5),
10     ),
11 ]

Using Dictionary Inputs

1 from vellum.workflows.inputs import DatasetRow
2 
3 dataset = [
4     DatasetRow(
5         label="With dict inputs",
6         inputs={"user_message": "Hello", "temperature": 0.5},
7     ),
8 ]

Triggers

Triggers allow you to test workflows that are activated by schedules or external integrations. The workflow_trigger attribute accepts any trigger type that extends BaseTrigger.

When using workflow_trigger, you should not define inputs as the trigger provides its own input context.

Available Trigger Types

Trigger	Description
`ScheduleTrigger`	For workflows triggered on a schedule (cron-based)
`IntegrationTrigger`	For workflows triggered by external integrations
`ManualTrigger`	For workflows triggered manually

Using Schedule Triggers

The trigger class is defined in your workflow’s ./triggers/scheduled.py file:

1 # ./triggers/scheduled.py
2 from vellum.workflows.triggers import ScheduleTrigger
3 
4 class MySchedule(ScheduleTrigger):
5     pass

Then reference it in your sandbox.py:

1 # ./sandbox.py
2 from datetime import datetime
3 from vellum.workflows.inputs import DatasetRow
4 
5 from .triggers.scheduled import MySchedule
6 
7 dataset = [
8     DatasetRow(
9         label="Scheduled execution",
10         workflow_trigger=MySchedule(
11             current_run_at=datetime.now(),
12             next_run_at=datetime.now(),
13         ),
14     ),
15 ]

1	from vellum.workflows.inputs import DatasetRow
2	from vellum.workflows.sandbox import WorkflowSandboxRunner
3
4	from .inputs import Inputs
5	from .workflow import Workflow
6
7	dataset = [
8	DatasetRow(label="Scenario 1", inputs=Inputs(user_message="Hello")),
9	DatasetRow(label="Scenario 2", inputs=Inputs(user_message="How are you?")),
10	]
11
12	runner = WorkflowSandboxRunner(workflow=Workflow(), dataset=dataset)
13
14	if __name__ == "__main__":
15	runner.run()

1	# ./inputs.py
2	from vellum.workflows.inputs import BaseInputs
3
4	class Inputs(BaseInputs):
5	user_message: str
6	temperature: float = 0.7

1	# ./sandbox.py
2	from vellum.workflows.inputs import DatasetRow
3
4	from .inputs import Inputs
5
6	dataset = [
7	DatasetRow(
8	label="With typed inputs",
9	inputs=Inputs(user_message="Hello", temperature=0.5),
10	),
11	]

1	# ./triggers/scheduled.py
2	from vellum.workflows.triggers import ScheduleTrigger
3
4	class MySchedule(ScheduleTrigger):
5	pass

1	# ./sandbox.py
2	from datetime import datetime
3	from vellum.workflows.inputs import DatasetRow
4
5	from .triggers.scheduled import MySchedule
6
7	dataset = [
8	DatasetRow(
9	label="Scheduled execution",
10	workflow_trigger=MySchedule(
11	current_run_at=datetime.now(),
12	next_run_at=datetime.now(),
13	),
14	),
15	]