For AI agents: a documentation index is available at the root level at /llms.txt and /llms-full.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
Vellum supports real-time monitoring of platform events through Webhook integration. This allows organizations to stream event data to external systems for monitoring, alerting, and custom integrations.
Configuration
To set up Webhook integration:
Navigate to your organization settings page in Vellum
Scroll down to the “Monitoring Integration” section
Click on “Webhook Integration” or the “Edit” button if it’s already configured
Configure the integration with your desired settings
Authentication
Webhook integration supports two authentication methods to ensure your data is securely transmitted:
API Key Header: Send an API key in a custom header with each webhook request
Bearer Token: Use a bearer token for authentication
For additional security, you can also implement HMAC Authentication to verify that webhook requests are genuinely from Vellum.
Supported Events
You can configure which events are sent to your webhook endpoint. The following events are available:
Category
Event
Description
Workflow Execution Events
workflow.execution.initiated
Triggered when a workflow starts execution
workflow.execution.fulfilled
Triggered when a workflow successfully completes
workflow.execution.rejected
Triggered when a workflow execution fails
Workflow Usage Calculation Events
workflow_execution.usage_calculation.fulfilled
Triggered when workflow usage is successfully calculated
workflow_execution.usage_calculation.rejected
Triggered when workflow usage calculation fails
Metric Execution Events
metric.execution.fulfilled
Triggered when a metric execution completes successfully
Event Schema
Webhook events follow a consistent schema that includes essential information about the event:
1
{
2
"id": "UUID",
3
"timestamp": "ISO-8601 datetime",
4
"api_version": "2024-10-25",
5
"trace_id": "UUID",
6
"span_id": "UUID",
7
"parent": {
8
// Parent context information
9
},
10
"name": "event.name.format",
11
"body": {
12
// Event-specific data
13
}
14
}
Workflow Execution Events
Workflow Execution Initiated
1
{
2
"id": "UUID",
3
"timestamp": "ISO-8601 datetime",
4
"api_version": "2024-10-25",
5
"trace_id": "UUID",
6
"span_id": "UUID",
7
"parent": {
8
// Parent context information
9
},
10
"name": "workflow.execution.initiated",
11
"body": {
12
"workflow_class": {
13
"name": "string",
14
"module": "string",
15
"bases": []
16
},
17
"inputs": {
18
// Input variables
19
}
20
}
21
}
Workflow Execution Fulfilled
1
{
2
"id": "UUID",
3
"timestamp": "ISO-8601 datetime",
4
"api_version": "2024-10-25",
5
"trace_id": "UUID",
6
"span_id": "UUID",
7
"parent": {
8
// Parent context information
9
},
10
"name": "workflow.execution.fulfilled",
11
"body": {
12
"workflow_class": {
13
"name": "string",
14
"module": "string",
15
"bases": []
16
},
17
"outputs": {
18
// Output variables
19
}
20
}
21
}
Workflow Execution Rejected
1
{
2
"id": "UUID",
3
"timestamp": "ISO-8601 datetime",
4
"api_version": "2024-10-25",
5
"trace_id": "UUID",
6
"span_id": "UUID",
7
"parent": {
8
// Parent context information
9
},
10
"name": "workflow.execution.rejected",
11
"body": {
12
"workflow_class": {
13
"name": "string",
14
"module": "string",
15
"bases": []
16
},
17
"error": {
18
"code": "ERROR_CODE",
19
"message": "Error message"
20
}
21
}
22
}
Workflow Usage Calculation Events
1
{
2
"id": "UUID",
3
"timestamp": "ISO-8601 datetime",
4
"api_version": "2024-10-25",
5
"trace_id": "UUID",
6
"span_id": "UUID",
7
"parent": {
8
// Parent context information
9
},
10
"name": "workflow.usage_calculation.fulfilled",
11
"body": {
12
"usage": {
13
"input_token_count": 123,
14
"output_token_count": 456,
15
"input_char_count": 789,
16
"output_char_count": 101,
17
"cache_creation_input_tokens": 0,
18
"cache_read_input_tokens": 0,
19
"compute_nanos": 1000000000,
20
},
21
"cost": {
22
"value": 0.0123,
23
"currency": "USD",
24
}
25
}
26
}
Parent Context
The parent field provides context about what triggered the event, which can include:
1
{
2
"span_id": "UUID",
3
"parent": null,
4
"deployment_id": "UUID",
5
"deployment_name": "my-deployment",
6
"deployment_history_item_id": "UUID",
7
"workflow_version_id": "UUID",
8
"release_tag_id": "UUID",
9
"release_tag_name": "PRODUCTION",
10
"external_id": "optional-external-id"
11
}
Use Cases
The Webhook integration enables several key monitoring and integration capabilities:
Real-time Monitoring: Track workflow executions as they happen
Custom Alerting: Build custom alerting systems based on workflow failures or performance metrics
Cost Tracking: Monitor resource consumption and costs
Integration with External Systems: Connect Vellum events to your existing monitoring infrastructure
Audit Logging: Maintain a record of all executions for compliance and debugging
Custom Analytics: Build custom dashboards and reports using your Vellum data
Best Practices
When setting up your Webhook integration:
Start with Critical Events: Begin by monitoring the most important events like execution failures
Implement Proper Error Handling: Ensure your webhook endpoint can handle occasional failures or retries
Use HMAC Authentication: For production systems, implement HMAC verification to ensure webhook authenticity
Monitor Webhook Performance: Ensure your webhook endpoint can handle the volume of events without introducing latency
Implement Idempotency: Design your webhook handler to be idempotent in case of duplicate events
