Long Running Workflows

When building production applications with Vellum Workflows, you may encounter scenarios where Workflows take several minutes or longer to complete. This guide covers best practices for handling these long-running Workflows effectively.

Understanding the Challenge

Long-running Workflows can present challenges in production environments:

• Client timeouts: HTTP clients may timeout before Workflow completion
• Resource constraints: Keeping connections open for extended periods
• Error handling: Managing failures in long-running processes
• User experience: Providing feedback during extended operations

Recommended Approaches

Option 1: Webhooks (Recommended)

The most robust approach for long-running Workflows is to execute them asynchronously and use webhooks to receive completion notifications. See our Webhooks documentation for detailed setup instructions.

import vellum
client = vellum.VellumClient(api_key="your-api-key")
# Use async endpoint to get execution_id immediately
response = client.execute_workflow_async(
    workflow_deployment_name="your-workflow",
    inputs=[
        vellum.WorkflowRequestStringInput(
            name="user_query",
            value="Process this complex request"
        )
    ],
    external_id="task-12345"  # Your internal task ID for correlation
)
execution_id = response.execution_id
print(f"Workflow started with execution_id: {execution_id}")
# Store execution_id for later correlation with webhook

from flask import Flask, request, jsonify
import hmac
import hashlib
app = Flask(__name__)
@app.route('/webhook/vellum', methods=['POST'])
def handle_vellum_webhook():
    # Verify HMAC signature (recommended)
    signature = request.headers.get('X-Vellum-Signature')
    timestamp = request.headers.get('X-Vellum-Timestamp')
    
    if not verify_hmac_signature(request.data, signature, timestamp):
        return jsonify({'error': 'Invalid signature'}), 401
    
    event = request.json
    
    if event['type'] == 'workflow.execution.fulfilled':
        # Workflow completed successfully
        external_id = event['data']['parent']['external_id']
        outputs = event['data']['outputs']
        
        # Update your internal task status
        update_task_status(external_id, 'completed', outputs)
        
    elif event['type'] == 'workflow.execution.rejected':
        # Workflow failed
        external_id = event['data']['parent']['external_id']
        error = event['data']['error']
        
        # Update your internal task status
        update_task_status(external_id, 'failed', error)
    
    return jsonify({'status': 'received'}), 200
def verify_hmac_signature(payload, signature, timestamp):
    # Implement HMAC verification
    # See our HMAC Authentication documentation for implementation details:
    # https://docs.vellum.ai/product/security/hmac-authentication
    pass
def update_task_status(external_id, status, data):
    # Update your database/system with the Workflow result
    print(f"Task {external_id} status: {status}")

Option 2: Async Execution with Status Polling

Use the async execution endpoint to initiate a Workflow and immediately receive an execution_id, then poll for the execution status using the Check Workflow Execution Status endpoint. This is the recommended approach for polling-based async execution.

Async executions automatically queue when you exceed your concurrency limit, making this endpoint ideal for batch jobs where you don’t need everything to complete at once. You can initiate many executions quickly and they’ll process as capacity becomes available:

import vellum
import time
client = vellum.VellumClient(api_key="your-api-key")
# Step 1: Start Workflow asynchronously and get execution_id
response = client.execute_workflow_async(
    workflow_deployment_name="your-workflow",
    inputs=[
        vellum.WorkflowRequestStringInput(
            name="user_query",
            value="Process this complex request"
        )
    ],
    external_id="task-12345"  # Optional: your internal ID for tracking
)
execution_id = response.execution_id
print(f"Workflow started with execution_id: {execution_id}")
# Step 2: Poll for execution status
while True:
    try:
        # Check execution status
        status_response = client.check_workflow_execution_status(
            execution_id=execution_id
        )
        
        if status_response.status == "FULFILLED":
            print("Workflow completed successfully!")
            print(f"Results: {status_response.outputs}")
            if status_response.execution_detail_url:
                print(f"View details: {status_response.execution_detail_url}")
            break
        elif status_response.status == "REJECTED":
            print("Workflow failed!")
            break
        elif status_response.status == "PENDING":
            print("Workflow is still pending...")
            time.sleep(5)  # Poll every 5 seconds for pending
        else:
            print(f"Workflow still running... Status: {status_response.status}")
            time.sleep(30)  # Poll every 30 seconds for running workflows
            
    except Exception as e:
        print(f"Error checking execution status: {e}")
        time.sleep(30)

Option 3: API Node

This approach is beneficial when you want to specify a specific callback URL or payload format. You can use an API Node at the end of your Workflow to send results directly to your system:

Option 4: Streaming Updates

For Workflows where you want to provide real-time progress updates, use the streaming execution endpoint. This can still be problematic if updates are really long, but at least enables you to stream updated messaging from your LLMs as they invoke tools:

import vellum
client = vellum.VellumClient(api_key="your-api-key")
# Stream Workflow execution for real-time updates
stream = client.execute_workflow_stream(
    workflow_deployment_name="your-workflow",
    inputs=[
        vellum.WorkflowRequestStringInput(
            name="user_query",
            value="Process this complex request"
        )
    ],
    external_id="task-12345"
)
for event in stream:
    if event.type == "workflow.execution.initiated":
        print(f"Workflow started: {event.execution_id}")
    elif event.type == "workflow.execution.streaming":
        # Handle intermediate results
        print(f"Progress update: {event.data}")
    elif event.type == "workflow.execution.fulfilled":
        print(f"Workflow completed: {event.data.outputs}")
        break
    elif event.type == "workflow.execution.rejected":
        print(f"Workflow failed: {event.data.error}")
        break

Timeout Management

Client-Side Timeouts

When using synchronous execution, configure appropriate timeouts:

import vellum
from vellum.core import RequestOptions
client = vellum.VellumClient(api_key="your-api-key")
try:
    response = client.execute_workflow(
        workflow_deployment_name="your-workflow",
        inputs=[...],
        request_options=RequestOptions(
            timeout_in_seconds=600  # 10 minute timeout
        )
    )
except TimeoutError:
    print("Workflow execution timed out")
    # Handle timeout - workflow may still be running

Infrastructure Considerations

AWS Lambda

If using AWS Lambda, be aware of the 15-minute maximum execution time:

import json
import vellum
def lambda_handler(event, context):
    client = vellum.VellumClient(api_key=os.environ['VELLUM_API_KEY'])
    
    # For long Workflows, use async pattern
    response = client.execute_workflow(
        workflow_deployment_name="long-running-workflow",
        inputs=event['inputs'],
        external_id=event['task_id']  # Use for webhook correlation
    )
    
    # Return immediately with execution_id
    return {
        'statusCode': 200,
        'body': json.dumps({
            'execution_id': response.execution_id,
            'status': 'initiated',
            'message': 'Workflow started. Results will be sent via webhook.'
        })
    }

Container Environments

For containerized applications, ensure your containers can handle long-running connections if using streaming:

# Dockerfile
FROM python:3.11
# ... other setup ...
# Set appropriate timeouts
ENV REQUESTS_TIMEOUT=900
ENV WORKFLOW_TIMEOUT=900
# Health check for long-running processes
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
  CMD curl -f http://localhost:8000/health || exit 1

Error Handling and Retry Strategies

Workflow-Level Retry

For Workflows that may fail due to transient issues, implement retry logic:

import time
import vellum
from typing import Optional
def execute_workflow_with_retry(
    client: vellum.VellumClient,
    workflow_name: str,
    inputs: list,
    external_id: str,
    max_retries: int = 3,
    retry_delay: int = 60
) -> Optional[str]:
    """Execute Workflow with retry logic for transient failures."""
    
    for attempt in range(max_retries + 1):
        try:
            response = client.execute_workflow(
                workflow_deployment_name=workflow_name,
                inputs=inputs,
                external_id=f"{external_id}-attempt-{attempt}"
            )
            return response.execution_id
            
        except Exception as e:
            if attempt == max_retries:
                print(f"Workflow failed after {max_retries} retries: {e}")
                return None
            
            print(f"Attempt {attempt + 1} failed: {e}. Retrying in {retry_delay}s...")
            time.sleep(retry_delay)
    
    return None

Node-Level Resilience

Prompts may experience nondeterministic errors from model providers. Longer running Workflows are particularly prone to these issues. To mitigate this, it’s a good idea to implement retry logic with Node Adornemnts. You can also disable streaming from the Model Settings while editing a Prompt to mitigate other nondeterministic connection issues.

Use Node Adornments to add resilience to individual Workflow nodes:

• Retry Node Adornments: Automatically retry failed nodes
• Try Node Adornments: Gracefully handle node failures with fallback paths

See our Node Adornments documentation for detailed configuration.

Monitoring and Observability

Execution Tracking

Monitor long-running Workflows through the Monitoring tab of your Workflow Deployment (see our monitoring documentation for details):

1. Executions Tab: View real-time execution status
2. Timeline View: Analyze execution flow and bottlenecks
3. Cost Tracking: Monitor resource usage for long Workflows

Best Practices Summary

Related Documentation

• Batch Jobs - Process large volumes of Workflow executions efficiently
• Webhooks Configuration
• HMAC Authentication
• Node Adornments
• Workflow Monitoring
• API Reference - Execute Workflow
• API Reference - Execute Workflow Async
• API Reference - Check Workflow Execution Status