External Workflow Server

Note: This capability is under active development and considered to be in Beta. If you’re interested in deploying your organization’s Workflow, please reach out to Support.

In this tutorial, we show you how to deploy your Workflow to your own infrastructure. For the purposes of this tutorial, we will use AWS as the example cloud provider, but these steps should apply to any cloud provider that supports VM instances that can run Docker Images.

1. Stand up the VM Instance

First, create an EC2 Instance in your AWS account. Be sure to:

• create a Key Pair so that you could SSH in, or have some way to run setup commands inside of the instance
• ensure the instance belongs to a security group with sufficient inbound rules to allow HTTP requests on port 8000.

2. Setup Docker

Install Docker on the instance. Run the following commands to do so for an Amazon Linux Image:

$
sudo yum update -y
>
sudo yum install docker -y
>
sudo systemctl start docker
>
sudo systemctl enable docker
>
sudo usermod -a -G docker ec2-user

After performing the last command, you will need to leave and return to the instance for the credentials to refresh.

3. Define your Workflow Server Image

Next, you’ll want to define a Docker Image locally that extends Vellum’s default image:

1
FROM vellumai/python-workflow-runtime:1.10.2
2
3
CMD ["vellum_start_server"]

4. Build the image

Run the command below locally to build a Docker image from the Dockerfile

$
docker build -t demo:1.0.0 .

5. Push Workflow Server Image to ECR

Next, you’ll want to push the Docker Image to a container repository. For the purposes of this tutorial, we will use AWS Elastic Container Registry.

Ensure you are locally authenticated with AWS:

$
aws configure

Start by authenticating Docker:

$
# replace <region> with the desired region
>
# replace <account-id> with your account id
>
aws ecr get-login-password --region <region> | \
>
  docker login --username AWS --password-stdin \
>
  <account-id>.dkr.ecr.<region>.amazonaws.com

Create the ECR Eepository if it doesn’t exist:

$
aws ecr create-repository \ 
>
  --repository-name demo \
>
  --region <region>

Then push it to the Repository:

$
docker tag demo:1.0.0 \
>
  <account-id>.dkr.ecr.<region>.amazonaws.com/demo:1.0.0
>
docker push <account-id>.dkr.ecr.<region>.amazonaws.com/demo:1.0.0

6. Pull Workflow Server Image from docker

Now we will pull the Docker Image you previously pushed into the standalone image. You will want to make sure that your EC2 has an IAM policy allowing it to pull from AWS ECR.

First login to docker:

$
aws ecr get-login-password --region <region> | \
>
  docker login --username AWS \
>
  --password-stdin <account-id>.dkr.ecr.<region>.amazonaws.com

Then pull the image:

$
docker pull \
>
  <account-id>.dkr.ecr.<region>.amazonaws.com/demo:1.0.0

7. Run the server!

Now that the image is in the EC2 instance, you can run the server with:

$
docker run -p 8000:8000 \
>
  009994482511.dkr.ecr.us-east-1.amazonaws.com/demo:1.0.0

Two logs that you’ll want to see are one specifying Running with SDK version: 1.10.2 and one alerting that the server is Listening at: http://0.0.0.0:8000 to confirm that the server is running properly

8. Execute the workflow!

There are some notable differences when invoking Workflows against the Vellum Workflow Server vs the traditional Vellum API. While we work to consolidate the two, you can use the following script to invoke it:

1
import json
2
from uuid import uuid4
3
import requests
4
import os
5
from pydantic import TypeAdapter
6
from vellum.client.types import WorkflowEvent
7
8
workflow_event_type: TypeAdapter[WorkflowEvent] = TypeAdapter(
9
    WorkflowEvent
10
)
11
12
13
def main():
14
    """
15
    This script is assumed to be in the same directory as 
16
    the `workflow.py` file.
17
    """
18
    
19
    parent_dir = os.path.dirname(os.path.abspath(__file__))
20
    files = {}
21
    current_file = os.path.basename(__file__)
22
23
    for filename in os.listdir(parent_dir):
24
        filepath = os.path.join(parent_dir, filename)
25
        if not os.path.isfile(filepath):
26
            continue
27
        if not filename.endswith(".py"):
28
            continue
29
        if filename == current_file:
30
            continue
31
32
        with open(filepath, "r") as f:
33
            files[filename] = f.read()
34
35
    public_ip_address = "0.0.0.0" # grab this from the EC2 dashboard
36
    response = requests.post(
37
        f"http://{public_ip_address}:8000/workflow/stream",
38
        json={
39
            "files": files,
40
            "environment_api_key": os.getenv("VELLUM_API_KEY"),
41
            "module": __name__,
42
            # Don't worry about the value of these fields.
43
            # They are currently required but considered legacy and 
44
            # will be removed in the future.
45
            "execution_id": str(uuid4()),
46
        },
47
        timeout=10,
48
    )
49
    print(response.status_code)
50
    
51
    outputs = {}
52
    error = None
53
    for line in response.iter_lines():
54
        if not line:
55
            continue
56
        if b"vembda.execution" in line: 
57
            continue
58
        if b"END" == line:
59
            continue
60
        if b".execution.streaming" in line:
61
            continue
62
63
        event = workflow_event_type.validate_json(line)
64
        if event.name == "workflow.execution.rejected":
65
            error = event.body.error
66
            continue
67
68
        if event.name != "workflow.execution.fulfilled":
69
            continue
70
71
        outputs = event.body.outputs
72
73
    if error:
74
        print("Error!")
75
        print(error)
76
        return
77
    
78
    print("Success!")
79
    print(outputs)
80
81
82
if __name__ == "__main__":
83
    main()