Streamline AI App Development with Vellum's Workflows

About Workflows

Workflows is a product in Vellum’s LLM dev platform that helps you quickly prototype, deploy, and manage complex chains of LLM calls and the business logic that tie them together. We solve the “whack-a-mole” problem encountered by companies that use popular open source frameworks to build AI applications, but are scared to make changes for fear of introducing regressions in production.

The Workflows UI consists of a graphical app builder where you can string together various nodes and test various input values through this system. Each prompt can also be tested extensively through Playground & Test Suites. When implemented effectively, Workflows can help you build advanced LLM applications

Supported Node Types

The names of nodes can be edited in-line once the node has been added to the UI. You can add as many nodes as you want to your Workflow. Since there is extreme flexibility in how Workflows can be created, you can modify your architecture to your desire by starting from these building blocks. Common architectures are shared in a separate help center doc (Common Workflow Architectures) and will be updated over time.

Workflow Nodes

Drag and drop a node from the side panel into the UI and start building

Prompt Nodes

A core part of any LLM application. This node represents a call to a Large Language Model. Similar to Vellum Prompts, you can use test out models from any of the major providers or open source community, including OpenAI, Anthropic, Cohere, Google, Mosaic, Falcon-40b & Llama-2.

Upon creating a Prompt Node you’ll be asked to import a prompt from an existing Deployment, Sandbox, or create one from scratch. Prompts are defined by their variables, prompt template, model provider, and parameters. Refer to this help center article to learn more about our prompt syntax (Vellum Prompt Template Syntax).

Prompt Node

Templating Nodes

The Templating Node allows you to perform custom data transformations on a set of defined inputs to create a new output. You can use this to define constants, manipulate data before feeding into a prompt, or massage a response to a format to your liking.

Check out our common data transformation templates for some common examples.

Templating Node

Search Node

The Search Node returns results from a Document Index stored inside Vellum Search. Once your documents are uploaded in an index (details on how to do that here: Uploading Documents), you can start using them in a Workflow.

The index in a Search Node can be fixed for the Workflow or chosen dynamically based on the output of an upstream node. Additional configuration options, similar to the ones in Vellum Search are also available in the Search Node.

Search Node

API Node

The API Node invokes an API endpoint and returns back the status code, raw output, and JSON output if applicable. These APIs can be either publicly accessible or privately defined within your backend through the help of Authorization headers and Secrets. Simply define a URL, HTTP Method, relevant additional headers, and the body that you would like to send to the desired endpoint.

API Node

Code Execution Nodes

The Code Execution Node empowers you to include custom logic defined directly in the workflow. You can even import custom public packages within the node’s logic. We support the following languages:

  • Python
  • TypeScript

Code Execution Node

Subworkflow Nodes

Subworkflow Nodes are essential for managing giant, complex workflows. Define reusable groups of nodes in one Workflow Sandbox and have them directly accessible upon deployment from any other workflow in your workspace. Subworkflow nodes also support release tag specification, allowing you the option to always invoke the latest workflow, or pinning to a specific release tag defined by you.

Check out the video below to see Subworkflow Nodes in action!

Conditional Node

Conditional Nodes are extremely powerful because they can help you diverge the execution path of your Workflow based on the results of an upstream node. The Conditional Node supports as many if-else-if conditions as you’d like and the rules can be grouped / nested within each other.

The number of exit options from a conditional node equal the number of if-else-if conditions created on the node

Conditional Node

Merge Node

Merge Nodes are used when the goal is to bring back the execution of divergent paths into one path. You can configure the number of inputs to a Merge Node and choose between “Await All” or “Await Any” as your merge strategy. The merge strategy determines the logic that will continue workflow execution.

Merge Node

Final Output Node

The Final Output Node represents the end of your workflow. Your workflow may have multiple Final Output Nodes if the execution has been branched off from an upstream node.

A name for the output and an output type must be configured here because the response streamed back from the endpoint (when the workflow is taken to production) has this information included.

Final Output Node

Error Node

The Error Node enables you to reject the full workflow, terminating execution with an error event wherever you define it in your execution flow. There are two types of errors you could raise with this node:

  • Pass-through - Use an Error output from an upstream node and pass it through to this node.
  • Custom - Define your own String output that this node will use as an error message

Error Node

Connecting Workflow Nodes and Defining Variables

Workflow nodes are connected by linking the output of one node to the input of another node. For any node the variables can be populated either by the results of an upstream node or the values of global variables.

When 2 nodes are successfully connected there’s a solid purple line between the nodes and the connection points turn blue. Here’s an example of a workflow that’s connected successfully:

Connecting Workflow Nodes and Defining Variables

Running a Workflow

Each variable in a node can either take the value of an upstream node or the value can be defined globally. To define them globally, you can populate them in the Input Variables dropdown before running a workflow. You can define as many scenarios as you want, each scenario is a unique set of input values that will be sent to the workflow.

Variables can be added one-by-one using the Add button or automatically using Auto-Add. Auto-Add looks at all the variables in the workflow and adds them to the scenario.

Workflow Inputs

Once all the variables are selected for each prompt (either as values of upstream nodes or defined globally), you are now ready to Run your workflow!

When you Run the Workflow (purple button on the top right corner), you will see the execution path of the Workflow in green and the intermediate results at each step of the workflow. If the results at the end of the Workflow look surprising then may be a good idea to check what the responses look like at each step.

Here’s an example of a workflow that’s executed successfully:

Executed Workflow

Node Mocking

Workflow development is best done iteratively. However, this can become prohibitively expensive both in terms of token consumption and runtime if there are Prompt Nodes defined early in the Workflow that you have to frequently re-run just to get to the part of the Workflow that you actually want to test. To help speed up Workflow development, you can mock out the execution of a given node. This will skip the node’s execution and return the hard-coded output(s) you define rather than running the node itself.

Workflow Node Mocking

Once defined, you can easily toggle the mock on and off to go back and forth between mocking the node and actually executing the Prompt to see your Workflow work end-to-end. This also allows you to save your mocks without needing to delete them when you’d like to actually execute the node. During a workflow run, nodes that are mocked will be outlined in yellow to differentiate from nodes that are actually executed.

Workflow Node Mocking

These mocks are only defined within the context of Workflow Sandboxes, and are defined per Scenario. They do not get deployed with your Workflow Deployments and do not affect behavior when invoking Workflow Deployment APIs.

The following nodes support mocking:

  • Prompt Nodes
  • Subworkflow Nodes

Check out the video below for a full demo of Workflow Node Mocking.