About Workflows
Workflows is a product in Vellum’s LLM dev platform that helps you quickly prototype, deploy, and manage complex chains of LLM calls and the business logic that tie them together. We solve the “whack-a-mole” problem encountered by companies that use popular open source frameworks to build AI applications, but are scared to make changes for fear of introducing regressions in production.
The Workflows UI consists of a graphical app builder where you can string together various nodes and test various input values through this system. Each prompt can also be tested extensively through Playground & Test Suites. When implemented effectively, Workflows can help you build advanced LLM applications
Supported Node Types
The names of nodes can be edited in-line once the node has been added to the UI. You can add as many nodes as you want to your Workflow. Since there is extreme flexibility in how Workflows can be created, you can modify your architecture to your desire by starting from these building blocks. Common architectures are shared in a separate help center doc (Common Workflow Architectures) and will be updated over time.
Drag and drop a node from the side panel into the UI and start building
Prompt Nodes
A core part of any LLM application. This node represents a call to a Large Language Model. Similar to Vellum Prompts, you can use test out models from any of the major providers or open source community, including OpenAI, Anthropic, Cohere, Google, Mosaic, Falcon-40b & Llama-2.
Upon creating a Prompt Node you’ll be asked to import a prompt from an existing Deployment, Sandbox, or create one from scratch. Prompts are defined by their variables, prompt template, model provider, and parameters. Refer to this help center article to learn more about our prompt syntax (Vellum Prompt Template Syntax).
Templating Nodes
The Templating Node allows you to perform custom data transformations on a set of defined inputs to create a new output. You can use this to define constants, manipulate data before feeding into a prompt, or massage a response to a format to your liking.
Check out our common data transformation templates for some common examples.
Search Node
The Search Node returns results from a Document Index stored inside Vellum Search. Once your documents are uploaded in an index (details on how to do that here: Uploading Documents), you can start using them in a Workflow.
The index in a Search Node can be fixed for the Workflow or chosen dynamically based on the output of an upstream node. Additional configuration options, similar to the ones in Vellum Search are also available in the Search Node.
API Node
The API Node invokes an API endpoint and returns back the status code, raw output, and JSON output if applicable. These APIs can be either publicly accessible or privately defined within your backend through the help of Authorization headers and Secrets. Simply define a URL, HTTP Method, relevant additional headers, and the body that you would like to send to the desired endpoint.
Code Execution Nodes
The Code Execution Node empowers you to include custom logic defined directly in the workflow. You can even import custom public packages within the node’s logic. We support the following languages:
- Python
- TypeScript
Subworkflow Nodes
Subworkflow Nodes are essential for managing giant, complex workflows. Define reusable groups of nodes in one Workflow Sandbox and have them directly accessible upon deployment from any other workflow in your workspace. Subworkflow nodes also support release tag specification, allowing you the option to always invoke the latest workflow, or pinning to a specific release tag defined by you.
Check out the video below to see Subworkflow Nodes in action!
Conditional Node
Conditional Nodes are extremely powerful because they can help you diverge the execution path of your Workflow based on the results of an upstream node. The Conditional Node supports as many if-else-if conditions as you’d like and the rules can be grouped / nested within each other.
The number of exit options from a conditional node equal the number of if-else-if conditions created on the node
Merge Node
Merge Nodes are used when the goal is to bring back the execution of divergent paths into one path. You can configure the number of inputs to a Merge Node and choose between “Await All” or “Await Any” as your merge strategy. The merge strategy determines the logic that will continue workflow execution.
Final Output Node
The Final Output Node represents the end of your workflow. Your workflow may have multiple Final Output Nodes if the execution has been branched off from an upstream node.
A name for the output and an output type must be configured here because the response streamed back from the endpoint (when the workflow is taken to production) has this information included.
Error Node
The Error Node enables you to reject the full workflow, terminating execution with an error event wherever you define it in your execution flow. There are two types of errors you could raise with this node:
- Pass-through - Use an
Erroroutput from an upstream node and pass it through to this node. - Custom - Define your own
Stringoutput that this node will use as an error message
Connecting Workflow Nodes and Defining Variables
Workflow nodes are connected by linking the output of one node to the input of another node. For any node the variables can be populated either by the results of an upstream node or the values of global variables.
When 2 nodes are successfully connected there’s a solid purple line between the nodes and the connection points turn blue. Here’s an example of a workflow that’s connected successfully:
Running a Workflow
Each variable in a node can either take the value of an upstream node or the value can be defined globally. To define them globally, you can populate them in the Input Variables dropdown before running a workflow. You can define as many scenarios as you want, each scenario is a unique set of input values that will be sent to the workflow.
Variables can be added one-by-one using the Add button or automatically using Auto-Add. Auto-Add looks at all the variables in the workflow and adds them to the scenario.
Once all the variables are selected for each prompt (either as values of upstream nodes or defined globally), you are now ready to Run your workflow!
When you Run the Workflow (purple button on the top right corner), you will see the execution path of the Workflow in green and the intermediate results at each step of the workflow. If the results at the end of the Workflow look surprising then may be a good idea to check what the responses look like at each step.
Here’s an example of a workflow that’s executed successfully:
Node Mocking
Workflow development is best done iteratively. However, this can become prohibitively expensive both in terms of token consumption and runtime if there are Prompt Nodes defined early in the Workflow that you have to frequently re-run just to get to the part of the Workflow that you actually want to test. To help speed up Workflow development, you can mock out the execution of a given node. This will skip the node’s execution and return the hard-coded output(s) you define rather than running the node itself.
Once defined, you can easily toggle the mock on and off to go back and forth between mocking the node and actually executing the Prompt to see your Workflow work end-to-end. This also allows you to save your mocks without needing to delete them when you’d like to actually execute the node. During a workflow run, nodes that are mocked will be outlined in yellow to differentiate from nodes that are actually executed.
These mocks are only defined within the context of Workflow Sandboxes, and are defined per Scenario. They do not get deployed with your Workflow Deployments and do not affect behavior when invoking Workflow Deployment APIs.
The following nodes support mocking:
- Prompt Nodes
- Subworkflow Nodes
Check out the video below for a full demo of Workflow Node Mocking.