Changelog | August, 2024
Anthropic Google Vertex AI Support
August 30th, 2024
We now support using Anthropic’s Claude 3.5 Sonnet, Claude 3 Opus and Claude 3 Haiku Models with Google Vertex AI. You can add them to your workspace from the models page.
Anthropic Tool Use API for Function Calling
August 30th, 2024
We now support using Anthropic’s Tool Use API for function calling with Claude 3.5 Sonnet, Claude 3 Opus and Claude 3 Haiku Models. Previously Anthropic function calling had been supported by shimming function call XML into the prompt.
Prompt Node Linked Deployments
August 29th, 2024
We have reworked the relationship of how Prompt Node’s interact with Deployments. Previously, there was:
- No way to update a Prompt in one spot and have it update in multiple Workflows
- Confusing UX around what it meant to import a Prompt
Today we are releasing this new setup modal that appears when you create a Prompt Node:
The setup modal contains a new Link to Deployment option. This is a Prompt Node that references a Prompt Deployment directly with a Release Tag. This
allows for Workflows both in the Sandbox and as a Deployment to automatically pick up changes to the underlying Prompt without needing to update the Workflow
by pointing to LATEST. To maintain a specific version of a Prompt Deployment, you can specify a user-defined Release Tag to keep the Prompt Node pinned to
a specific version. In this way, they now work exactly as Subworkflow Nodes when you select Link to Deployment there:
Workflow Executed By Filterable Column
August 29th, 2024
Earlier this month, we restricted the Workflow Deployment Executions table to only show executions invoked via API requests. This helped to filter out all of the noise from other contexts in which a Workflow Deployment could be invoked, bringing focus to only data from production traffic. However, we’ve found that are still other contexts in which it’s useful to see Workflow Executions.
You’ll now find a new Executed By column that shows what the immediate “parent” context was in which the Workflow was executed. This table is filtered down to just API Request by default, but you can opt in to include additional contexts, such invocation as a Subworkflow via a parent Workflow:
Tool Choice Parameter Support for OpenAI
August 28th, 2024
We are excited to announce that you can now natively specify how prompts handle functions using OpenAI’s Tool Choice parameter. With the Tool Choice parameter, you can now dictate exactly when tools are used, allowing more precise and effective control of your prompt tools.
This feature is now available across all OpenAI models that support functions.
Add Metadata to Workflow Executions
August 27th, 2024
You can now add metadata to your Workflow Executions through the API. This is useful for tracking additional information about your executions, such as the source of the request or any other custom data you want to associate with the execution.
This metadata is visible in the Workflow Execution Details page in the Vellum UI.
You can view more information at the API documentation.
New Workflow Editor Beta Release
August 26th, 2024
Our new Workflow Editor is now available as an opt-in beta release. Next time you open the Workflow Editor, you’ll see an announcement with the option to turn on the new Editor experience.
We’ve made a ton of improvements to the Editor UI, and more improvements are in the works. You should find that your Workflows are easier to navigate and edit, and more performant.
The beta can also be toggled on or off in the workflow builder settings at any time.
We’d love to get your feedback about the new experience, so please let us know what you think!
View the Provider Payload on a Workflow’s Prompt Node
August 26th, 2024
You can now view the compiled provider payload on a Workflow’s Prompt Node. This is useful for debugging and understanding the exact data that was sent to the provider during a run, especially if you got some unexpected results.
Merging Two Adjacent Prompt Blocks
August 26th, 2024
Merging two adjacent prompt blocks in the prompt editor is now possible! This feature is especially useful when you want to combine two prompt long prompt blocks into one.
You can find this button in the top right drop down in the prompt editor.
Only blocks of the same type can be merged. For example, you can merge two rich text blocks or two Jinja blocks, but you cannot merge a rich text block with a Jinja block. You can easily convert between the two, however, by clicking the three dots in the top right of the block and selecting “Convert to Jinja” or “Convert to Rich Text”.
Asynchronous Exports of Evaluation Reports
August 26th, 2024
Exports of evaluation reports are now asynchronous. You can export your evaluation report along with its results in CSV or JSON format, and an email will be sent to you once the export is done.
This change is especially useful for large evaluation reports, where the export process and download can take some time.
JSON Schema Editor with $ref Support
August 26th, 2024
Vellum let’s you define JSON Schemas in a few different places throughout the app to do things like define Structured Outputs and Function Calls. Previously this UI was just a simple form that allowed you to define basic JSON schemas. This UI has been improved to support direct edits via a raw JSON editor.
From here, you can edit your JSON schema directly. This raw editor allows you to make use of all features supported by the JSON Schema spec, even if they may not yet be supported by our basic form UI. For example, you can now defined references (i.e. $ref) like this:
as references:
Support for Excel Files in Document Indexes
August 23rd, 2024
We now support uploading .xls and .xlsx files to Document Indexes for indexing and searching.
Prompt Caching Support for Anthropic
August 22nd, 2024
Anthropic recently released some exciting API changes that allow for Prompt Caching. This new feature allows for caching of frequently used portions of your Prompt for up to 5 minutes; which reduces the latency and cost of subsequent executions that include the same Prompt context.
This powerful feature is now natively supported within Vellum! In order to use it, simply toggle the new cache options on a given Prompt Block for the supported Claude Sonnet 3.5 and Claude Haiku 3.0 models.
Prompt Execution Pages
August 22nd, 2024
If you wanted to drill into a single Prompt Execution, previously you’d have to navigate to the Prompt Deployment’s Executions table and try to filter for the specific Execution ID you’re looking for. Now each row has a navigable link accessible from the table:
This will navigate you to a dedicated page representing that specific Prompt Execution. From here, you can see details about the Execution like the raw HTTP data sent to and from the provider, any actuals recorded, the Vellum inputs and outputs to the prompt, and more!
Historical Versions of Entities in Evaluation Reports
August 21st, 2024
Earlier this month, we introduced Evaluation Report History, which allows you to view a history of all Evaluation runs and revisit the results of any prior state. We’ve now enhanced this feature by adding the ability to preview or navigate directly to the version of the Workflow or Prompt as it existed during that specific run.
GPT-4o Finetuning
August 19th, 2024
OpenAI’s newest GPT-4o models gpt-4o-2024-08-06 and gpt-4o-mini-2024-07-18 are now available as base models to add as OpenAI finetuned models.
Workflow Execution Replay & Scrubbing
August 18th, 2024
You can now replay and scrub through the execution of a Workflow in Workflow Sandbox and Deployment Execution Details pages. This feature is particularly useful for debugging and understanding the flow of your Workflow, especially if it contains loops where a single node might be run more than once.
OpenAI Structured Outputs Support
August 15th, 2024
OpenAI released some API changes that allow their newest models to support Structured Outputs. This powerful new feature enables developers to strictly define the expected JSON object schemas from the model as part of the response through a model parameter, or through a function call. This new functionality is now natively integrated within Vellum!
To use within the context of Function Calling, simply toggle on the Strict checkbox for any given Function Call:
To enable Structured Outputs as part of a general OpenAI response, configure the JSON Schema setting as part of model parameters:
Both places come with upload/download functionality built into the form. Note that for function calling, this means we’ve reduced the scope of the upload/download to be just the Parameters
JSON schema field. This allows schemas to be cross-compatible between either location since we are working with an open specification.
Native JSON Input Variable Support for Prompts
August 14th, 2024
Vellum Prompts have historically been able to accept strings and chat histories as dynamic inputs to their template.
If you wanted to operate on JSON, you’d have to pass it as a string and then parse it within the Prompt itself
(i.e. perform json.loads() within a Jinja Block).
Vellum Prompts now support native JSON as inputs! When you add an input variable to a Prompt, you can now select the new “JSON” type.
JSON input values will render as prettified JSON objects when referenced in Rich Text Blocks and can be operated on directly
without the need for json.loads() when referenced in Jinja Blocks.
Workflow Deployment Executions Filtered to Just API Executions
August 12th, 2024
Our Workflow Deployment Executions page used to list all executions of a Workflow Deployment, no matter where they were invoked from. However, this would often get confusing because you’d see a mix of results from both eval runs and production traffic in the same view.
Our Workflow Deployment Executions page now filters down to just those executions that were invoked via the API. Executions from evaluations are still accessible from within the Evaluations UI by hovering over a row and clicking the “View Workflow Details” button:
Add Specific Releases to Evaluation Reports
August 12th, 2024
We’ve updated Evaluation Reports to give you more control over the releases you evaluate. Previously, you could only add the latest release of a Deployment to your reports. Now, you can select specific releases by their tag, allowing you to compare different versions within your Evaluation Reports.
Workflow Sandbox Latency
August 9th, 2024
You can now view the latency of Workflow Sandboxes and their Nodes. To enable viewing latency click the Workflow Sandbox settings gear icon in the top right and turn on the “View Latency” option.
Prompt Sandbox Cost Tracking
August 9th, 2024
You can now see the dollar cost of a Prompt’s execution within both a Prompt Sandbox’s Prompt Editor and Comparison Mode views. These costs are calculated using model providers’ publicly available pricing data in conjunction with the number of input/output tokens used.
If you’re curious about a given model’s pricing, you can view details in the Model’s detail page.
GPT-4o 2024-08-06
August 6th, 2024
OpenAI’s newest GPT-4o model gpt-4o-2024-08-06 is now available in Vellum and has been added to all workspaces!
Deployment Descriptions
August 2nd, 2024
You can now update your Prompt and Workflow Deployments to include a human-readable description. This is useful for giving other members of your team a high-level summary of what the Prompt or Workflow does without needing to parse through the configuration or control flow.
Once set, the description will appear as part of the Deployment Details page within the Deployment Info section:
Evaluation Report History
August 1st, 2024
It used to be that you could only view the latest set of Evaluation results for a given Prompt or Workflow. But now, you can view a history of all Evaluation runs and go back to view the results of any prior state.
This is particularly helpful if you want to do things like compare the results of two different Evaluation runs, download the results of a past Evaluation run, or simply view the Test Cases that existed at that time.