> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.vellum.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.vellum.ai/_mcp/server.

# Changelog | July, 2024

## Metadata Filtering in Search Nodes

*July 31st, 2024*

For a while now you've been able to supply structured JSON metadata alongside Documents and then filtering on that
metadata when making an [API call](/api-reference/document-indexes/search) to search across Documents in a Document index (see [here](/product/documents/metadata-filtering) for more info).

However, Search Nodes within Workflows didn't offer this same functionality through the UI. The workaround has been to use a
Code Node or API Node and invoke Vellum's Search API manually.

We're happy to share that the UI has reached parity with the API and you can now filter on metadata natively in Search Nodes.
You'll be able to construct arbitrarily complex boolean logic using the new Metadata Filters section of the Search Node's
Advanced settings.

![Search Node Metadata Filtering](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2024-07/search-node-metadata-filtering.png)

## Test Suite Test Case External IDs

*July 30th, 2024*

We've added a new feature to Test Suites that allows you to optionally assign an external ID to each Test Case.
This is useful if you track your Test Cases in an external system and you want to periodically sync them with Vellum.
You assign an external ID to each Test Case upon creation and then later upsert Test Cases to that Test Suite,
keying off of the external ID.

![Upload Test Suite Test Cases Modal](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2024-07/test-case-upsert-external-id.png)

## Index Page Sorting

*July 30th, 2024*

We've added another quality-of-life improvement for the index/file browser pages for Prompts, Documents, Test Suites, and Workflows. You'll now see a "Sort by" dropdown next to the other page-level controls. You can now sort both folders' and files' by created date, modified date, and label. If there are other sort fields that you'd find useful, please let us know!

![Index page sort dropdown](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2024-07/index-page-sort-closed.png)
![Index page sort options](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2024-07/index-page-sort-open.png)

## Auto-Conversion to Variable Chips on Paste

*July 30, 2024*

Building on our recent update that introduced [Prompt Variable Chips](https://docs.vellum.ai/changelog/2024/july#prompt-variable-chips), we've improved the experience by adding support for copy/pasting variables across blocks of different types. Now, when you copy text that includes a `{{ my_var }}` variable reference from a Jinja block and paste it into a Rich Text block, it's seamlessly converted into a variable chip.

## Google Vertex AI Support

*July 29th, 2024*

We now support [Google Vertex AI](https://cloud.google.com/vertex-ai) models. Previously you could only use Google AI Studio for using Google's models. You can add them to your workspace from the [models page](https://app.vellum.ai/models).

![Vertex AI Usage](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2024-07/vertex-ai.png)

## Expandable Meta Params in Retrieve Provider Payload Endpoint

*July 26th, 2024*

For a while now we've had [an API](/api-reference/prompts/retrieve-provider-payload) for compiling a Prompt and retrieving the exact payload that Vellum would send
to a model provider on your behalf. We now support a new parameter in this API – `expand_meta`. With `expand_meta`,
you can opt-in to return additional metadata relating to the compiled prompt payload. Learn more about which
fields are expandable in our [API docs here](/api-reference/prompts/retrieve-provider-payload#request.body.expand_meta).

This new field is available in our SDKs starting v0.7.3.

## Prompt Node Usage in Workflows

*July 25th, 2024*

You can now see token counts and other usage metrics appear in Prompt Node results when invoking Workflows in the Workflow Sandbox:

![Prompt Node Usage](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2024-07/prompt-node-usage.png)

This setting is now on by default, but can be toggled off in the Workflow Builder Settings.

You can also now return usage data when invoking a *Workflow Deployment* via API, by passing in `True`
to the `expand_meta.usage` parameter on either Execute Workflow endpoints.

```python
stream = client.execute_workflow_stream(
  workflow_deployment_name="demo",
  inputs=[
    WorkflowRequestInputRequest_String(
      type="STRING",
      name="foo",
      value="bar",
    ),
  ],
  event_types=["WORKFLOW", "NODE"],
  expand_meta=WorkflowExpandMetaRequest(
    usage=True
  )
)

for event in stream:
  if event.type == "NODE" and event.data.state == "FULFILLED":
    node_result_data = event.data.data
    if node_result_data and node_result_data.type == "PROMPT":
      print(node_result_data.data.execution_meta.usage)
```

## Enable/Disable All Workflow Node Mocks

*July 25th, 2024*

Mocking Prompt Nodes helps to save token usage and time when developing the later stages of your Workflow. However, once the Workflow is in a good state, it's often useful to run
the full Workflow end-to-end without mocks to make sure it all comes together. Previously, you had to enable/disable each mock individually. Now, beneath the scenario inputs there
is a toggle that allows you to enable/disable all mocks in a workflow at once.

![Enable Disable All Mocks](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2024-07/enable-disable-all-mocks.png)

## Support for Bulk Upserting Test Suite Test Cases via API

*July 24th, 2024*

For a while now we've had [an API](/api-reference/test-suites/test-cases/bulk-update) for creating, replacing, and deleting Test Cases in a Test Suite in bulk.
We now support a fourth operation in this API – upsert. With upsert, you can provide an `external_id` and a Test Case
payload. If there is already a Test Case with that `external_id`, it'll be replaced. Otherwise, it'll be created.

This new operation is available in our SDKs starting v0.6.12.

## Llama 3.1 on Groq

*July 23rd, 2024*

Meta's newest [Llama 3.1 models](https://ai.meta.com/blog/meta-llama-3-1/) are now available in Vellum through our [Groq](https://wow.groq.com/now-available-on-groq-the-largest-and-most-capable-openly-available-foundation-model-to-date-llama-3-1-405b/) integration!

## Deployed Prompt Variant Display

*July 19th, 2024*

When on the Prompt Deployment Overview page, you can now see the name of the Prompt Variant that's been deployed.
This is useful if your Prompt Sandbox has multiple Prompt Variants that you were comparing against one another
and you're not sure which one is currently deployed.

![Deployed Prompt Variant Display](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2024-07/deployed-prompt-variant-display.png)

## Improvements to Prompt Chat History Variables

*July 18th, 2024*

It used to be that Prompts that accepted a dynamic Chat History required an input variable whose name was specifically `$chat_history`.
This nomenclature caused frequent confusion and was a bit cumbersome to work with.

Now, you can name Chat History input variables whatever you want and even rename them after-the-fact. As part of this,
we've also centralized input variable definitions so that whether you want to create a String variable or a Chat History variable,
you can do so via the "Add" button in the "Input Variables" section of the Prompt Editor.

![Add Prompt Input Variable Button](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2024-07/add-prompt-input-variable-button.png)

## Copyable Text to Clipboard

*July 18th, 2024*

We’ve introduced the ability to copy Prompt Variant IDs, Document Indexes, Models, Workflow Deployment Names and IDs, Document Keys, and Prompt Deployment Names and IDs to clipboard.
This feature comes with an enhanced UI with intuitive indicators and tooltips for copyable fields.

![Copy Text to Clipboard](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2024-07/copy-text-to-clipboard-1.png)
![Copy Text to Clipboard](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2024-07/copy-text-to-clipboard-2.png)

## GPT-4o Mini

*July 18th, 2024*

OpenAI's newest [GPT-4o Mini models](https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/) `gpt-4o-mini` & `gpt-4o-mini-2024-07-18` are now available in Vellum and have been added to all workspaces!

## Prompt Variable Chips

*July 18th, 2024*

It used to be that any time you wanted to reference a variable in a Prompt, you did so using `{{ myVariable }}` syntax.
While powerful if you need to use more complex Jinja templating syntax, using double-curlies for simple variable substitution
can be a bit cumbersome.

1. They are harder to visually parse from the rest of your Prompt
2. They can get confusing when dealing with json, which also uses double-curly brackets
3. Whenever you rename a variable, you need to hunt down its usages.

To make this easier, we've introduced a new way to reference variables in Prompts: Variable Chips.
Variable Chips are small, clickable chips that you can reference in your Prompt text. You can add them by beginning to
type `{{ ` or by typing `/`. Renaming a variable automatically renames all of its references.

Variable chips can be used in the new "Rich Text" block type. New Prompt blocks will default to Rich Text, but you can
change existing blocks to Rich Text by clicking the block type dropdown in the block's toolbar and converting from Jinja
to Rich Text and vice versa.

Check out a full video demo here:

## New Layout for Sandbox Evaluations

*July 17th, 2024*

Previously, when a Prompt/Workflow had multiple Test Suites associated with it, we'd shown them all on the page at once. This made navigation difficult (you had to scroll up and down to see each) and could also lead to performance issues. We've addressed these issues updating the page layout to display just one Test Suite at a time with a searchable select input that allows you to easily load and view each table one at a time.

![Sandbox Evaluation Select](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2024-07/eval-select-closed.png)

## New "Add Document to Document Index" API

*July 16th, 2024*

We've introduced a new API for adding previously uploaded Documents to a Document Index. This API is useful when you have a Document that
had previously been added to one Document Index and you want to add it to another without having to re-upload its contents altogether. It's available in our SDKs
beginning version 0.6.10. You can find docs for this new API [here](/api-reference/document-indexes/add-document).

## Prompt Deployment Executions Table Improvements

*July 12th, 2024*

We've made several quality-of-life enhancements to the Prompt Deployment Executions table, simplifying the process of adding and editing 'Desired Output' values. The entire table has been updated to align with the design of our other tables, such as Evaluations, ensuring a familiar editing experience. Additionally, it is now easier than ever to expand/collapse and copy values.

Moreover, we've significantly improved the consistency and usability of the 'Quality' column. You can now edit quality ratings with a single click, and the 'Desired Output' column will automatically update to reflect your rating where applicable.

![Prompt Deployment Executions Table](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2024-07/executions-quality-input.png)

## Constant Values in Workflow Node Inputs

*July 11th, 2024*

It's often the case that you might want to specify a constant value as a Workflow Node Input, either as the input's primary value or as its fallback value.
The solution up until now was to specify a Templating Node, have it output a constant value, and then feed its output to the downstream Node.

Today, we are releasing the ability to inline constant values directly within Workflow Node inputs! First, start typing in the Node Input until the no options modal shows:

![New Constant Link](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2024-07/new-constant-link.png)

A modal will appear to specify your value:

![New Constant Modal](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2024-07/new-constant-modal.png)

Upon confirming, Vellum will use an icon to denote that the input value represents a constant. As part of this work, we also added icons for all other Node Input types:

![Constant Value Display](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2024-07/constant-value-display.png)

Note that constant values will always drop to the last fallback option of a given Node input, and there can only be maximum one constant defined per input.
This is due to the nature fallback values – fallbacks are only used if other values aren't available (i.e. the node that produced the value hadn't executed yet). In the case of constants, their values are always present.

## Test Case CSV Upload in Evaluation Reports

*July 9th, 2024*

We’ve introduced the ability to upload Test Cases to a Test Suite directly from within the Evaluations tab of a Prompt or Workflow. Now, you'll find an "Upload Test Cases" button in the table header of every Evaluations table, for both Workflows and Prompt Sandboxes whereas previously, you needed to first navigate to the Test Suite itself and upload from there.

![Test Case Upload in Evaluation Reports](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2024-07/eval-page-test-case-upload.png)

## Index Page List View

*July 3rd, 2024*

We’ve introduced a list-view toggle to the index/file browser pages for Prompts, Documents, Test Suites, and Workflows. Your preferred view will be saved automatically by entity type, allowing you to, for instance, default to list view for Prompts and grid view for Documents.

![Index Page List View](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2024-07/index-page-list-view.jpg)

## Collapsible Index Page Sections

*July 2nd, 2024*

You can now collapse sections on the index/file browser pages for Prompts, Documents, Test Suites, and Workflows. Simply click the heading of any section to toggle the visibility of all folders and items within that section.

![Collapsible Index Page Sections](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2024-07/index-page-collapse.png)