> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.vellum.ai/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.vellum.ai/_mcp/server. # Changelog | March, 2025 ## Gemini Function Calling Support **March 31st, 2025** We've added support for function calling using Google's Gemini models. ## Prompt Comparison/Diffing **March 28th, 2025** You can now compare two different Prompt Variants side-by-side and see a diff of the changes between them. This is useful for understanding the differences made between one version of a Prompt and another. You can access this feature by clicking on the "View Diff" button in the top right of a Prompt Sandbox's Comparison Mode. ![Prompt Comparison View Diff Button](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2025-03/prompt-comparison-view-diff-button.png) This will open a modal with a side-by-side comparison of the two Prompt Variants. ![Prompt Comparison Modal](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2025-03/prompt-comparison-view-diff.png) ## New Workflow Deployment Executions APIs **March 26th, 2025** We now have two new apis [List Workflow Deployment Executions](/api-reference/workflows/deployments/list-workflow-deployment-event-executions) and [Retrieve Workflow Deployment Execution](/api-reference/workflows/deployments/workflow-deployment-event-execution) for listing your Executions for a specific Workflow Deployment, and retrieving any specific Workflow Deployment Execution. ## Support for Gemini 2.5 Pro Model **March 25th, 2025** We've added support for Gemini 2.5 Pro Experiemental (Version 03-25) with a whopping 1M input token context window and 64k output tokens via [Google's Gemini API](https://blog.google/technology/google-deepmind/gemini-model-thinking-updates-march-2025/#gemini-2-5-pro) ## Select all rows in Evaluation Reports **March 25th, 2025** You can now select all rows on a page in the Evaluation Report by clicking the checkbox in the header. This selection persists across pages, with the checkbox applying only to the current page. ![](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2025-03/evaluation-table-select-all-rows.png) ## Workflow Execution Detail Cost Breakdown **March 24th, 2025** We are now displaying the aggregated cost for a Workflow Execution's inside of our Workflow Execution's Detail Page. This aggregation is available for all Workflow spans within your Workflow Execution, giving a holistic view of your total execution cost. ![Cost Breakdown](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2025-03/workflow_details_cost.png) ## Deployment Release Reviews Inspired by Github PR reviews, you can now provide reviews on a Prompt or Workflow Deployment Release after it has been deployed. This is helpful if your company mandates a review process for all changes that make it to production. To see Release Reviews in action, check out the demo video below: ## Support for Llama 3.3 70B via Cerebras **March 21st, 2025** We've added support for Llama 3.3 70B via [Cerebras AI](https://inference-docs.cerebras.ai/introduction) ## New Webhook and Datadog Events **March 20th, 2025** We are releasing 2 new events for Webhooks and Datadog integrations: * `workflow.execution.initiated` * `workflow.execution.fulfilled` These events can be used for better visibility in the execution of a Workflow, such as retrieving the input values used to kick it off, as well as the output values that it produced. You might also use these events to track latency, data drift, and more. ## Retry and Try Node adornments **March 22nd, 2025** Error handling and retry logic have both been traditionally quite cumbersome in Vellum. Often, you want to catch errors around a single node and "adorn" it with different error handling behavior. Today we are releasing Retry and Try *Node Adornments*. ![SDK Node Adornments](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2025-03/sdk-node-adornments.png) Adornments are accessible in the Node sidepanel after clicking on a Node. They act as Nodes *themselves*, that reference the node it wraps as a single node Subworkflow: * Retry Node Adornments repeatedly invokes the Node until it either succeeds or the max number of attempts are hit. * Try Node Adornments attempts to invoke the Node, and continues with an `Error` output if it fails. Monitoring views for Node Adornment invocations show as if the targeted Node was invoked as a single node Subworkflow: ![Node Adornment Monitoring](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2025-03/node-adornment-monitoring.png) ## Monitoring View Overhaul **March 22nd, 2025** We're excited to introduce completely revamped Monitoring interfaces for Prompts and Workflows Deployments that bring significant improvements to how you visualize and analyze your AI system performance. ![Revamped Monitoring View](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2025-03/grafana-monotoring-revamp.png) Our new Grafana-based implementation loads data faster and handles metric processing more efficiently, making it easier than ever to monitor your AI applications at scale. Key improvements include: * Faster dashboard load times * Improved date range and Release Tag selection * Zoom in on specific time ranges ## Copy/Paste Prompt Variants **March 20th, 2025** It is now possible to copy a JSON representation of a Vellum prompt to your clipboard and then paste over another Prompt Variant. This is especially useful if you want to go back to a prior history state, copy just one Prompt, and then paste it in your current live draft. ![Copy/Paste Prompt Variants](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2025-03/copy-paste-prompt-variants.png) ## Updated Workflow Node Inputs **March 20th, 2025** Workflow Nodes used to use a chunky UI for configuring their inputs. We've now updated this UI to be simpler by making use of what we call "Expression Inputs." These Node inputs are functionally the same as before, but tease soon-to-come functionality where you'll be able to define more complex expressions for what gets passed to a Node. ![Workflow Node Expression Inputs](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2025-03/workflow-node-expression-inputs.png) ## Metric Setup Improvements **March 20th, 2025** We've significantly simplified the process of configuring and editing Evaluation Metrics in Vellum. You can now manage your evaluation metrics directly within an evaluation report – adding, editing, and removing metrics – all without leaving the Prompt/Workflow that you're evaluating. ![Add a Metric](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2025-03/add-metric.png) You'll also find a new Metric Settings button that opens a modal where you can configure your metrics directly ![Metric Settings](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2025-03/metric-settings.png) ## Datadog Integration **March 20th, 2025** It's now possible to receive real-time updates about actions taking place in Vellum using Datadog. ![Datadog](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2025-03/datadog.png) From the organization settings page, you can configure a Datadog integration with a custom list of events you care about. This is useful if your organization already uses Datadog, or you'd like to leverage Datadog's monitoring, alerting, and BI capabilities using your Vellum data. ## All New Workflows are SDK-Enabled **March 20th, 2025** About two months ago, we began the Beta period of SDK-enabled Workflows. These Workflows use the new [Workflows SDK](https://github.com/vellum-ai/vellum-python-sdks/tree/main/src/vellum/workflows#--vellum-workflows-sdk--) as the underlying engine and run these Workflows in a, secure, more performant, isolated environment. The Workflows SDK enables exciting new functionality including custom nodes, custom docker runtimes, new expression inputs, Node adornments, and so much more. As of today, *all new Workflows* going forward will be SDK-enabled by default. We expect that all features from the old Workflows engine are supported, with the exception of "Run from Node", which we hope to reenable later this month. To revert a Workflow that is SDK-enabled back to the legacy Workflows engine, simply toggle off this checkbox from the Workflow settings: ![SDK Compatible Workflow](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2025-01/toggle-sdk-compatibility.png) If you have an existing Workflow and would like to make it SDK-enabled, you can use the same toggle. ## JSON Outputs for Prompt Nodes **March 19th, 2025** Until now in SDK enabled workflows, you only had string or array outputs for Prompt Nodes. With this update, you will now be able to reference a Prompt Node's JSON output if the `json_mode` or `json_schema` parameter is enabled. This is useful as it enables you to not have to do any additional casting in a Templating Node or Code Execution Node. ![prompt\_node\_json\_output](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2025-03/prompt_node_json_output.png) ## PDF Support for Gemini 2.0 Flash Models **March 19th, 2025** The following Gemini 2.0 Flash models now allow for drag-and-drop PDF documents to be used within your Prompts. * Gemini 2.0 Flash Experimental * Gemini 2.0 Flash Experimental Thinking Mode * Gemini 2.0 Flash ## Automatic Evaluations Setup **March 14th, 2025** Now, when navigating to the Evaluations tab of a Prompt/Workflow for the first time, Vellum will auto-generate an initial Test Suite for you. We'll automatically create one Test Case per Scenario found in the Sandbox and prepare everything needed for you to add Metrics and Ground Truth data. ![Auto Evaluation Report Initialization](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2025-03/auto-eval-report-initialization.png) ## File Hosting for Images and PDFs **March 11th, 2025** Until now, when you provided an image to a Prompt in Vellum, it'd either have to be a public URL or the base64 encoded representation of the image. With this update, we now support secure file hosting, such that when you drag-and-drop an image (and also now PDFs!) into a Prompt, Vellum securely hosts the file on your behalf. The end result is that you can now provide much larger images and PDFs to LLMs within Vellum without worrying about a decrease in performance or page load times. ## PDFs as a Prompt Input **March 11th, 2025** Recently, certain model providers like Anthropic have begun supporting PDFs as native LLM inputs using a special content type called `document` (check out their docs for details [here](https://docs.anthropic.com/en/docs/build-with-claude/pdf-support). This is similar to how you might provide a multi-modal model with an image as an input, but now you can provide a PDF as well. Vellum now also supports passing PDFs as inputs to a Prompt for models that support it. You can do this by drag-and-dropping a PDF file into a Chat History variable in a Prompt. The mechanics are very similar to how you might work with images (see details [here](https://docs.vellum.ai/product/prompts/images)). This is particularly useful for data extraction tasks, where you might want to extract structured data from a PDF and then use that data to power some downstream process. ## Support for Qwen QwQ Models via Groq **March 11th, 2025** We've added support for a variety of Qwen's newest [QwQ 32B](https://console.groq.com/docs/models#preview-models) models via Groq's preview models. We've added the following models: * QwQ 32B * QwQ 2.5 Coder 32B * QWQ 2.5 32B ## Support for Qwen QwQ 32B via Fireworks AI **March 11th, 2025** We've added support for Qwen's newest [QwQ 32B](https://fireworks.ai/models/fireworks/qwq-32b) model via Fireworks AI. ## Webhooks **March 10th, 2025** It's now possible to receive real-time updates about actions taking place in Vellum using Webhooks. ![Webhooks](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2025-03/webhooks.png) From the organization settings page, you can configure a webhook endpoint with a custom list of events you care about. You can further customize it with your own auth configurations. This is useful if you'd like to store Vellum monitoring data in your own external data stores. For example, you might save events to a Data Warehouse to power a custom health dashboard. Keep an eye out, as more event types will be added soon! ## Workflow Deployment Executions - Cost Column **March 7th, 2025** You can now see the total cost per Workflow Execution for a given Workflow Deployment in its Executions table. This toggle can be shown/hidden via the "columns" menu. ![Cost Column](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2025-03/cost_column.png) This is useful for getting a sense of how much a given AI use-case costs to support. This column will be populated for new Workflow Executions going forward and sums the costs associated with all Prompt invocations within the Workflow's execution (included nested invocations within Subworkflow Nodes, Map Nodes, etc.). We'll be exposing more cost metrics throughout Vellum in the coming weeks. Stay tuned! ## Prompt Sandbox Pagination **March 5th, 2025** We've added pagination to the Prompt Sandbox page. Now, when you have a large number of Scenarios in a Prompt Sandbox, they'll be split across multiple pages. You can navigate between pages using the pagination controls at the bottom ![Prompt Sandbox Pagination](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2025-02/prompt-sandbox-scenario-pagination.png) This should result in performance improvements for those with large Prompt Sandboxes. ## Global Search **March 3rd, 2025** We've added an eagerly-awaited-for and long-overdue feature to Vellum – Global Search 🎉 You can now search across all your Prompts, Workflows, Document Indexes, and more using the new Search bar in the Vellum side nav. ![Global Search Side Nav](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2025-02/global-search-side-nav.png) Doing so will pull up a search bar where you can search for any resource in your Workspace. You can directly navigate to the resource from the search results. ![Global Search Omnibox](https://storage.googleapis.com/vellum-public/help-docs/changelogs/2025-02/global-search-omni-bar.png) You can also access Global Search from any page through the keyboard shortcut `Cmd/Ctrl + K`. Give it a try and let us know what you think!