Streamline Your Prompt Deployment with Vellum

Introduction to Prompt Deployments

Now that you’ve used Vellum Playground for prompt engineering and have a prompt that clears your test cases, you’re ready to start making requests against it. In production, Vellum acts as a high reliability, low latency proxy between your application and the underlying model provider.

By deploying a Prompt through Vellum and integrating a 10-line code snippet you get:

• Observability into individual completions and their quality: Tracking completions & measuring quality
• Version Controlled changes to prompts/model without updating code: Changing prompts in production & versioning
• Request Replay to back-test changes and avoid regressions: Backtesting with Vellum
• Monitoring of aggregate data to spot trends: Monitoring production traffic

Let’s take a look at how to actually deploy a Prompt in Vellum

Creating a Prompt Deployment

View Deployment Details

The Deployment Overview page shows you details about the currently live version of the Prompt.

Integrating w/ Vellum’s API

The Deployment Overview page also contains code snippets to make integration simple. We support Python & Typescript clients and have an option to make Curl requests. Optionally, you can also integrate with our Actuals Endpoint to start keeping track of output quality for monitoring and eventually fine tuning. More details about this in the completions & quality help center article.

Note that our full API docs can be found at docs.vellum.ai

Changing Prompts in Production

With Vellum, you can make changes to your prompts in production without having to make any code changes! This might be useful for a variety of reasons:

1. When you encounter edge cases in production, you may want to tweak the prompt to accommodate for them
2. A new model comes out and can provide similar quality at lower cost or lower latency
3. Product requirements change and a non-technical member of the team with the proper permissions wants to make changes

You can do this by updating a Prompt Deployment. All updates are version-controlled and past versions can be immediately reverted to at any time (no code chnages required).

Updating a Prompt Deployment

Find the Prompt Sandbox you’d like to deploy and click the “Deploy” button.

This’ll provide the option to update an existing deployment or create a new one. Select “Update Existing Deployment” and choose the deployment you’d like to update.

Prompt Versioning

After a Prompt Deployment is updated, you’ll find a new entry in the “History” tab. You can visually inspect how the Prompt has changed over time across versions. You can also revert to prior versions at any time. After reverting to a prior version, it’s immediately live – no code changes required.