Introduction to Prompt Deployments
Now that you’ve used Vellum Playground for prompt engineering and have a prompt that clears your test cases, you’re ready to start making requests against it. In production, Vellum acts as a high reliability, low latency proxy between your application and the underlying model provider.
By deploying a Prompt through Vellum and integrating a 10-line code snippet you get:
- Observability into individual completions and their quality: Tracking completions & measuring quality
- Version Controlled changes to prompts/model without updating code: Changing prompts in production & versioning
- Request Replay to back-test changes and avoid regressions: Backtesting with Vellum
- Monitoring of aggregate data to spot trends: Monitoring production traffic
Let’s take a look at how to actually deploy a Prompt in Vellum
Creating a Prompt Deployment
View Deployment Details
The Deployment Overview page shows you details about the currently live version of the Prompt.
Integrating w/ Vellum’s API
The Deployment Overview page also contains code snippets to make integration simple. We support Python & Typescript clients and have an option to make Curl requests. Optionally, you can also integrate with our Actuals Endpoint to start keeping track of output quality for monitoring and eventually fine tuning. More details about this in the completions & quality help center article.
Note that our full API docs can be found at docs.vellum.ai
Changing Prompts in Production
With Vellum, you can make changes to your prompts in production without having to make any code changes! This might be useful for a variety of reasons:
- When you encounter edge cases in production, you may want to tweak the prompt to accommodate for them
- A new model comes out and can provide similar quality at lower cost or lower latency
- Product requirements change and a non-technical member of the team with the proper permissions wants to make changes
You can do this by updating a Prompt Deployment. All updates are version-controlled and past versions can be immediately reverted to at any time (no code chnages required).
Updating a Prompt Deployment
Find the Prompt Sandbox you’d like to deploy and click the “Deploy” button.
This’ll provide the option to update an existing deployment or create a new one. Select “Update Existing Deployment” and choose the deployment you’d like to update.
Note that code changes will likely be required if you change which input variables the Prompt replies on.
Prompt Versioning
After a Prompt Deployment is updated, you’ll find a new entry in the “History” tab. You can visually inspect how the Prompt has changed over time across versions. You can also revert to prior versions at any time. After reverting to a prior version, it’s immediately live – no code changes required.
