Vellum supports several of the industry's most popular models by default available in your workspace right away. However, you may wish to use a custom model that gives your business some additional advantage not provided by these off the shelf models, such as higher rate limits or more domain-specific training. These models can also be set up for use within Vellum!

There are several different types of models that fall under this category. We outline the steps below on how to add these models to your workspace.

OpenAI Fine-tuned Models

When you fine-tune a model on OpenAI, you'll be given a model identifier that uniquely identifies your new model. You can see and use this model natively in Vellum by adding it to Vellum via the below unofficial API. Note that this API is not yet found in our API docs as this is an early-access feature and subject to change.

Azure OpenAI

Deploying an OpenAI model in your Azure environment has a number of advantages compared to off-the-shelf OpenAI models, including higher rate limits and more predictable uptime. The Azure deployment will come with a endpoint URL and an API Key. To setup the model in your workspace:

1. Add your api key as a Workspace Secret from within the API Keys page, named AZURE_OPENAI_API_KEY
2. Reach out to us on Slack specifying which model you deployed and what endpoint it can be reached at

We'll get started on setting up the model in your workspace and it should be accessible within Vellum the same day.

AWS Bedrock Claude

Deploying Claude on AWS Bedrock within your AWS managed infrastructure has a number of advantages compared to off-the-shelf Anthropic models, including higher rate limits and more predictable uptime. Once you finish setting up the model on Bedrock, create an IAM user that has permissions to invoke the model. Here's an example IAM Policy document:

Be sure to replace the 123412341234 with your actual AWS Account ID. Create an AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY key pair for that user, and set those values as Workspace Secrets of the same name, found within the API Keys tab in Vellum.

Once your secrets are set up, notify us on Slack which AWS region your model is hosted in and we can get it connected to your Workspace.

Request a Model

Don't see a custom model listed here but want to try it within Vellum? Reach out to us on Slack for support!