Generate Stream

GA
POST

Generate a stream of completions using a previously defined deployment.

Note: Uses a base url of https://predict.vellum.ai.

Request

This endpoint expects an object.
requests
list of objectsRequired
The generation request to make. Bulk requests are no longer supported, this field must be an array of length 1.
deployment_id
stringOptional

The ID of the deployment. Must provide either this or deployment_name.

deployment_name
stringOptional

The name of the deployment. Must provide either this or deployment_id.

options
objectOptional
Additional configuration that can be used to control what's included in the response.

Response

This endpoint returns a stream of object
delta
object
POST
1curl -X POST https://predict.vellum.ai/v1/generate-stream \
2     -H "X_API_KEY: <apiKey>" \
3     -H "Content-Type: application/json" \
4     -d '{
5  "requests": [
6    {
7      "input_values": {
8        "string": {}
9      }
10    }
11  ]
12}'
Streamed Response
Built with