Once in production, there’s a 3 step process to add search results in your queries at run-time:
- Call Search API to obtain relevant context (details below)
- Format the returned context and include as a single variable value when making requests to a Vellum Deployment
- Pass search results to request endpoint while calling the LLM
For Step 3, make sure you have a variable in Vellum Playground where search results are entered. Details to set that up are here: Running Searches in Playground
Search API
There’s a code snippet for the Search API in the Document Index. There are 3 variables to call the API:
- index_name - Index that is searched across
- query - Search query (usually a user input)
- options - Optional configuration that drives search behavior. Namely used to determine the max number of results returned in the response. You can also use:
- weights - to change the prioritization between keyword matches vs semantic similarity
- result_merging - to automatically merge overlapping chunks into larger chunks without redundant content
- filters - to perform rule-based filtering prior to matching on keywords / semantic similarity. For more info, see Metadata Filtering
