Function Calling in Prompt Evaluation
Prompts that output function calls can now be evaluated via Test Suites. This allows you to define Test Cases consisting of the inputs to the prompt, and the expected function call, then assert that there’s a match. For more, check out our docs.
Subworkflow Node Streaming
May 7th, 2024
Subworkflow Nodes can now stream their output(s) to parent workflows.
This allows you to compose workflows using modular subworkflows without sacrificing the ability to delivery incremental results to your end user.
Note that only nodes immediately prior to Final Output Nodes can have their output(s) streamed.
Default Test Case Concurrency in Evaluations
May 4th, 2024
You can now configure how many Test Cases should be run in parallel during an Evaluation. You might lower this value if you’re running into rate limits from the LLM provider, or might increase this value if your rate limits are high.
