The Cost evaluator calculates costs based on actual token usage returned by the LLM Provider and it’s public pricing.

​
Examples

All costs are displayed in USD currency.

​
Setting Cost Thresholds

1

Select the Cost Evaluator from the action menu.

Product Dynamic Columns Pn
2

Link a dataset and provide the input to the evaluator using the comparison operator.

Product Dynamic Columns Pn
  1. less than: Keep costs below your threshold
  2. greater than: Flag expensive responses above your limit
  3. equal to: Match exact cost requirements
3

After executing, you will receive the results for all the prompts that passed or failed the evaluation.

Product Dynamic Columns Pn