The Cost evaluator calculates costs based on actual token usage returned by the LLM Provider and it’s public pricing.

Examples

All costs are displayed in USD currency.

Setting Cost Thresholds

1

Select the Cost Evaluator from the action menu.

2

Link a dataset and provide the input to the evaluator using the comparison operator.

  1. less than: Keep costs below your threshold
  2. greater than: Flag expensive responses above your limit
  3. equal to: Match exact cost requirements
3

After executing, you will receive the results for all the prompts that passed or failed the evaluation.