The Response Length evaluator counts the size of your prompt responses using three different measurement units.

Measurement Units

Response length offers three units of measurement:

  • Tokens: Exact token usage returned by your LLM provider for the prompt response.
  • Words: Count the number of words in your prompt response. A word is defined as any sequence of characters separated by whitespace.
  • Characters: Count the number of characters in your prompt response.

Setting up Response Length

1

Select the Response Length evaluation from the Add evaluator action menu.

2

Link the dataset and provide the evaluation threshold using the comparison operator and measurement unit.

  • Less than: Responses must be shorter than your limit
  • greater than: Responses must be longer than your minimum
  • equal to: Responses must match your exact length requirement
3

Execute the evaluation and see the results.