Skip to main content
The Response Length evaluator counts the size of your prompt responses and checks whether they fall within acceptable bounds. Use it to enforce brevity constraints, minimum detail requirements, or consistent response sizes across test cases.

​
Set up the Response Length evaluator

1

Open Evaluate from your prompt

Open the prompt you want to test and click Evaluate in the prompt header.Opening the Evaluate tab from a prompt
2

Select the evaluator

Add the Response Length evaluator from the evaluator menu.Adding the Response Length evaluator
3

Configure the threshold

Give the evaluator a name, link a dataset, and set the length threshold.Setting the evaluation thresholdChoose from the following threshold operators:
OperatorBehavior
less thanResponses must be shorter than your limit. Use this to enforce brevity.
greater thanResponses must be longer than your minimum. Use this to ensure sufficient detail.
equal toResponses must match an exact length requirement.
4

Select the unit of measure

Choose how length is measured.Selecting the unit of measure
UnitDescription
TokensExact token count as returned by the LLM provider. Most accurate for cost and context limit management.
WordsNumber of words (sequences of characters separated by whitespace). Easier to reason about for content length.
CharactersTotal character count. Best for strict UI or display constraints.
5

Run the evaluation

Click Evaluate to execute the evaluation and see the results.Response Length evaluator results

​
When to use

  • UI constraints — Ensure responses fit within character limits for display components (chat bubbles, cards, notifications).
  • Content consistency — Enforce a consistent level of detail across all responses.
  • Brevity enforcement — Prevent the model from generating overly verbose outputs.
  • Minimum detail requirements — Ensure responses contain enough information to be useful.
  • Token budget management — Work alongside the Cost evaluator to keep token usage in check.

​
Next steps

Cost Evaluator

Track costs alongside response length.

Latency Evaluator

Measure response time alongside length.