The JavaScript Evaluator gives you complete control over how your prompts are evaluated. When writing the JS evaluator function, ensure that:

  1. The data is the object provided to this function.
  2. You return the correct object format that this method expects.

Here are some examples,

Checking if the response starts with a certain phrase
return data.completion && data.completion.includes("Analyzing the") 
  ? { grade: 'pass', score: 1, reason: 'Response contains "Analyzing the".' }
  : { grade: 'fail', score: 0, reason: 'Response does not contain "Analyzing the".' };
Checking if the response has certain keywords
const hasKeywords = data.completion && 
  data.completion.includes("medication") && data.completion.includes("follow-up");
return hasKeywords 
  ? { grade: 'pass', score: 1, reason: 'Response contains both summary and conclusion.' }
  : { grade: 'fail', score: 0, reason: 'Missing required keywords.' };
Checking if the response contains harmful words
const hasBadWords = data.completion && 
  (data.completion.includes("error") || data.completion.includes("cannot"));
return !hasBadWords 
  ? { grade: 'pass', score: 1, reason: 'Response avoids error language.' }
  : { grade: 'fail', score: 0, reason: 'Response contains error or limitation words.' };

​
Implementing a JavaScript Evaluator

1

Select the JavaScript evaluator from the “Add evaluator” action menu.

2

Define JavaScript code to evaluate the LLM response against your linked dataset.

3

Run the evaluation and see the results.