Case sources
Where synthetic cases fit
Synthetic cases appear in the Datasets stage of Improve. Adaline uses the prompt variables, evaluator criteria, and recent Behavior evidence to build a small test space around the problem being fixed. That test space is organized into dimensions: user intent, topic, difficulty, request shape, expected output, policy boundary, tool context, or any other axis that helps compare the current prompt against candidates. Adaline then creates cases across a mix of strategies:
When production Behavior data is available, Adaline biases generation toward real patterns: broad and granular Behaviors, newly observed patterns, high-error Behaviors, rare requests, and representative snippets. It may generate more rows than needed first, then prune near-duplicates so the final dataset is more diverse and useful for scoring.
Review generated cases
Before trusting a synthetic case, make sure it is plausible for your product, has clear expected behavior, includes enough context to score fairly, and does not duplicate existing coverage or include private customer details. Good synthetic cases should protect healthy behavior as well as the target failure. If a case feels unrealistic or overfit, do not let it drive approval; keep only the examples that make future releases safer.Preserve useful cases
After the cycle, promote good examples into durable datasets:Auto Generated Evaluators
Understand generated checks created from production evidence.
Build datasets from logs
Turn useful cases into durable test coverage.