Auto Prompt Optimization

Auto Prompt Optimization is the part of Improve that proposes prompt changes. Adaline generates candidate prompt snapshots, scores them against available evidence, rejects unsafe or regressing candidates, and packages the selected candidate for review.

​

What can change

For tool-using and coding agents, optimization may also affect routing rules, verification policy, tool descriptions, and few-shot examples. Always review the full diff.

​

Candidate exploration

The Prompts stage summarizes the search. More variants are not automatically better. A strong run finds a narrow candidate that improves the target issue and preserves healthy behavior.

​

Read the diff first

The diff is the source of truth for what will change. Use the diagnosis and scores to understand why the change exists, but use the diff to decide whether the change is acceptable. Reject or edit candidates that try to solve retrieval, provider, data, or backend failures through prompt text.

​

Check regression evidence

The regression report compares baseline and candidate behavior across authored evaluators, auto generated evaluators, and validation cases. Watch for:

• Protected evaluators moving down.
• Blank baseline or aggregate cells, which usually mean weak comparable scoring coverage.
• New generated checks that need review before becoming hard gates.
• Healthy dataset rows failing after the candidate improves the target issue.

An evaluator drop is not automatically fatal, but it needs a named owner and a reason.

​

Inspect traffic comparison

Traffic comparison answers the question metrics cannot fully answer: would you want users to receive this new output? Use it to check tone, format, tool behavior, verbosity, structured output shape, and whether the original failure is actually fixed.

​

Runtime tradeoffs

Prompt improvements can increase runtime cost. Before review, check: For high-volume prompts, cost or latency increases should be explicit release tradeoffs, not surprises.