Production readiness

Use this checklist before a project sends real production traffic to Adaline or reads deployed prompt versions from Adaline. Production readiness is a cross-functional job: admins own workspace controls, prompt owners own quality gates, application owners own integration, and release owners own deployment handoff. For the operating cadence after launch, see Operate your AI agent. For prompt-release review, see Release prompts safely.

​

Workspace setup

Confirm workspace-level configuration:

• Workspace name and ownership are correct.
• Admins and members have the right roles.
• Former teammates and unused guest accounts are removed.
• Provider credentials are configured in workspace settings.
• API keys are named after the service or automation that uses them.
• Billing and usage limits are understood.
• Security controls match your company policy.

​

Provider setup

Before launch:

• Configure provider credentials in Workspace settings.
• Confirm the intended models are visible in the prompt editor.
• Run playground tests for each provider/model path.
• Confirm provider errors are handled gracefully by your application.
• Keep provider secrets out of prompts, traces, datasets, screenshots, and docs.

If a prompt cannot run because there is no provider key, fix provider setup before changing prompt content.

​

API keys

For each runtime service:

• Create a separate API key.
• Store it in a secret manager.
• Do not expose it in client-side code.
• Name it after the service, environment, and owner.
• Rotate it when ownership changes or exposure is suspected.
• Delete keys that are no longer used.
• Review usage after launch.

See Create API keys and View API usage.

​

Project release setup

Confirm each production project has:

• Deployment environments that match your release lanes.
• At least one production-ready prompt deployment.
• Prompts linked to the tools they need.
• Evaluators for quality, safety, cost, latency, and format requirements.
• Datasets for golden and regression cases.
• Webhooks or cache refresh logic if your runtime caches deployments.
• Trace instrumentation with environment, release, route, and meaningful span names.

​

Prompt quality gates

Before deploying a prompt:

• Run representative playground cases.
• Run the golden dataset.
• Run regression datasets created from prior issues.
• Inspect evaluator failures.
• Compare the deployment snapshot with the current environment.
• Confirm cost, token, and latency budgets.
• Decide whether approval should deploy immediately or route through Edit & approve.
• Record the rollback target before changing production.

​

Observability launch checklist

Before traffic ramps:

• Traces are arriving in the project.
• Monitor shows logs, latency, cost, tokens, and eval score where configured.
• Trace filters work for environment, route, release, and key segments.
• Deep Search can find known semantic examples.
• Behavior analysis has enough traffic to produce useful clusters.
• Operators know where to look during incidents.

​

Deployment handoff

Your application should know:

• Which workspace, project, prompt, and environment to read.
• How long to cache deployment configuration.
• How to refresh after a deployment event.
• How to roll back if production behavior regresses.
• Which trace metadata identifies the deployed release.

When webhooks are configured, store webhook signing secrets securely and rotate them when required.

​

After launch

Within the first release window:

1. Watch Monitor for traffic, latency, cost, tokens, and eval score.
2. Open Traces for representative production requests.
3. Review Behaviors once clustering has enough evidence.
4. Add failed production cases to Datasets.
5. Start Improve only when evidence points to a prompt-level fix.