AI Production Release Lifecycle

Key jargon

Term	Plain-language meaning
Release candidate	A version proposed for deployment after testing.
Canary	A limited deployment used to observe behavior before broad release.
Rollback	Restoring a previously known-good version.
Feature flag	A runtime control that enables or disables behavior without rebuilding the application.

Key concepts

Version the complete AI system: model, prompt, retrieval index, tools, policies, and code.
Promote through offline gates, limited exposure, monitored rollout, and an exercised rollback path.

Concept map

flowchart LR
    A["Build versioned candidate"] --> B["Run offline evaluation gates"]
    B --> C["Canary and monitor"]
    C --> D["Promote or rollback"]

Release unit

Version model, prompt/instructions, tool definitions, retrieval corpus/index, code, policy, evaluation set, and deployment configuration. “Same model” does not mean same system.

Gates

Offline representative and safety evals.
Dependency/image provenance and security review.
Load, failure, and recovery tests.
Shadow or internal traffic.
Limited canary with kill switch.
Metric and human-review comparison.
Progressive rollout or rollback.
Post-release review and dated documentation.

Exercise

Write a release checklist for changing only the system prompt. Identify which regressions could still occur and why “no code change” is not “no risk.”

Ownership

Name product, model, data, security, privacy, operations, and incident owners. An AI system without an accountable owner is not production ready.