Blue-Green Deployments for Full-Stack Apps

The operational pain blue-green solves is the risky, irreversible production push: you deploy, users hit errors, and rolling back means another slow deploy while the incident burns. Blue-green replaces that with two identical environments and a single atomic switch — the new version is fully validated in isolation before it serves a single real request, and the previous version stays warm so recovery is a routing flip measured in seconds. This page covers the full mechanism for a containerized full-stack app, including the database compatibility rules that trip up most first implementations.

Prerequisites

• Kubernetes cluster ≥ 1.27 with kubectl access, or a platform exposing immutable deploys with alias switching
• A single router — a Kubernetes Service, ingress, or load balancer — that selects backends by a mutable label
• Container registry with images tagged by commit SHA (never latest)
• A /health (liveness) and /ready (readiness) endpoint on the application
• Database migrations authored as backward-compatible, expand-then-contract steps
• Enough cluster headroom to run both environments at full size during the overlap window

How the Cutover Works Under the Hood

Blue-green has exactly one moving part: the selector on the router. Two Deployment objects, app-blue and app-green, are both running, but the router’s Service selector matches only one color label at a time. All production traffic follows that selector.

A deploy is therefore three phases:

1. Deploy to idle. Whichever color the selector is not pointing at is idle. You update its image and wait for readiness. No user traffic touches it because the selector has not moved.
2. Validate in isolation. You address the idle deployment directly — by pod IP, a headless service, or a dedicated app-green.internal service — and run smoke tests. This validates the actual new version, not the old one behind the live router.
3. Flip. A single kubectl patch changes the selector. Kubernetes updates the endpoints, and traffic moves to the new color atomically. The old color keeps running, warm and ready, so reverting is the same patch in reverse.

The subtle part is state. Both colors talk to the same database, cache, and queue. The green version and the blue version are alive simultaneously during validation and during the warm-standby window, so any schema they share must be readable and writable by both. This is why destructive migrations are the number-one cause of blue-green incidents, and why the expand-then-contract discipline is non-negotiable.

Step-by-Step Implementation

Step 1 — Define both environments and the router

Both deployments differ only by their color label. The Service selects one color; that is the switch.

# k8s/blue-green.yaml
apiVersion: apps/v1
kind: Deployment
metadata: { name: app-blue }
spec:
  replicas: 3
  selector: { matchLabels: { app: web, color: blue } }
  template:
    metadata: { labels: { app: web, color: blue } }
    spec:
      containers:
        - name: app
          image: ghcr.io/acme/app:PLACEHOLDER   # patched per deploy
          readinessProbe:
            httpGet: { path: /ready, port: 8080 }
            initialDelaySeconds: 5
---
apiVersion: apps/v1
kind: Deployment
metadata: { name: app-green }
spec:
  replicas: 3
  selector: { matchLabels: { app: web, color: green } }
  template:
    metadata: { labels: { app: web, color: green } }
    spec:
      containers:
        - name: app
          image: ghcr.io/acme/app:PLACEHOLDER
          readinessProbe:
            httpGet: { path: /ready, port: 8080 }
---
apiVersion: v1
kind: Service
metadata: { name: app-router }
spec:
  selector: { app: web, color: blue }   # the live color — this is what the flip changes
  ports: [{ port: 80, targetPort: 8080 }]

Verification: kubectl get service app-router -o jsonpath='{.spec.selector.color}' prints the currently live color.

Step 2 — Deploy to the idle color

LIVE=$(kubectl get service app-router -o jsonpath='{.spec.selector.color}')
IDLE=$([ "$LIVE" = "blue" ] && echo green || echo blue)

kubectl set image deploy/app-"$IDLE" app=ghcr.io/acme/app:"$GIT_SHA"
kubectl rollout status deploy/app-"$IDLE" --timeout=120s

Verification: kubectl rollout status exits 0 only when every replica of the idle color is ready.

Step 3 — Smoke-test the idle environment directly

Expose the idle color through a non-production service (app-green.internal) and test that, not the live router.

# Hit the idle color directly — this validates the NEW version
IDLE_URL="http://app-${IDLE}.internal"
curl -sf "$IDLE_URL/ready" | jq -e '.status == "ok"'
curl -sf "$IDLE_URL/api/version" | jq -e --arg sha "$GIT_SHA" '.sha == $sha'

Verification: The version endpoint returns the SHA you just deployed. If it returns the old SHA, you are testing the wrong environment — check the internal service selector.

Step 4 — Flip the router atomically

kubectl patch service app-router -p "{\"spec\":{\"selector\":{\"color\":\"$IDLE\"}}}"
echo "Traffic now on $IDLE; $LIVE is warm standby"

Verification: curl -sf https://app.example.com/api/version (through the live router) now returns the new SHA. Endpoint updates propagate in under a second.

Step 5 — Hold, then retire

Keep the previous color running for one analysis window. If metrics stay clean, scale it down; if not, flip back.

# Rollback (instant) — only if metrics regress within the window:
# kubectl patch service app-router -p "{\"spec\":{\"selector\":{\"color\":\"$LIVE\"}}}"

# Retire after the window closes cleanly:
sleep 900   # illustrative; in a pipeline this is a scheduled follow-up job
kubectl scale deploy/app-"$LIVE" --replicas=0

Verification: After retirement, kubectl get deploy app-"$LIVE" -o jsonpath='{.status.replicas}' returns 0 while the live color serves all traffic.

A ready-to-run pipeline that wraps these steps is in zero-downtime blue-green deploys with GitHub Actions.

Configuration Reference

Integration with Upstream and Downstream Topics

Blue-green is one of three sibling strategies in the deployment strategies & progressive delivery domain:

• Upstream — build and image tags. The atomic flip depends on immutable, SHA-tagged images from your Docker layer caching build, so the idle color runs an exactly-known artifact.
• Sibling — canary. Where blue-green flips 100% at once, canary releases shift a growing slice. Teams often use blue-green for the frontend and canary for stateful services.
• Downstream — rollback. The warm-standby window is what makes automated rollback instant; the rollback trigger simply patches the selector back.
• Validation — preview. Behaviour is proven per-PR in preview environments long before it reaches a production blue-green cutover.

Performance and Cost Impact

The dominant cost is the doubled-capacity overlap. Shrinking the warm-standby window trades rollback safety for cost — never take it below one analysis window.

Troubleshooting

Error: error: services "app-router" not found during flip

Cause: The router service name in the patch command does not match the deployed Service, or the deploy targeted the wrong namespace.

Fix: Confirm the service exists and namespace with kubectl get svc -A | grep app-router, then include -n <namespace> in the patch. Wire the namespace into the pipeline as a variable rather than hard-coding it.

Symptom: 502 spike at the exact moment of cutover

Cause: The flip happened before the idle color’s pods were ready, so the router forwarded to endpoints that could not yet serve.

Fix: Ensure kubectl rollout status --timeout=120s completes before the patch step, and confirm the readiness probe path returns 200 only when the app can serve traffic (dependencies connected, not just process started).

Symptom: Rollback flip has no effect — errors continue

Cause: The old color was already scaled to zero (a premature cost-optimization step), so flipping back points the router at an environment with no endpoints.

Fix: Never scale down the previous color inside the deploy job. Move retirement to a separate, delayed job that runs only after the analysis window closes cleanly. Guard it with a check that the new color is healthy.

Error: column "old_col" does not exist on the standby color

Cause: A destructive migration dropped a column the still-running previous color reads during the overlap window.

Fix: Revert to expand-then-contract. Reintroduce the column, deploy compatible code to both colors, and only drop it in a later release once no running color references it. See the schema-compatibility discussion in the failure modes section above.

Frequently Asked Questions

Related

• Zero-Downtime Blue-Green Deploys with GitHub Actions — the complete copy-paste workflow that wraps the steps on this page.
• Atomic Symlink Deploys for Static Frontends — blue-green semantics for static sites using a symlink swap.
• Blue-Green vs Canary vs Rolling Deployment Strategies — a decision guide for choosing among the three.
• Canary Releases & Progressive Rollouts — the sibling strategy for gradual, metric-gated exposure.

← Back to Deployment Strategies & Progressive Delivery