Progressive Canary Rollouts with Argo Rollouts

You want a production-ready canary that walks traffic up in weighted steps and aborts itself on a metric regression — this is the complete Argo Rollouts and Istio configuration to do it.

When to use this pattern

  • You run on Kubernetes with Istio (or another supported traffic provider) and a Prometheus-compatible metrics backend.
  • You want automated, metric-gated promotion rather than manual traffic shifting.
  • You need the rollout to abort and revert on its own when the new version regresses.

Prerequisites

Complete working example

# rollout.yaml — Rollout + services + VirtualService + AnalysisTemplate
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata: { name: app }
spec:
  replicas: 10
  selector: { matchLabels: { app: web } }
  template:
    metadata: { labels: { app: web } }
    spec:
      containers:
        - name: app
          image: ghcr.io/acme/app:PLACEHOLDER     # bumped to trigger a rollout
          ports: [{ containerPort: 8080 }]
          readinessProbe: { httpGet: { path: /ready, port: 8080 } }
  strategy:
    canary:
      canaryService: app-canary
      stableService: app-stable
      trafficRouting:
        istio:
          virtualService:
            name: app-vs
            routes: [primary]
      steps:
        - setWeight: 5
        - pause: { duration: 5m }
        - analysis:
            templates: [{ templateName: canary-health }]
            args: [{ name: canary-svc, value: app-canary }]
        - setWeight: 25
        - pause: { duration: 5m }
        - analysis:
            templates: [{ templateName: canary-health }]
            args: [{ name: canary-svc, value: app-canary }]
        - setWeight: 50
        - pause: { duration: 10m }
        - setWeight: 100
---
apiVersion: v1
kind: Service
metadata: { name: app-stable }
spec: { selector: { app: web }, ports: [{ port: 80, targetPort: 8080 }] }
---
apiVersion: v1
kind: Service
metadata: { name: app-canary }
spec: { selector: { app: web }, ports: [{ port: 80, targetPort: 8080 }] }
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata: { name: app-vs }
spec:
  hosts: [app.example.com]
  gateways: [app-gateway]
  http:
    - name: primary                 # Argo rewrites these weights per step
      route:
        - destination: { host: app-stable }
          weight: 100
        - destination: { host: app-canary }
          weight: 0
---
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata: { name: canary-health }
spec:
  args: [{ name: canary-svc }]
  metrics:
    - name: error-rate
      interval: 1m
      count: 5
      failureLimit: 1               # one breach aborts the rollout
      successCondition: result < 0.02
      provider:
        prometheus:
          address: http://prometheus.monitoring:9090
          query: |
            sum(rate(istio_requests_total{destination_service_name="{{args.canary-svc}}",response_code=~"5.."}[1m]))
            / sum(rate(istio_requests_total{destination_service_name="{{args.canary-svc}}"}[1m]))

Step-by-step walkthrough

The Rollout object replaces a standard Deployment. Argo manages two ReplicaSets — stable and canary — and rewrites the Istio VirtualService weights as it advances through steps.

The traffic ladder (setWeight / pause / analysis) is the canary itself: 5% for 5 minutes with analysis, then 25%, then 50% for a longer window, then 100%. Each analysis block runs the canary-health template; a failure aborts the whole rollout.

Stable and canary services give Istio two destinations to weight between. The VirtualService starts at 100/0 and Argo adjusts it — you never edit those weights by hand.

The AnalysisTemplate queries Istio’s istio_requests_total for the canary’s 5xx ratio, sampled five times at one-minute intervals. failureLimit: 1 means a single breaching sample aborts. The full query design, including latency and baseline comparison, is covered in automating canary analysis with Prometheus metrics.

Verification

# Trigger a rollout by bumping the image
kubectl argo rollouts set image app app=ghcr.io/acme/app:$GIT_SHA

# Watch the weighted steps and analysis status live
kubectl argo rollouts get rollout app --watch

Expected output during a healthy rollout shows the weight climbing (5 → 25 → 50 → 100) with each AnalysisRun marked ✔ Successful, ending in Status: ✔ Healthy. A regression shows ✖ Failed analysis and Status: ✖ Degraded with canary weight back at 0.

To drive it manually:

kubectl argo rollouts promote app     # advance past a pause
kubectl argo rollouts abort app       # halt and revert to stable

Common pitfalls

  • Analysis query returns no data. If metrics are not labelled by service/version, istio_requests_total{destination_service_name="app-canary"} is empty and the run fails with “no data points.” Test the query in the Prometheus UI first.
  • First weight too small for the traffic. At 5% of a low-traffic service, five one-minute samples may cover too few requests to detect a regression. Raise the first setWeight or lengthen interval/count so each analysis sees a meaningful request count.
  • No session affinity. Without consistent hashing at the Istio DestinationRule, users flap between versions. Add affinity as shown in the canary release guide.

← Back to Canary Releases & Progressive Rollouts