Feature-Flag-Driven Release Management

The operational pain here is that deployment and release are usually the same event: the moment you deploy, every user is exposed, so a risky feature and a routine bug fix carry the same all-or-nothing risk. Feature flags break that coupling — code ships to production dark, and a runtime toggle decides who sees it, when, and for how long. This page covers how to run releases as independent, reversible flag operations: cohort ramps, instant kill switches, safe evaluation under flag-service failure, and the lifecycle discipline that keeps flags from rotting into production liabilities.

Prerequisites

How Flag Evaluation Works Under the Hood

A feature flag is a runtime decision function: given an evaluation context (user ID, plan tier, region), the flag service returns a variant. The application branches on that variant. Because the branch lives in code that is already deployed to everyone, changing the release is just changing the flag rule — no pipeline runs, no traffic shift.

Four properties make this production-safe:

  1. Ship dark. New code merges and deploys behind a flag defaulted to off in production. The deploy is invisible; the deployment already succeeded before anyone decides to release.
  2. Ramp by cohort. The flag rule grows the exposed set — 1% of users, then internal staff, then a region, then everyone — while metrics for each cohort are compared to the control group.
  3. Kill switch. Flipping the flag off is a sub-second, infrastructure-free rollback. It is faster than any blue-green flip or canary abort because nothing redeploys.
  4. Resilient evaluation. The SDK caches the last-known ruleset locally. If the flag service is unreachable, evaluation falls back to the cached value or a hard-coded default — always the stable path — so a flag outage degrades to safe, not broken.
Deploy-Release Decoupling with Feature Flags A single deployed binary holds both a new path and a stable path. A request carries an evaluation context to a flag evaluator, which consults a local cache backed by the flag service and returns a variant; enabled users take the new path, everyone else takes the stable path. Request Flag Evaluator context → variant Flag Service local cache fallback New Path (enabled) ramped cohort Stable Path (default) everyone else · fallback one deployed binary — release ≠ deploy

Step-by-Step Implementation

Step 1 — Wrap the new path in a flag

Use OpenFeature so the code is provider-agnostic. The default must be the stable path.

// checkout.ts
import { OpenFeature } from "@openfeature/server-sdk";

const client = OpenFeature.getClient();

export async function handleCheckout(req: Request, user: User) {
  // Default `false` → stable path. Shipping this line does NOT release the feature.
  const useNewFlow = await client.getBooleanValue("new-checkout-flow", false, {
    targetingKey: user.id,
    plan: user.plan,
    region: user.region,
  });

  return useNewFlow ? newCheckout(req, user) : legacyCheckout(req, user);
}

Verification: Deploy with the flag off and confirm 100% of traffic still takes legacyCheckout — the deploy released nothing.

Step 2 — Ramp by cohort

Grow exposure in the flag service, not in code. A typical ladder: internal users → 1% → 10% → 50% → 100%.

// flag rule (managed in the flag service UI or as config)
{
  "key": "new-checkout-flow",
  "defaultVariant": "off",
  "rules": [
    { "if": { "attribute": "email", "endsWith": "@acme.com" }, "serve": "on" },
    { "rollout": { "on": 10, "off": 90 }, "bucketBy": "targetingKey" }
  ]
}

Verification: Query metrics split by flag variant; the on cohort’s conversion and error rate should track the off control within tolerance before you widen the rollout.

Step 3 — Guarantee a safe fallback

Configure the SDK so a flag-service outage returns cached or default values, never an error.

OpenFeature.setProvider(
  new FlagProvider({
    cacheTtlSeconds: 60,        // serve cached rules for up to 60s
    staleWhileRevalidate: true, // keep serving stale rules if refresh fails
    onError: () => "off",       // last resort: stable path
  })
);

Verification: Block the flag service at the network level in staging and confirm requests still succeed on the stable path with no 5xx.

Step 4 — Enforce flag lifecycle in CI

Record each flag’s TTL and fail the build when one overstays.

#!/usr/bin/env bash
# scripts/flag-ttl-check.sh — run in CI
set -euo pipefail
NOW=$(date +%s)
fail=0
# flags.json: [{ "key": "...", "created": <epoch>, "ttlDays": 30 }]
jq -c '.[]' flags.json | while read -r flag; do
  key=$(jq -r '.key' <<<"$flag")
  created=$(jq -r '.created' <<<"$flag")
  ttl=$(jq -r '.ttlDays' <<<"$flag")
  age=$(( (NOW - created) / 86400 ))
  if (( age > ttl )); then
    echo "FLAG DEBT: '$key' is ${age}d old (TTL ${ttl}d) — remove it or extend TTL with justification"
    fail=1
  fi
done
exit $fail

Verification: Add a flag with a past-due created timestamp and confirm CI fails with the FLAG DEBT message.

Configuration Reference

Option Type Default Effect
Default variant enum off The value served with no matching rule; must be the stable path in production
bucketBy string targetingKey Attribute the percentage rollout hashes on; keep stable so a user’s bucket does not change
cacheTtlSeconds integer 30–60 How long cached rules are served before refresh; higher survives longer outages
staleWhileRevalidate boolean true Serve stale rules when refresh fails instead of erroring
ttlDays (flag registry) integer 30 Age after which CI flags the toggle as debt
Evaluation location enum server Server-side for backend logic; client-side for UI — never gate security on client flags

Integration with Upstream and Downstream Topics

Feature flags layer on top of the infrastructure deployment strategies:

  • Complementary — canary. A flag ramp can run inside a canary rollout: the binary is canaried per-request while a feature is flag-ramped per-user, two independent control loops.
  • Complementary — blue-green. After a blue-green cutover, flags let you release features from the freshly live version gradually rather than all at once.
  • Downstream — rollback. A flag kill switch must be part of the rollback runbook; an infra rollback that leaves a flag on can re-expose a broken feature.
  • Upstream — config parity. Flag defaults and rulesets are configuration; keep them consistent across stages using the same discipline as synchronizing environment variables across stages.

Performance and Cost Impact

Metric Value Notes
Flag evaluation latency < 1 ms Local cache hit; no network call on the hot path
Kill-switch propagation 1–60 s Bounded by cacheTtlSeconds; lower for faster kills, higher for outage resilience
Rollback speed (flag off) sub-second at origin Faster than any infrastructure rollback
Flag-service outage impact 0 (safe fallback) Requests continue on the stable path
Cost of flag debt compounding Each stale flag multiplies untested branch combinations and test matrix size

The real cost of flags is not runtime — it is debt. A codebase with dozens of long-lived flags has an exponential number of untested path combinations. The lifecycle check in Step 4 is the single most important control.

Troubleshooting

Symptom: Feature appears for users outside the target cohort

Cause: The percentage rollout buckets on an unstable attribute (session ID that rotates, or request timestamp), so a user’s bucket changes between requests.

Fix: Bucket on a stable targetingKey such as a persistent user ID. Confirm the same user consistently resolves to the same variant across sessions.

Symptom: Requests error when the flag service is down

Cause: The SDK is configured to throw on evaluation failure instead of returning a default.

Fix: Set an onError default of the stable variant and enable staleWhileRevalidate (Step 3). Flag evaluation must be non-throwing on the request path.

Symptom: A rolled-back deploy still shows the broken feature

Cause: Infrastructure rolled back but the flag stayed enabled, so the reverted binary still exposed the flagged path.

Fix: Add the flag kill switch to the automated rollback action so flipping the flag off happens first, before or alongside the traffic shift. Coordinate this in the rollback runbook.

Symptom: CI green but production has 40 stale flags

Cause: No lifecycle enforcement — flags are created but never removed, and nothing fails the build.

Fix: Adopt the flag-ttl-check.sh gate (Step 4) as a required status check. Treat flag removal as the definition-of-done for a rollout, not optional cleanup.

Frequently Asked Questions

What is the difference between a feature flag and a canary release?

A canary shifts infrastructure traffic between two binaries at the request level; a feature flag toggles a code path inside a single binary at the feature level per user. Canary answers “is this build healthy?”; flags answer “who should see this feature?”. They compose cleanly — you can run a per-user flag ramp inside a per-request canaried binary, giving you two independent, reversible control loops.

How do I prevent feature flag debt?

Give every temporary flag a TTL and a named owner in a registry, and fail CI when a flag exceeds its TTL without justification. Treat flag removal — deleting the toggle and the now-dead branch — as the final, required step of a rollout, not optional cleanup. Long-lived flags multiply untested path combinations; a dozen boolean flags is 4096 theoretical states, most never exercised, and that is where latent incidents hide.

What happens if the flag service is unavailable?

Nothing user-visible, if configured correctly. The SDK serves the last-known ruleset from a local cache (staleWhileRevalidate) and, if it has no cached value, returns a hard-coded default that is always the stable path. Evaluation must never throw on the request path and must never default a half-finished feature to on. Test this explicitly by blocking the flag service in staging.

Do feature flags replace the need for schema migrations to be backward compatible?

No — they raise the stakes. Because a flag keeps old and new code paths alive at the same time for the flag’s entire lifetime, which may be days or weeks, the database schema must remain compatible with both paths that whole time. Use expand-then-contract migrations and do not drop or rename anything the stable path still reads until the flag is fully rolled out and removed.

← Back to Deployment Strategies & Progressive Delivery