Cleaning Up Stale Feature Flags Automatically

Every flag was going to be temporary. Two years later there are 140 of them, nobody knows which are live, and a conditional four levels deep guards code that has been fully rolled out since last spring. Flag debt is not a discipline problem — it is a missing automation, because removal is the one step no feature’s definition of done includes.

When to use this pattern

  • Your flag count only ever goes up.
  • Engineers hesitate to delete a flag because they cannot tell whether anything still depends on it.
  • Conditionals are nesting, and the dead branch of an old flag is still being maintained.

Prerequisites

The three states a flag can be in

Cleanup automation works because “stale” is mechanically detectable, not a judgement call. A flag is either doing work, done, or forgotten.

Flag States and the Action Each One Warrants A flag serving a partial rollout is active and left alone. A flag serving one value to all traffic for longer than the rollback window is done, and a removal pull request is opened. A flag past its declared expiry with no traffic change is forgotten, and its owner is notified before a removal pull request follows. ACTIVE serving 25% / 75% changed this week leave alone DONE 100% one value, 21 days past the rollback window open a removal pull request FORGOTTEN past its declared expiry unchanged for 8 months notify the owner, then remove the state most flags end in Only the first state needs human judgement. The other two are detectable from the flag service's own data. A kill switch is a deliberate permanent flag — tag it so the detector never proposes removing it.

Complete working example

// src/flags.ts — flags are declared with metadata, not invented at call sites.
export const FLAGS = {
  'checkout-express-lane': {
    owner: 'team-payments',
    expires: '2026-09-30',        // required; CI rejects a flag without one
    kind: 'release' as const,     // 'release' flags are removable; 'kill-switch' are not
  },
  'emergency-disable-payments': {
    owner: 'team-payments',
    expires: null,
    kind: 'kill-switch' as const, // permanent by design — never proposed for removal
  },
} satisfies Record<string, FlagMeta>;
// scripts/flag-audit.mjs — detect stale flags and open removal pull requests.
import { execSync } from 'node:child_process';
import { FLAGS } from '../src/flags.ts';

const ROLLBACK_WINDOW_DAYS = 14;
const today = new Date(process.env.AUDIT_DATE);         // injected, so runs are reproducible

const remote = await fetch('https://flags.example.com/api/v1/flags', {
  headers: { authorization: `Bearer ${process.env.FLAG_API_TOKEN}` },
}).then((r) => r.json());

const days = (iso) => Math.floor((today - new Date(iso)) / 86_400_000);
const stale = [];

for (const [key, meta] of Object.entries(FLAGS)) {
  if (meta.kind === 'kill-switch') continue;            // permanent by design
  const live = remote.find((f) => f.key === key);
  if (!live) { stale.push({ key, reason: 'declared in code but absent from the service' }); continue; }

  // DONE: one variation has had all traffic for longer than the rollback window.
  const settled = live.rollout.some((r) => r.weight === 100);
  if (settled && days(live.lastModified) > ROLLBACK_WINDOW_DAYS) {
    stale.push({ key, reason: `fully rolled out for ${days(live.lastModified)} days`, owner: meta.owner });
    continue;
  }
  // FORGOTTEN: past its own declared expiry, whatever it is doing.
  if (meta.expires && today > new Date(meta.expires)) {
    stale.push({ key, reason: `expired on ${meta.expires}`, owner: meta.owner });
  }
}

for (const f of stale) {
  const branch = `chore/remove-flag-${f.key}`;
  execSync(`git switch -c ${branch} origin/main`);
  // The transform inlines the winning branch and deletes the conditional.
  execSync(`npx flag-remove --flag ${f.key} --keep on`);
  execSync(`git commit -am "chore: remove flag ${f.key} (${f.reason})" || true`);
  execSync(`git push -u origin ${branch} || true`);
  execSync(`gh pr create --title "Remove flag: ${f.key}" --body \
    "Detected stale: ${f.reason}\\n\\nOwner: @${f.owner}\\n\\nThe transform kept the ON branch. \
Please confirm the removed branch is genuinely dead before merging." || true`);
}

console.log(`${stale.length} stale flag(s)`);
# .github/workflows/flag-hygiene.yml
name: Flag hygiene
on:
  schedule: [{ cron: '0 6 * * 1' }]     # weekly, so the noise is predictable
  pull_request:                          # plus a per-PR guard, below

permissions: { contents: write, pull-requests: write }

jobs:
  guard:
    if: github.event_name == 'pull_request'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Every new release flag declares owner and expiry
        run: |
          # A flag added in this diff without an expires field fails the check.
          git diff -U0 origin/${{ github.base_ref }}... -- src/flags.ts \
            | grep '^+' | grep -q "kind: 'release'" || exit 0
          npx tsx scripts/assert-flag-metadata.ts

  audit:
    if: github.event_name == 'schedule'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci --ignore-scripts
      - run: node scripts/flag-audit.mjs
        env:
          AUDIT_DATE: ${{ github.event.repository.updated_at }}
          FLAG_API_TOKEN: ${{ secrets.FLAG_API_TOKEN }}

Step-by-step walkthrough

Require an expiry at creation. This is the only step that stops the debt at source. A flag without a declared removal date is a permanent flag that nobody has admitted to; the CI guard turns that decision into an explicit one made in review.

Distinguish release flags from kill switches. A kill switch is a legitimate permanent flag and proposing its removal weekly trains people to ignore the bot. One metadata field removes an entire category of false positives.

Measure staleness from the flag service, not the codebase. The code cannot tell you a flag has served 100% for three weeks. The service can, and that is the only reliable signal that removal is safe — which is why the audit needs API access rather than just a grep.

Open a pull request; do not merge it. The code transform inlines one branch and deletes the other, which is a semantic change. Automated transforms handle simple conditionals well and occasionally mangle a nested ternary or a flag used in a config object. A human confirming that the removed branch is genuinely dead costs two minutes and prevents the class of incident where cleanup deletes live behaviour.

Run weekly, not daily. Cleanup is not urgent, and a bot that opens pull requests every morning gets muted. A predictable Monday batch is read; a continuous trickle is not.

What the removal transform actually does

The transform is the part reviewers should understand before merging its output, because “delete the flag” means four different edits depending on how the flag was used.

Four Shapes a Flag Removal Has to Handle A simple conditional becomes the kept branch inlined. A ternary in markup becomes the kept expression. A configuration object key is deleted along with its value. A test that toggles the flag must have the toggling removed and the assertion kept, which is the case automated transforms most often get wrong. USAGE SHAPE TRANSFORM RISK if (flag('x')) { A } else { B } inline A, delete B low {flag('x') ? <New/> : <Old/>} keep <New/>, drop the expression low config = { ...(flag('x') && extra) } spread the object unconditionally medium test: setFlag('x', false); expect(...) the whole test may be obsolete high The fourth row is why the bot opens a pull request rather than merging: a test asserting the removed branch is now testing nothing.

The first two shapes are mechanical and safe. The third requires the reviewer to confirm that the spread’s contents are genuinely wanted unconditionally rather than only under the flag. The fourth is the one that produces silently weakened test suites: a test that set the flag to its losing value and asserted the old behaviour is, after removal, either meaningless or actively misleading, and the transform cannot tell which. Reviewing the test changes in a flag-removal pull request is worth more attention than reviewing the source changes.

A related detail worth deciding once: which branch the transform keeps. Passing --keep on assumes the flag reached 100% on its enabled value, which is true for a successful rollout and false for one that was disabled after a problem and then forgotten. Reading the winning variation from the flag service rather than assuming it removes a whole class of wrong removals, and costs one extra field in the audit output.

Verification

# 1. The audit is honest — run it read-only and inspect the classification.
DRY_RUN=1 AUDIT_DATE="$(date -Iseconds)" node scripts/flag-audit.mjs

Expected:

checkout-express-lane   fully rolled out for 34 days     → removal PR
legacy-search-ranking   expired on 2026-05-01            → removal PR
emergency-disable-payments  kill-switch                  → skipped
3 flag(s) evaluated, 2 stale
# 2. No flag in code is missing from the service, or vice versa.
node -e "import('./src/flags.ts').then(m => console.log(Object.keys(m.FLAGS).sort().join('\n')))" > /tmp/code.txt
curl -s -H "authorization: Bearer $FLAG_API_TOKEN" https://flags.example.com/api/v1/flags \
  | jq -r '.[].key' | sort > /tmp/service.txt
diff /tmp/code.txt /tmp/service.txt || echo "drift — investigate before trusting the audit"

# 3. The guard actually blocks a flag added without an expiry.
git switch -c probe && sed -i "s/} satisfies/  'probe-flag': { owner: 'x', kind: 'release' as const },\n} satisfies/" src/flags.ts
npx tsx scripts/assert-flag-metadata.ts; echo "exit=$?"      # → exit=1

Drift in the second check matters more than it looks: a flag present in the service but not in code is evaluated by nothing, and a flag in code but not in the service falls back to its default on every request — silently, and often not the default anyone intended.

What the debt actually costs

The argument for spending engineering time on this is easier to make with the second-order costs named, because the first-order cost — a few unused conditionals — sounds trivial.

Each live flag doubles the notional paths through the code it guards, and flags compose: three overlapping flags in one checkout flow describe eight possible behaviours, of which the test suite probably exercises two. That is where the real cost sits — not in the conditional, but in the fact that nobody can any longer say what the code does without enumerating flag states. It also degrades incident response directly: the first question during an incident is “what changed”, and a flag toggled by someone in another team three weeks ago is a change with no commit, no review, and no entry in the deploy history that automated rollback consults.

A useful target is that no release flag lives longer than one quarter, and that the total count stays roughly flat over a year. Both are trivially measurable from the audit output, and publishing them alongside the weekly batch is what keeps the cleanup from quietly stopping after the first enthusiastic month.

Overlapping Flags Multiply Untested Behaviour With one flag in a code path there are two possible behaviours and both are typically tested. With two flags there are four and three are tested. With three flags there are eight and three are tested. With four there are sixteen and still only three are tested, leaving thirteen combinations that have never run. POSSIBLE BEHAVIOURS (bar) vs COMBINATIONS ACTUALLY TESTED (marked) 1 flag 2 behaviours, 2 tested 2 flags 4 behaviours, 3 tested 3 flags 8 behaviours, 3 tested 4 flags 16 vs 3 The shaded remainder is behaviour that exists in production and has never been executed anywhere else.

Common pitfalls

Removing a flag still inside the rollback window. The flag was the rollback mechanism; deleting it converts a toggle into a deploy. Wait out the window, which is why the detector compares against it explicitly.

Deleting the flag from the service before the code. The code then falls back to its default, which is frequently the losing branch. Remove from code first, deploy, then delete the flag.

Letting the bot merge. Automated conditional inlining is right most of the time and wrong in ways that are hard to spot in a large diff. Keep a human on the merge.

← Back to Feature-Flag-Driven Release Management