Secrets Injection for Preview Environments

The operational pain here is that every ephemeral preview environment needs real credentials — a database URL, an API key, a third-party token — but baking those into images or copying production secrets into short-lived namespaces is how leaks and blast-radius incidents happen. This page covers how to deliver secrets to previews using short-lived, OIDC-minted credentials scoped to a preview-only policy, injected at container boot, masked in logs, and expired on a TTL aligned to the preview lifetime. The goal is that a compromised preview credential is useless against production and useless minutes after the PR closes.

Prerequisites

How Short-Lived Secret Injection Works Under the Hood

The insecure default is a long-lived secret stored in CI: it never expires, it is copied into every job’s environment, and if the log redaction ever slips, it leaks permanently. OIDC-based injection replaces that stored secret with an on-demand exchange.

  1. Identity, not secret. The CI runner is issued a signed OIDC token describing who is running (repository, workflow, event, ref). No credential is stored — the token is minted fresh per job and expires in minutes.
  2. Exchange for a scoped credential. The runner presents that token to Vault or the cloud secret manager, which verifies the signature and the bound claims, then mints a short-lived credential limited to the preview policy.
  3. Boot-time injection. The preview container receives secrets at startup — as a mounted file or environment variables — never baked into the image layer. The image itself contains no secrets and is safe to cache and share.
  4. Scope and expiry. The preview credential can read only the preview scope, not production. Its TTL is set to just longer than the preview lifetime, so it dies when the environment is torn down.

This mirrors the discipline in synchronizing environment variables across stages: definitions are version-controlled, values come from a manager at runtime, and nothing sensitive is committed or baked in.

OIDC-Based Secret Injection for Previews The CI runner receives a signed OIDC identity token and presents it to the secret manager. The manager verifies bound claims and mints a short-lived, preview-scoped credential. That credential fetches preview secrets, which are injected into the preview container at boot; production scope is unreachable. CI Runner OIDC id-token present token Secret Manager verify claims → mint scoped cred short-lived Preview Container boot-time inject preview scope only Production scope — denied ✕ no access

Step-by-Step Implementation

Step 1 — Configure OIDC trust and a preview-scoped role

# Vault: trust GitHub's OIDC issuer and bind a role to PR-triggered runs only.
vault write auth/jwt/config \
  oidc_discovery_url="https://token.actions.githubusercontent.com" \
  bound_issuer="https://token.actions.githubusercontent.com"

vault write auth/jwt/role/preview \
  role_type="jwt" \
  user_claim="sub" \
  bound_audiences="https://github.com/acme" \
  bound_claims='{"repository":"acme/app","event_name":"pull_request"}' \
  token_policies="preview-read" \
  token_ttl="2h"

Verification: vault read auth/jwt/role/preview shows token_policies=[preview-read] and the PR-scoped bound claims.

Step 2 — Restrict the preview policy to the preview scope

# preview-read.hcl — readable ONLY under the preview path
path "secret/data/preview/*" {
  capabilities = ["read"]
}
# No production paths. A leaked preview token cannot reach secret/data/production/*.

Verification: With a preview token, vault kv get secret/production/app returns permission denied, while vault kv get secret/preview/app succeeds.

Step 3 — Exchange the OIDC token and inject at boot

# .github/workflows/preview-secrets.yml (excerpt)
permissions:
  id-token: write        # required to mint the OIDC token
  contents: read
steps:
  - name: Fetch preview secrets via OIDC
    uses: hashicorp/vault-action@v3
    with:
      url: ${{ secrets.VAULT_ADDR }}
      method: jwt
      role: preview
      # Values are masked in logs automatically by the action
      secrets: |
        secret/data/preview/app DATABASE_URL | DATABASE_URL ;
        secret/data/preview/app API_KEY      | API_KEY

  - name: Inject into the preview at boot (not into the image)
    run: |
      kubectl -n preview-${PR_NUMBER} create secret generic app-secrets \
        --from-literal=DATABASE_URL="$DATABASE_URL" \
        --from-literal=API_KEY="$API_KEY" \
        --dry-run=client -o yaml | kubectl apply -f -

Verification: kubectl -n preview-${PR} get secret app-secrets exists, and the container image built earlier contains no secret layer (docker history shows no secret ARG).

Step 4 — Confirm masking and TTL expiry

# Masking: grep the job log for the raw value — it must appear only as ***
# TTL: the Vault token dies after 2h, so a leaked cred is useless post-teardown.
vault token lookup -format=json | jq '.data.ttl'   # → seconds remaining, ≤ 7200

Verification: The credential’s remaining TTL is at most the configured 2 hours, and no raw secret value appears in any log line.

How Long a Preview Secret Stays Valid A repository secret copied into the environment lives until someone rotates it, typically months. A per-environment secret lives as long as the environment. A short-lived token minted per deploy expires in minutes and cannot be replayed after the job ends. VALIDITY WINDOW AFTER THE JOB ENDS repository secret months — until someone remembers to rotate it per-environment secret days — as long as the preview exists minted per deploy 15 minutes — useless the moment the job finishes Preview environments run untrusted code, which makes the top row the least appropriate place to use it.

Configuration Reference

Option Type Default Effect
bound_claims object — Restricts which repo/event can assume the role; scope to pull_request for previews
token_ttl duration policy default Credential lifetime; set just above preview lifetime so it expires on teardown
token_policies list — Must grant read to the preview scope only, never production
id-token permission enum none Must be write for the workflow to mint an OIDC token
Injection method enum env/secret Boot-time mount or K8s Secret; never a baked image layer
Log masking boolean on Keep enabled so fetched values render as ***

Integration with Upstream and Downstream Topics

Secrets injection is a sibling of the other preview environment concerns:

Performance and Cost Impact

Activity Long-lived CI secret OIDC short-lived Benefit
Credential lifetime until manually rotated minutes to hours Bounded leak window
Blast radius of a leak whatever the secret grants preview scope only Production stays isolated
OIDC exchange latency 0 1–3 s per job Small, network-bound cost
Rotation effort manual, error-prone automatic on TTL No standing rotation toil
Secret in image layers risk if mishandled never (boot injection) Images safe to cache and share

The cost is a 1–3 second token exchange per job; the benefit is that no long-lived production-capable secret ever sits on a runner or in an image.

Three Routes a Preview Secret Escapes By A secret echoed during a build step appears in logs that are readable by anyone with repository access. A secret given a public bundler prefix is inlined into client JavaScript. A secret included in an error payload is transmitted to a third-party error reporting service. build logs an echo in a debug step, readable by everyone with repo access the client bundle a public prefix inlines it into JavaScript anyone can read error reports included in a payload sent to a third-party service Short lifetimes bound the damage of all three, which is why they matter more than any amount of careful handling.

Troubleshooting

Error: permission denied fetching preview secrets

Cause: The OIDC token’s claims do not match the role’s bound_claims — commonly the workflow runs on an event other than pull_request, or from a fork whose repository claim differs.

Fix: Align bound_claims with the actual event, and decide deliberately whether fork PRs may fetch secrets (usually not). Inspect the token claims in the job log to compare against the role.

Error: id-token request fails with 403

Cause: The workflow lacks permissions: id-token: write, so no OIDC token can be minted.

Fix: Add the permission at the job or workflow level. It is not granted by default.

Symptom: Secret value visible in logs

Cause: A value was echoed or written to a file whose contents were later printed, bypassing the action’s automatic masking.

Fix: Never echo secret values; reference them only as environment variables passed directly to the consuming command. Add a log scanner in CI that greps for known secret prefixes as a backstop.

Symptom: Preview cannot connect to its database

Cause: The preview credential points at production (denied) or the preview database URL was not populated because the fetch step ran after the deploy.

Fix: Ensure the preview scope contains a working preview DATABASE_URL, and order the secret fetch before the deploy step. Pair with database mocking and seeding so previews use isolated data.

Frequently Asked Questions

Why use OIDC instead of storing secrets in CI?

A stored CI secret is long-lived, copied into every job, and leaks permanently if redaction ever fails. OIDC stores no secret: the runner is issued a short-lived, cryptographically signed identity token scoped to the repository and event, and exchanges it for a credential minted on demand. That credential expires in minutes to hours and is bound to a narrow policy, so there is nothing durable to steal and any leak has a small, self-closing window.

Should preview environments use production secrets?

No. Previews must read a separate preview scope with their own database, API keys, and third-party sandbox tokens. Two rules follow: a leaked preview credential must never grant production access (enforced by the scoped policy in Step 2), and a preview must never write to production systems. This isolation is what lets you run hundreds of untrusted per-PR environments safely.

How long should preview credentials live?

Set the TTL just longer than the preview’s expected lifetime — often two to four hours. The credential then expires automatically around the time the environment is torn down or goes idle, so even an undetected leak is useless shortly after. Avoid the temptation of long TTLs “to be safe”: a long TTL is precisely what turns a minor leak into a lasting one.

Choosing Between Injection Points

Secrets can enter a preview environment at three moments, and the choice determines both how long they live and how much of the system has to be trusted.

At build time. The value is baked into the artifact. This is the least flexible option and the most dangerous for a preview, because the artifact is then environment-specific and any secret in it is permanent — it lives as long as the image, in whatever registry holds it, readable by anyone who can pull. The only values that belong here are ones that are not secret at all, such as a public analytics key. If a value must differ between preview and production, that alone is sufficient reason to keep it out of the build.

At deploy time. The pipeline fetches short-lived credentials, renders them into the platform’s own secret store, and the workload reads them from there. This is the pragmatic default for most teams: the secret exists in the environment for the environment’s lifetime, the fetch is auditable, and nothing sensitive touches the artifact. The cost is that the secret outlives the deploy job, so a compromised preview environment yields whatever it holds until teardown.

At run time. The workload authenticates itself to the secret store on startup and holds a lease it renews. Nothing is ever written to a secret store the pipeline controls, and revoking access is immediate rather than waiting for a redeploy. This is the strongest option and the most work: it requires the application to carry a client, handle renewal failure, and have a workload identity of its own — which for a preview environment means that identity has to be created and destroyed alongside it.

Most teams end up mixing the second and third: deploy-time injection for configuration the application reads once at boot, and run-time fetching for anything with a genuine revocation requirement, such as a database credential or a payment provider key.

Scoping rules that hold under pressure

Whichever injection point you choose, three scoping rules do most of the work. First, a preview role reads only preview paths — never a production path, even a read-only one, because a preview environment executing untrusted pull-request code is the least trustworthy compute you operate. Second, the lease is the shortest that completes the job, which for a deploy is usually well under fifteen minutes. Third, the identity is bound to the pull-request context rather than to the repository as a whole, so a workflow on another branch cannot assume it.

The rules are easy to state and easy to erode. The usual erosion path is a debugging session: someone widens the preview role to reach a production path “just to check something”, the check works, and the widening is never reverted because nothing fails afterwards. Reviewing the trust policy on a schedule — quarterly is enough — catches that class of drift, and takes about ten minutes when the policies are small enough to read in one sitting.

Reviewing the Trust Policies on a Schedule

Scoping rules erode quietly. The usual path is a debugging session: someone widens a preview role to reach a production path in order to check something, the check works, and the widening is never reverted because nothing subsequently fails. Six months later the preview role — assumable by untrusted pull-request code — can read production secrets, and nothing in the system will ever point this out.

A quarterly read-through of the policies catches it, and takes about ten minutes when the policies are small enough to read in one sitting. Three questions are enough: can any preview identity reach a production path, is any lease longer than the job that uses it, and does any trust policy accept a subject pattern broader than a single repository and ref.

Keeping the policies small is what keeps the review cheap, which is the practical argument for one role per environment and per privilege level rather than one parameterised role that tries to serve everything.

Finally, test the negative case as part of setup rather than assuming it. Confirm that a preview identity requesting a production path is refused, and that the refusal appears in the audit log. A control nobody has watched fail has not actually been verified, and this one takes a single command to exercise while the context is fresh.

← Back to Preview Environments & Environment Parity