Secrets Injection for Preview Environments
The operational pain here is that every ephemeral preview environment needs real credentials — a database URL, an API key, a third-party token — but baking those into images or copying production secrets into short-lived namespaces is how leaks and blast-radius incidents happen. This page covers how to deliver secrets to previews using short-lived, OIDC-minted credentials scoped to a preview-only policy, injected at container boot, masked in logs, and expired on a TTL aligned to the preview lifetime. The goal is that a compromised preview credential is useless against production and useless minutes after the PR closes.
Prerequisites
How Short-Lived Secret Injection Works Under the Hood
The insecure default is a long-lived secret stored in CI: it never expires, it is copied into every job’s environment, and if the log redaction ever slips, it leaks permanently. OIDC-based injection replaces that stored secret with an on-demand exchange.
- Identity, not secret. The CI runner is issued a signed OIDC token describing who is running (repository, workflow, event, ref). No credential is stored — the token is minted fresh per job and expires in minutes.
- Exchange for a scoped credential. The runner presents that token to Vault or the cloud secret manager, which verifies the signature and the bound claims, then mints a short-lived credential limited to the preview policy.
- Boot-time injection. The preview container receives secrets at startup — as a mounted file or environment variables — never baked into the image layer. The image itself contains no secrets and is safe to cache and share.
- Scope and expiry. The preview credential can read only the preview scope, not production. Its TTL is set to just longer than the preview lifetime, so it dies when the environment is torn down.
This mirrors the discipline in synchronizing environment variables across stages: definitions are version-controlled, values come from a manager at runtime, and nothing sensitive is committed or baked in.
Step-by-Step Implementation
Step 1 — Configure OIDC trust and a preview-scoped role
# Vault: trust GitHub's OIDC issuer and bind a role to PR-triggered runs only.
vault write auth/jwt/config \
oidc_discovery_url="https://token.actions.githubusercontent.com" \
bound_issuer="https://token.actions.githubusercontent.com"
vault write auth/jwt/role/preview \
role_type="jwt" \
user_claim="sub" \
bound_audiences="https://github.com/acme" \
bound_claims='{"repository":"acme/app","event_name":"pull_request"}' \
token_policies="preview-read" \
token_ttl="2h"Verification: vault read auth/jwt/role/preview shows token_policies=[preview-read] and the PR-scoped bound claims.
Step 2 — Restrict the preview policy to the preview scope
# preview-read.hcl — readable ONLY under the preview path
path "secret/data/preview/*" {
capabilities = ["read"]
}
# No production paths. A leaked preview token cannot reach secret/data/production/*.Verification: With a preview token, vault kv get secret/production/app returns permission denied, while vault kv get secret/preview/app succeeds.
Step 3 — Exchange the OIDC token and inject at boot
# .github/workflows/preview-secrets.yml (excerpt)
permissions:
id-token: write # required to mint the OIDC token
contents: read
steps:
- name: Fetch preview secrets via OIDC
uses: hashicorp/vault-action@v3
with:
url: ${{ secrets.VAULT_ADDR }}
method: jwt
role: preview
# Values are masked in logs automatically by the action
secrets: |
secret/data/preview/app DATABASE_URL | DATABASE_URL ;
secret/data/preview/app API_KEY | API_KEY
- name: Inject into the preview at boot (not into the image)
run: |
kubectl -n preview-${PR_NUMBER} create secret generic app-secrets \
--from-literal=DATABASE_URL="$DATABASE_URL" \
--from-literal=API_KEY="$API_KEY" \
--dry-run=client -o yaml | kubectl apply -f -Verification: kubectl -n preview-${PR} get secret app-secrets exists, and the container image built earlier contains no secret layer (docker history shows no secret ARG).
Step 4 — Confirm masking and TTL expiry
# Masking: grep the job log for the raw value — it must appear only as ***
# TTL: the Vault token dies after 2h, so a leaked cred is useless post-teardown.
vault token lookup -format=json | jq '.data.ttl' # → seconds remaining, ≤ 7200Verification: The credential’s remaining TTL is at most the configured 2 hours, and no raw secret value appears in any log line.
Configuration Reference
| Option | Type | Default | Effect |
|---|---|---|---|
bound_claims |
object | — | Restricts which repo/event can assume the role; scope to pull_request for previews |
token_ttl |
duration | policy default | Credential lifetime; set just above preview lifetime so it expires on teardown |
token_policies |
list | — | Must grant read to the preview scope only, never production |
id-token permission |
enum | none | Must be write for the workflow to mint an OIDC token |
| Injection method | enum | env/secret | Boot-time mount or K8s Secret; never a baked image layer |
| Log masking | boolean | on | Keep enabled so fetched values render as *** |
Integration with Upstream and Downstream Topics
Secrets injection is a sibling of the other preview environment concerns:
- Upstream — provisioning. Automated preview deployments create the namespace this secret is written into.
- Sibling — variables. Synchronizing environment variables across stages governs the non-secret config that accompanies these credentials.
- Sibling — routing. Ephemeral preview URL management exposes the app that consumes these secrets.
- Downstream — parity. Environment parity validation gates verify the injected secret keys match production even though the values differ.
Performance and Cost Impact
| Activity | Long-lived CI secret | OIDC short-lived | Benefit |
|---|---|---|---|
| Credential lifetime | until manually rotated | minutes to hours | Bounded leak window |
| Blast radius of a leak | whatever the secret grants | preview scope only | Production stays isolated |
| OIDC exchange latency | 0 | 1–3 s per job | Small, network-bound cost |
| Rotation effort | manual, error-prone | automatic on TTL | No standing rotation toil |
| Secret in image layers | risk if mishandled | never (boot injection) | Images safe to cache and share |
The cost is a 1–3 second token exchange per job; the benefit is that no long-lived production-capable secret ever sits on a runner or in an image.
Troubleshooting
Error: permission denied fetching preview secrets
Cause: The OIDC token’s claims do not match the role’s bound_claims — commonly the workflow runs on an event other than pull_request, or from a fork whose repository claim differs.
Fix: Align bound_claims with the actual event, and decide deliberately whether fork PRs may fetch secrets (usually not). Inspect the token claims in the job log to compare against the role.
Error: id-token request fails with 403
Cause: The workflow lacks permissions: id-token: write, so no OIDC token can be minted.
Fix: Add the permission at the job or workflow level. It is not granted by default.
Symptom: Secret value visible in logs
Cause: A value was echoed or written to a file whose contents were later printed, bypassing the action’s automatic masking.
Fix: Never echo secret values; reference them only as environment variables passed directly to the consuming command. Add a log scanner in CI that greps for known secret prefixes as a backstop.
Symptom: Preview cannot connect to its database
Cause: The preview credential points at production (denied) or the preview database URL was not populated because the fetch step ran after the deploy.
Fix: Ensure the preview scope contains a working preview DATABASE_URL, and order the secret fetch before the deploy step. Pair with database mocking and seeding so previews use isolated data.
Frequently Asked Questions
Why use OIDC instead of storing secrets in CI?
A stored CI secret is long-lived, copied into every job, and leaks permanently if redaction ever fails. OIDC stores no secret: the runner is issued a short-lived, cryptographically signed identity token scoped to the repository and event, and exchanges it for a credential minted on demand. That credential expires in minutes to hours and is bound to a narrow policy, so there is nothing durable to steal and any leak has a small, self-closing window.
Should preview environments use production secrets?
No. Previews must read a separate preview scope with their own database, API keys, and third-party sandbox tokens. Two rules follow: a leaked preview credential must never grant production access (enforced by the scoped policy in Step 2), and a preview must never write to production systems. This isolation is what lets you run hundreds of untrusted per-PR environments safely.
How long should preview credentials live?
Set the TTL just longer than the preview’s expected lifetime — often two to four hours. The credential then expires automatically around the time the environment is torn down or goes idle, so even an undetected leak is useless shortly after. Avoid the temptation of long TTLs “to be safe”: a long TTL is precisely what turns a minor leak into a lasting one.
Related
- Injecting Vault Secrets into Preview Deploys with OIDC — the complete, copy-paste OIDC-to-Vault workflow.
- Synchronizing Environment Variables Across Stages — the non-secret configuration companion to injected credentials.
- Ephemeral Preview URL Management and Routing — exposing the app that consumes these secrets.
- Environment Parity Validation Gates — verifying injected secret keys match production.