Generating a Dynamic Job Matrix from JSON in GitHub Actions

Your matrix lists eleven workspaces, a pull request touches one, and CI builds all eleven anyway. A dynamic matrix fixes that: a small setup job decides what needs to run, prints it as JSON, and the matrix expands to exactly those entries — nothing hard-coded, nothing stale when someone adds a workspace.

When to use this pattern

• The set of things to build changes over time — workspaces, packages, target environments.
• Most pull requests touch a small subset, and you are paying to build the rest.
• The matrix currently duplicates a list that already exists somewhere else in the repository.

Prerequisites

• GitHub Actions with job outputs available (all supported runners)
• jq on the runner — present on all GitHub-hosted images
• A deterministic way to enumerate build units: workspace globs, a config file, or nx show projects
• Branch protection you can point at a single aggregate check

How the expansion works

strategy.matrix is evaluated before the job starts, from an expression. That expression can reference a previous job’s output, and fromJSON turns a string into the array or object the matrix needs. The sequencing is the part worth internalising, because every failure mode is a sequencing failure.

Complete working example

# .github/workflows/build.yml — matrix computed from what the pull request touched.
name: Build
on: [pull_request]

jobs:
  plan:
    runs-on: ubuntu-latest
    outputs:
      targets: ${{ steps.plan.outputs.targets }}
      count:   ${{ steps.plan.outputs.count }}
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }        # full history: the diff below needs the merge base

      - id: plan
        run: |
          BASE="origin/${{ github.base_ref }}"
          # Every workspace directory that has at least one changed file under it.
          changed=$(git diff --name-only "$BASE"... \
            | awk -F/ '$1=="packages" && NF>2 {print $2}' | sort -u)

          # Shared code invalidates everything — build the full set instead.
          if git diff --name-only "$BASE"... | grep -qE '^(package-lock.json|tsconfig.base.json)$'; then
            changed=$(ls packages)
          fi

          # jq -c produces ONE line, which is what GITHUB_OUTPUT requires.
          json=$(printf '%s\n' $changed | jq -R . | jq -sc .)
          echo "targets=$json"            >> "$GITHUB_OUTPUT"
          echo "count=$(echo "$json" | jq 'length')" >> "$GITHUB_OUTPUT"
          echo "matrix will be: $json"

  build:
    needs: plan
    if: needs.plan.outputs.count != '0'   # a zero-length matrix is a hard error, so skip instead
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        target: ${{ fromJSON(needs.plan.outputs.targets) }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20, cache: npm }
      - run: npm ci --ignore-scripts
      - run: npm run build --workspace "packages/${{ matrix.target }}"

  # Fixed name, always runs — this is the required status check.
  build-complete:
    needs: [plan, build]
    if: always()
    runs-on: ubuntu-latest
    steps:
      - name: Assert the matrix succeeded or was legitimately skipped
        run: |
          # "skipped" is success here: it means nothing needed building.
          case "${{ needs.build.result }}" in
            success|skipped) echo "ok (${{ needs.plan.outputs.count }} target(s))" ;;
            *) echo "build matrix failed: ${{ needs.build.result }}"; exit 1 ;;
          esac

Step-by-step walkthrough

fetch-depth: 0 is not optional. The default shallow checkout has no merge base, so git diff origin/main... compares against nothing useful and either errors or returns the whole tree. The extra fetch costs a few seconds and is the difference between a correct plan and a plan that silently builds everything.

Emit compact JSON on one line. GITHUB_OUTPUT is a key-value file parsed line by line; pretty-printed JSON spanning several lines is truncated at the first newline and produces a parse error in fromJSON that names neither the job nor the value. jq -sc guarantees a single line.

Handle the “shared file changed” case explicitly. A lockfile or base tsconfig change affects every workspace, and a diff-based plan will happily report zero targets because no packages/* path changed. Widening to the full set on those paths is the cheap, safe answer; the alternative is a green pull request that never built anything.

Guard the empty matrix. GitHub rejects a zero-length matrix with Matrix must define at least one vector — a job-level error, not a skip. The if: on count != '0' turns “nothing to do” into a clean skip.

Know which JSON shape you are emitting. fromJSON accepts three different shapes and each one means something different to the matrix, which is where most confusing expansions come from.

The practical advice is to keep the flat array as the default and reach for the object form only when a target genuinely needs per-entry settings, such as a different Node version or a longer timeout. The full-matrix form is powerful and easy to get wrong, because a computed cross product can quietly grow past the concurrency budget discussed in optimizing pipeline concurrency and queue limits.

Add an aggregate job for branch protection. Matrix job names embed the matrix values, so build (web) exists on one pull request and not the next; you cannot list them as required checks. A fixed-name job that inspects needs.build.result gives branch protection something stable to require, and treating skipped as success is what stops a documentation-only pull request from being blocked forever.

Verification

# 1. The plan step's JSON is valid and single-line, before you trust it in CI.
BASE=origin/main
git diff --name-only "$BASE"... | awk -F/ '$1=="packages" && NF>2 {print $2}' | sort -u \
  | jq -R . | jq -sc . | tee /tmp/targets.json | jq 'type, length'
# → "array"
# → 3

# 2. The workflow expression parses the same way GitHub will.
jq -e 'type == "array" and (map(type == "string") | all)' /tmp/targets.json \
  && echo "matrix-safe"

# 3. After a run, confirm only the expected jobs were created.
gh run view --json jobs --jq '.jobs[].name' | sort

Expected from the third command on a single-workspace pull request:

build (web)
build-complete
plan

If you see every workspace listed, the diff fell back to the full set — check the merge base and the shared-file rule before assuming the filter is wrong.

Common pitfalls

Building the matrix from a stale list. If the setup job reads a hard-coded array, you have moved the duplication rather than removed it. Derive the list from the filesystem or from the tool that already owns it, as in configuring Nx affected commands.

Forgetting fail-fast: false. With the default, the first failing target cancels the rest, so a developer fixes one workspace, pushes, and finds another — turning one red run into several. The exception is a matrix used purely for compatibility smoke tests, where a single failure genuinely does invalidate the run and cancelling saves real minutes.

Emitting objects when you meant strings. matrix.target is whatever the array element was. An array of objects gives you matrix.target.name, and forgetting the extra field produces the literal string [object Object] in a path. Keep the array flat unless you genuinely need several dimensions, which is covered in excluding and including matrix combinations.

Related

• Managing Environment Matrices in GitHub Actions — the parent guide on matrix design and environment targeting.
• Excluding & Including Matrix Combinations Without Duplication — trimming a multi-dimensional matrix once it expands.
• Incremental Builds & Affected Detection in Monorepos — richer ways to compute the affected set.
• CI/CD Pipeline Architecture & Fundamentals — the section overview.

← Back to Managing Environment Matrices in GitHub Actions