Incremental Builds and Affected Detection in Monorepos

PR validation pipelines in a growing monorepo routinely rebuild every workspace regardless of what changed — burning runner minutes, delaying developer feedback, and masking legitimate failures in noise. Affected detection solves this by computing a directed acyclic graph (DAG) of workspace dependencies, diffing the change set against it, and dispatching only the tasks whose transitive inputs actually changed. This guide covers the full production path: graph construction, CI integration, remote cache synchronisation, and the failure modes that corrupt results at scale.

Prerequisites

Before wiring affected detection into CI, confirm the following are in place:

• Node.js 20 or later (node --version)
• Nx 19+ or Turborepo 2+ installed in the repo root
• Workspace manifests (package.json workspaces or pnpm-workspace.yaml) list every app and package
• Full git history available in CI (fetch-depth: 0 in GitHub Actions; GIT_DEPTH: 0 in GitLab)
• Write access to a remote cache bucket or Nx Cloud token (optional but strongly recommended)
• Runner has network egress to the cache endpoint

How the Dependency Graph and Affected Scope Work

Both Nx and Turborepo parse workspace manifests and build a DAG where each node is a project and each directed edge is a dependencies / devDependencies declaration pointing to another workspace package. When a change set arrives (a list of file paths from git diff base..HEAD), the tool walks the graph upward — any project that owns a changed file, plus any project that depends on it transitively, is marked affected.

The critical constraint: the graph reflects declared dependencies only. A dynamic import() evaluated at runtime, or a path alias that bypasses the manifest, is invisible to the static analyser. This is why teams maintain strict eslint-plugin-import rules and explicit outputs declarations in turbo.json — undeclared edges are the root cause of most false-negative affected misses.

The base commit for the diff matters enormously. In a PR pipeline the correct base is git merge-base HEAD origin/main, not origin/main directly — the latter may exclude commits from the feature branch that landed after the branch diverged, producing a wider diff and triggering unnecessary rebuilds.

Step-by-Step Implementation

1. Initialize workspace dependency declarations

Verify that every internal dependency is declared explicitly in each workspace’s package.json:

{
  "name": "@acme/web",
  "dependencies": {
    "@acme/ui": "workspace:*",
    "@acme/api-client": "workspace:*"
  }
}

Run the graph visualiser to confirm edges are resolved:

# Nx
npx nx graph --file=graph.json && cat graph.json | jq '.graph.nodes | keys'

# Turborepo
npx turbo run build --dry=json | jq '.tasks[].package'

If a workspace you expected to appear is missing, the dependency declaration is absent from its manifest.

2. Configure base-commit tracking

Export the merge-base SHA as an environment variable that all subsequent steps share. This is the single most important configuration decision — an incorrect base produces unreliable results:

# Compute the true divergence point from the target branch
BASE=$(git merge-base HEAD origin/main)
echo "BASE_SHA=$BASE" >> "$GITHUB_ENV"   # GitHub Actions
# or
echo "BASE_SHA=$BASE" >> build.env       # GitLab artifact env

3. Wire GitHub Actions

name: Monorepo — Affected CI
on:
  pull_request:
    branches: [main]

jobs:
  affected:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0          # full history required for merge-base calculation

      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: pnpm

      - run: corepack enable && pnpm install --frozen-lockfile

      # Restore Nx / Turborepo local cache between runs
      - uses: actions/cache@v4
        with:
          path: |
            .nx/cache
            .turbo
          key: ${{ runner.os }}-nx-${{ hashFiles('**/pnpm-lock.yaml') }}-${{ github.sha }}
          restore-keys: |
            ${{ runner.os }}-nx-${{ hashFiles('**/pnpm-lock.yaml') }}-
            ${{ runner.os }}-nx-

      # Nx: run lint + test + build only on affected projects
      - name: Nx affected
        run: |
          BASE=$(git merge-base HEAD origin/${{ github.base_ref }})
          npx nx affected \
            --target=lint,test,build \
            --base=$BASE \
            --head=${{ github.sha }} \
            --parallel=4
        env:
          NX_CLOUD_ACCESS_TOKEN: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}

Line-by-line explanation:

• fetch-depth: 0 — without full history git merge-base returns nothing.
• hashFiles('**/pnpm-lock.yaml') — cache key changes only when the lockfile changes, protecting against stale node_modules hits.
• --target=lint,test,build — a single affected invocation fans out across all three targets; Nx respects dependsOn ordering automatically.
• --parallel=4 — caps concurrent task processes; tune to available runner vCPUs.
• NX_CLOUD_ACCESS_TOKEN — enables distributed task execution and remote cache on Nx Cloud.

Verify the step produces the expected project list before running tasks:

npx nx show projects --affected --base=$BASE --head=${{ github.sha }}

4. Wire GitLab CI

GitLab does not expose the target branch tip SHA as a ready-made variable. Use git merge-base against the fetched target branch:

stages: [affected, build]

.affected_base: &affected_base
  image: node:20-alpine
  before_script:
    - git fetch --unshallow origin $CI_MERGE_REQUEST_TARGET_BRANCH_NAME
    - export BASE=$(git merge-base HEAD origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME)
  cache:
    key:
      files: [pnpm-lock.yaml]
    paths: [.nx/cache, node_modules/.cache]

lint-and-test:
  <<: *affected_base
  stage: affected
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
  script:
    - corepack enable && pnpm install --frozen-lockfile
    - npx nx affected --target=lint,test --base=$BASE --head=$CI_COMMIT_SHA --parallel=4

build:
  <<: *affected_base
  stage: build
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
  script:
    - corepack enable && pnpm install --frozen-lockfile
    - npx nx affected --target=build --base=$BASE --head=$CI_COMMIT_SHA --parallel=4
  artifacts:
    paths: [dist/]
    expire_in: 1 day

Line-by-line explanation:

• --unshallow — GitLab shallow clones omit ancestry required for merge-base; this restores it.
• cache.key.files — generates a unique cache key per lockfile state, preventing cross-branch pollution.
• CI_PIPELINE_SOURCE == "merge_request_event" — gates execution to MR pipelines only; branch pipelines on main should run a full build via a separate job.
• --parallel=4 — GitLab shared runners provide 2 vCPUs; 4 keeps the queue saturated without OOM.

Verify affected output before committing to the full pipeline:

npx nx show projects --affected --base=$BASE --head=$CI_COMMIT_SHA

5. Integrate remote caching

For Turborepo remote caching, inject the three required environment variables and enable remote-only mode on PR builds:

env:
  TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
  TURBO_TEAM:  ${{ vars.TURBO_TEAM }}
  TURBO_API:   ${{ vars.TURBO_API }}

- name: Build affected (Turborepo)
  run: |
    BASE=$(git merge-base HEAD origin/${{ github.base_ref }})
    npx turbo run build \
      --filter="...[origin/main]" \
      --remote-only \
      --cache-dir=.turbo

The --filter="...[origin/main]" syntax is Turborepo’s native affected scope: it expands to all workspaces with changes since origin/main.

Configuration Reference

Integration with Sibling Topics

Incremental builds do not operate in isolation — they are most effective when combined with two adjacent capabilities from the Build Optimization & Caching Strategies section:

Remote Build Caching: Affected detection narrows which tasks run; remote caching eliminates re-running those tasks when artifacts already exist from a previous identical input hash. Without remote cache, every new runner restarts the affected tasks from scratch even if the source code is unchanged from a sibling branch.

Docker Layer Caching for Full-Stack Applications: When containerised runners produce Docker images as build artifacts, align Dockerfile COPY directives with workspace boundaries so that only the layers corresponding to affected workspaces are invalidated. Mount the .nx/cache or .turbo directory into the build container to avoid redundant dependency resolution steps.

Nx affected PR gating: The sibling deep-dive covers Nx-specific configuration in detail — branch protection rules, required status check wiring, and handling the edge case where no projects are affected (the empty-affected guard).

For tracking the financial impact of incremental builds, feed affected-count metrics into CI/CD compute cost tracking to demonstrate ROI to stakeholders.

Performance Benchmarks

Field data from platform teams running Nx or Turborepo affected pipelines on mid-sized monorepos (20–60 workspaces, active team of 10–30 engineers):

The graph initialisation overhead (8–15 s) is a fixed cost that dominates on very small PRs. For single-file hotfixes touching a standalone utility package, the overhead can exceed the build time — acceptable given the savings across the broader PR queue.

Remote cache hit rates plateau at 72–85 % after ~2 weeks of steady-state traffic, once the most common input hashes have been populated. Hit rates below 60 % typically indicate environment drift (mismatched Node.js versions or OS between runners).

Troubleshooting

Cannot find module '@acme/ui' after workspace install

Root cause: pnpm install --frozen-lockfile ran before internal packages were built; a consumer workspace imports the compiled output that does not yet exist.

Fix: Add an explicit dependsOn in nx.json or turbo.json so the dependency’s build task runs before the consumer’s:

{
  "tasks": {
    "build": {
      "dependsOn": ["^build"]
    }
  }
}

The ^ prefix means “all upstream workspaces’ build tasks”.

merge-base returns empty string in GitLab

Error text: fatal: Not a valid object name ''

Root cause: The pipeline checked out a shallow clone. git merge-base requires ancestry data that shallow clones omit.

Fix:

git fetch --unshallow origin $CI_MERGE_REQUEST_TARGET_BRANCH_NAME

Add this to before_script before computing BASE.

Affected set includes every workspace on every PR

Root cause: A root-level file (package.json, nx.json, turbo.json, .eslintrc.js) changed, causing the entire graph to be marked affected. Or --base is set to the initial commit (HEAD~1 on a squash merge).

Fix: Exclude root config changes from the affected expansion using project-level tags and targetDefaults to scope impact. For release-config files, accept the full rebuild — correctness trumps speed here.

Cache hit rate drops below 50 % after adding a new runner pool

Root cause: The new runners run a different OS or Node.js patch version; the cache key includes the runner OS (runner.os in GitHub Actions), which differs.

Fix: Normalise all runners to the same base image. Include OS and Node version in the cache key explicitly:

key: ${{ runner.os }}-node20-nx-${{ hashFiles('**/pnpm-lock.yaml') }}

Frequently Asked Questions

Verifying the Affected Set Is Actually Correct

Affected detection is trusted implicitly once it works, which makes its silent failure mode expensive. A set that is too wide is slow and obvious; a set that is too narrow is fast and wrong, and the wrongness surfaces later as a production defect in a package nobody built.

Two checks are worth running continuously. The first is a full build on the main branch after merge, regardless of what the pull request built. It is not blocking anyone, it costs runner minutes rather than developer time, and it converts an incorrect affected set from a production incident into a red build with an obvious cause. The second is a lint rule forbidding cross-package relative imports, which removes the most common source of undeclared dependency edges — the edges the graph cannot see and therefore cannot follow.

Together they close the gap between what the dependency graph knows and what the code actually does, which is the only place a correct-looking affected computation can be wrong.

And measure the distribution before investing. If most pull requests in your repository already affect a single package, incremental detection will remove most of the work; if they consistently affect everything, the coupling that makes them wide is the more useful thing to fix, and no amount of graph tooling will substitute for it.

Related

• Implementing Remote Build Caching with Turborepo — eliminate redundant task execution by sharing artifact hashes across all runners in your organisation.
• How to Configure Nx Affected Commands for Faster PR Checks — Nx-specific branch protection rules, required status check wiring, and the empty-affected guard pattern.
• Docker Layer Caching for Full-Stack Applications — align container build layers with workspace boundaries to skip redundant image rebuilds.
• Tracking CI/CD Compute Costs for Platform Teams — quantify the runner-minute savings from incremental builds and report ROI against infrastructure spend.

← Back to Build Optimization & Caching Strategies