Sharding Playwright Tests Across GitHub Actions Runners

Playwright has native sharding, so the workflow is mostly plumbing — and the plumbing is where the time goes. Browser binaries downloaded four times, a report that only exists per shard, and a merge job that skips itself whenever a test fails: this is the configuration that avoids all three.

When to use this pattern

  • A Playwright suite is the longest job in the pipeline and its files are independent.
  • You already have a shard count in mind from measuring the suite, per parallelizing test suites in CI.
  • You need a single readable report and per-failure traces, not four partial ones.

Prerequisites

What each shard has to set up

The per-shard overhead is the thing that decides whether adding a shard helps, and on Playwright it is dominated by one avoidable download.

Per-Shard Setup: Uncached vs Cached Browsers For four shards, the uncached configuration spends eight seconds on checkout, sixteen on install and forty on browser download before any test runs, repeated per shard. With the browser cache restored, the download drops to four seconds, saving roughly two and a half runner-minutes across the matrix. WITHOUT A BROWSER CACHE — per shard 8s install 16s chromium download 40s tests × 4 shards = 2m 40s of pure download across the matrix WITH THE BROWSER CACHE RESTORED 8s install 16s 4s tests start 36 seconds earlier, on every shard The binaries live in ~/.cache/ms-playwright — outside node_modules, so the npm cache never covers them. This is why a shard count that "should" help sometimes does not: the fixed cost per shard was larger than anyone measured. Cache the browsers first, then decide the shard count — otherwise you are tuning against the wrong overhead.

Complete working example

# .github/workflows/e2e.yml
name: E2E
on: [pull_request]

jobs:
  e2e:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false          # one failing shard must not hide the others
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20, cache: npm }
      - run: npm ci --ignore-scripts

      # Pin the cache to the exact Playwright version: a version bump ships new
      # browser builds, and restoring the old ones produces confusing launch errors.
      - id: pw
        run: echo "version=$(node -p "require('@playwright/test/package.json').version")" >> "$GITHUB_OUTPUT"
      - uses: actions/cache@v4
        id: browsers
        with:
          path: ~/.cache/ms-playwright
          key: playwright-${{ runner.os }}-${{ steps.pw.outputs.version }}

      # On a hit, install only the OS packages; on a miss, download the browsers too.
      - if: steps.browsers.outputs.cache-hit == 'true'
        run: npx playwright install-deps chromium
      - if: steps.browsers.outputs.cache-hit != 'true'
        run: npx playwright install --with-deps chromium

      - name: Run shard ${{ matrix.shard }} of 4
        run: npx playwright test --shard=${{ matrix.shard }}/4 --reporter=blob
        env:
          BASE_URL: ${{ vars.PREVIEW_URL }}

      - uses: actions/upload-artifact@v4
        if: always()            # a failed shard's blob is the one you most want
        with:
          name: blob-${{ matrix.shard }}
          path: blob-report
          retention-days: 1

  report:
    needs: e2e
    if: always()                # merge even when a shard failed — otherwise no report exists
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20, cache: npm }
      - run: npm ci --ignore-scripts
      - uses: actions/download-artifact@v4
        with: { pattern: blob-*, path: blobs, merge-multiple: true }
      - run: npx playwright merge-reports --reporter=html,github ./blobs
      - uses: actions/upload-artifact@v4
        with: { name: playwright-report, path: playwright-report, retention-days: 7 }
// playwright.config.ts — worker count set explicitly for the runner, not the laptop.
import { defineConfig } from '@playwright/test';

export default defineConfig({
  // shards × workers = concurrent browsers. 4 × 2 = 8 against one preview backend
  // is usually fine; 4 × 4 on a 2-core runner is memory contention, not parallelism.
  workers: process.env.CI ? 2 : undefined,
  retries: process.env.CI ? 1 : 0,
  reporter: process.env.CI ? [['blob']] : [['list']],
  use: {
    baseURL: process.env.BASE_URL,
    trace: 'retain-on-failure',
    video: 'retain-on-failure',
  },
});

Step-by-step walkthrough

Cache keyed on the Playwright version, nothing else. Browser builds are tied to the package version, so the version is the complete cache key. Keying on the lockfile hash instead invalidates the browsers on every unrelated dependency change; keying on nothing invalidates never, and a version bump then launches a stale binary with an error that reads like a broken test.

install-deps on a hit, install --with-deps on a miss. The cache holds the browser binaries but not the system libraries they link against, and the runner image does not ship them all. Splitting the two paths keeps the warm case at four seconds instead of forty.

Blob is the only mergeable reporter. HTML and list reporters produce per-shard output that cannot be combined. Blob is an intermediate format designed for merge-reports, and switching to it is what makes the single-report goal reachable at all.

if: always() twice, for different reasons. On the upload, because a failing shard’s blob is the one containing the trace you need. On the merge job, because otherwise a single failure leaves you with four artifacts and no report — the exact situation the merge job exists to prevent.

Shards × Workers = Concurrent Browsers A grid with shard counts of two, four and six across the top and workers per shard of one, two and four down the side. Cells show the resulting concurrent browser count. Combinations up to eight are marked comfortable, twelve to sixteen borderline, and twenty-four or more as contention against a single preview backend. workers ↓ / shards → 2 shards 4 shards 6 shards 1 worker 2 browsers 4 browsers 6 browsers 2 workers 4 browsers 8 — the usual choice 12 — watch the backend 4 workers 8 browsers 16 — memory pressure 24 — timeouts, not speed The limit is usually the shared preview backend, not the runners — twenty-four concurrent sessions will saturate a single-instance staging API.

Set workers explicitly. Playwright defaults to roughly one worker per core. Four shards each defaulting to two workers on a two-core runner is eight concurrent browsers against one preview backend, which produces timeouts that look like application faults. This interaction is covered further in running E2E tests against preview environments.

Verification

# 1. The browser cache is being hit — check a shard's log.
gh run view --log --job "e2e (2)" | grep -iE 'cache (restored|not found)'

# 2. All four blobs arrived and the merge consumed them.
gh run view --log --job report | grep -E 'Downloading|merge-reports|report' | head

# 3. The merged report covers the whole suite, not one shard.
gh run download -n playwright-report -D /tmp/report
jq '[.stats.expected, .stats.unexpected, .stats.flaky] | @tsv' /tmp/report/report.json 2>/dev/null \
  || grep -o 'Total.*tests' /tmp/report/index.html | head -1

Expected total test count from the third command should match the serial run’s:

248 tests across 4 shards — the same number a single-shard run reports

If the merged count is roughly a quarter of that, merge-multiple: true was omitted from the download and only one blob was read.

Getting the shard count right for this suite

Playwright suites have a characteristic that changes the arithmetic: the fixed per-shard cost is unusually high, because browsers and system dependencies must be present before a single test runs. Even fully cached, a shard does not start testing for roughly 30 seconds.

That sets a floor. If the whole suite runs in three minutes, four shards means each does about 45 seconds of testing on top of 30 seconds of setup — 40% overhead, for a wall-clock of 75 seconds against 180. Real, but eight shards would be worse rather than better: each would do 22 seconds of testing on 30 seconds of setup, and the wall-clock would stop improving while runner-minutes nearly doubled.

The practical rule for Playwright specifically is to stop when the slowest shard’s test time falls below its setup time. In most repositories that lands between three and six shards, and it is a lower ceiling than for unit tests precisely because of the browser install. Measure with the timings from the run summary rather than guessing, and re-check after any change to the browser matrix — adding Firefox and WebKit multiplies the setup cost per shard.

Setup Time Sets the Floor for Shard Count Four configurations of a three-minute suite. At two shards, ninety seconds of testing sits on thirty seconds of setup. At four, forty-five on thirty. At six, thirty on thirty — the break-even. At ten, eighteen seconds of testing on thirty seconds of setup, where most of the job is overhead. SLOWEST SHARD — setup (solid) + testing (light) 2 shards 30s + 90s — testing dominates, room to split 4 shards 30s + 45s — a good place to stop for most suites 6 shards 30s + 30s — break-even; past here overhead wins 10 shards 30s + 18s — 62% of every job is setup Adding Firefox and WebKit raises the solid block on every bar and lowers the useful shard count accordingly.

Common pitfalls

Caching node_modules and expecting browsers to come with it. They do not — the binaries live in ~/.cache/ms-playwright. Two caches, two keys.

Using the HTML reporter per shard. You end up with four independent reports and no way to combine them. Blob per shard, HTML once at the end.

Leaving retries high to hide flakiness. Sharding surfaces genuine test coupling, and three retries buries it again while tripling the cost of a bad shard. One retry absorbs transient network faults; anything more is a decision to stop seeing the problem.

← Back to Parallelizing Test Suites in CI