Splitting Jest Tests by Timing Data for Balanced Shards

Jest’s --shard splits by file path hash, so shard 3 gets the four slowest integration suites and shard 1 gets forty trivial unit files. The job takes as long as shard 3 while three runners sit idle. A custom testSequencer fixes it in about forty lines, using durations Jest already reports.

When to use this pattern

  • You already shard a Jest suite and the shard durations differ by more than about 30%.
  • Test file runtimes vary widely — a mix of pure unit tests and database-backed suites.
  • You want the fix inside Jest rather than as an external file-list generator.

Prerequisites

Where the imbalance comes from

The default sequencer is deliberately simple: hash the path, take the modulus, done. It is deterministic and requires no state, which is the right default — and it is exactly wrong for a suite whose files differ by two orders of magnitude in runtime.

Why File Count Is a Poor Proxy for Duration A histogram of forty-eight test files by duration. Thirty-eight files complete in under two seconds, six take two to ten seconds, and four database-backed suites take twenty-five to forty seconds each, accounting for over half the total runtime. Hash-based assignment distributes files evenly but can place three of the four heavy suites in one shard. 0 20 40 files < 2s 38 2–10s 6 10–25s 0 25–40s 4 share of runtime 4 files: 54% 44 files: 46% Balancing file counts distributes the 44 evenly and says nothing about where the 4 land — which is what sets the job duration. With four shards and hashing, the probability that at least two heavy suites share a shard is well over half.

Complete working example

// jest-sequencer-timed.cjs — assign files to shards by recorded duration.
const Sequencer = require('@jest/test-sequencer').default;
const { readFileSync, existsSync } = require('node:fs');
const path = require('node:path');

const TIMINGS = path.resolve('test-timings.json');   // { "src/a.test.ts": 31.4, … } seconds
const DEFAULT_SECONDS = 1.5;                          // optimistic guess for unseen files

class TimedSequencer extends Sequencer {
  constructor(...args) {
    super(...args);
    this.timings = existsSync(TIMINGS) ? JSON.parse(readFileSync(TIMINGS, 'utf8')) : {};
  }

  duration(test) {
    const rel = path.relative(process.cwd(), test.path);
    return this.timings[rel] ?? DEFAULT_SECONDS;
  }

  // Jest calls shard() before sort(); returning only this shard's slice is the whole job.
  shard(tests, { shardIndex, shardCount }) {
    // Greedy longest-processing-time: place the biggest item in the emptiest bin.
    const bins = Array.from({ length: shardCount }, () => ({ total: 0, tests: [] }));
    for (const test of [...tests].sort((a, b) => this.duration(b) - this.duration(a))) {
      const target = bins.reduce((min, b) => (b.total < min.total ? b : min));
      target.tests.push(test);
      target.total += this.duration(test);
    }
    return bins[shardIndex - 1].tests;                // shardIndex is 1-based
  }

  // Within the shard, still run slowest-first so workers stay busy to the end.
  sort(tests) {
    return [...tests].sort((a, b) => this.duration(b) - this.duration(a));
  }
}

module.exports = TimedSequencer;
// jest.config.cjs
module.exports = {
  testSequencer: '<rootDir>/jest-sequencer-timed.cjs',
  // Jest still owns the --shard flag; the sequencer only changes how it partitions.
};
# .github/workflows/timings.yml — regenerate the data nightly, serially.
name: Refresh test timings
on:
  schedule: [{ cron: '0 3 * * *' }]

permissions: { contents: write }

jobs:
  timings:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20, cache: npm }
      - run: npm ci --ignore-scripts

      # --runInBand: measured durations must not be contaminated by worker parallelism.
      - run: npx jest --runInBand --json --outputFile=raw.json

      - run: |
          jq '[.testResults[] | { (.name | sub("^\(env.PWD)/";"")):
                ((.endTime - .startTime) / 1000) }] | add' raw.json > test-timings.json
          git add test-timings.json
          git diff --cached --quiet || git -c user.name=ci -c user.email=ci@local \
            commit -m "chore: refresh test timings" && git push

Step-by-step walkthrough

shard() is the extension point, not sort(). Jest calls shard() first to select this runner’s files, then sort() to order them. Overriding only sort() reorders within an already-unbalanced slice and changes nothing about the imbalance.

Every shard runs the same packing. The algorithm is deterministic and each runner computes the full assignment, then returns its own bin. That is why no coordination is needed between runners — an important property, since they have no way to talk to each other.

Sort longest-first before packing. Greedy bin packing is only close to optimal when large items are placed first. Feeding the list in arbitrary order can leave a 40-second suite as the last item, with nowhere balanced to put it.

Sequencer Hook Order: shard() Then sort() Jest discovers all test files, calls shard to select the subset for this runner, then calls sort to order that subset. Overriding sort alone reorders an already-unbalanced subset; the balancing decision belongs in shard. discover files all 48, every runner shard() ← balance here pack by duration, return this bin sort() slowest first, within the bin run Override sort() alone and the subset is still whatever the hash chose — reordered, but not rebalanced. Both runners compute the same packing independently, which is why no coordination between them is required.

Unseen files get an optimistic default. A new test file has no timing. Guessing 1.5 seconds and letting it land in the lightest bin costs a slightly uneven run; the alternatives — skipping it, or guessing high — are worse, because one silently loses coverage and the other permanently distorts the packing.

Measure with --runInBand. Durations recorded while four workers compete for two cores are inflated and unevenly so. The nightly job runs serially precisely so the numbers describe the tests rather than the contention.

Verification

# 1. The packing is balanced — print each shard's predicted total.
for i in 1 2 3 4; do
  npx jest --shard="$i/4" --listTests --silent 2>/dev/null \
    | sed "s|^$PWD/||" \
    | jq -R . | jq -s --slurpfile t test-timings.json \
        'map($t[0][.] // 1.5) | add | floor' \
    | xargs printf "shard %s: %ss\n" "$i"
done

Expected — a spread well inside 15%:

shard 1: 96s
shard 2: 94s
shard 3: 99s
shard 4: 92s
# 2. Every file is in exactly one shard — no duplicates, no omissions.
for i in 1 2 3 4; do npx jest --shard="$i/4" --listTests --silent; done \
  | sort | uniq -d | head          # must print nothing
for i in 1 2 3 4; do npx jest --shard="$i/4" --listTests --silent; done | wc -l
npx jest --listTests --silent | wc -l    # the two counts must match

The second check is the important one. A sequencer bug that drops files produces a green run that tested less than you think — the most dangerous possible failure mode for this change.

Keeping the timing data useful

Timing data decays, and it decays in a way that quietly restores the imbalance you removed. Every new test file added between refreshes carries the default estimate, so a fortnight of active development can put a dozen unmeasured files in whichever shard was lightest at generation time — which is exactly where the packing is most sensitive.

Two habits keep it honest. Refresh nightly rather than weekly: the cost is one serial run on a schedule, and it means no file carries a guess for more than a day. And track the spread as a number — the ratio of the slowest shard to the fastest — rather than eyeballing job durations. A spread that has drifted past 1.2 usually means the timing file is stale, a heavy new suite arrived, or something in the environment changed enough that the recorded durations no longer describe reality.

Shard Spread Drifts When the Timing File Goes Stale The ratio of slowest shard to fastest shard is tracked daily over four weeks. With nightly refresh it stays between one point zero two and one point zero eight. When the refresh job is paused at the start of week three, the ratio climbs steadily to one point six as new unmeasured files accumulate in one shard. 1.0× 1.3× 1.6× alert above 1.2× nightly refresh paused here W1 W2 W4 Nothing fails while this climbs — the job simply takes as long as its worst shard, and the runners idle again.

Common pitfalls

Recording timings from a sharded run. Durations measured under parallelism describe contention, not the tests, and feeding them back makes the next packing worse. Always measure serially.

Committing timings from a pull-request run. The pull-request environment may have a different database or cache state. Generate from the main branch on a schedule so the data has one consistent source.

Assuming the sequencer applies to --runInBand. It does — but with one shard the packing is a no-op, so a local single-shard run tells you nothing about whether the sequencer works. Test it with --shard, as in the verification step.

Commit the timing file rather than caching it. A cached timing file is invisible in review, expires silently, and cannot be inspected when the packing looks wrong. A committed one shows up in diffs, can be checked out at any past commit, and makes the nightly refresh job’s behaviour auditable.

← Back to Parallelizing Test Suites in CI