Self-Hosting a Turborepo Remote Cache Server

Turborepo’s remote cache protocol is two HTTP routes over a blob store. That is the entire surface — which means you can run it yourself in about eighty lines, keep build artifacts inside your own network, and stop paying per-seat for a cache that is fundamentally a key-value bucket.

When to use this pattern

  • Build artifacts must not leave your infrastructure for policy or contractual reasons.
  • You want cache storage and retention governed by your own lifecycle rules.
  • Your CI runners are already close to an object store, and egress to a third party is the slow part.

Prerequisites

The protocol you are implementing

The client does not need a database, a queue, or any knowledge of your build graph. It computes a hash for a task, asks for that hash, and either receives a tarball or uploads one. Everything interesting happens client-side.

Miss Then Hit: The Two-Route Protocol Run one computes a task hash, issues a GET that returns 404, executes the task, then PUTs the resulting tarball. Run two computes the same hash, issues a GET that returns 200, and restores the outputs without executing the task. turbo client your cache server RUN 1 hash = f(source, deps, env, tool versions) GET /v8/artifacts/{hash}?teamId=… 404 — miss executes the task locally (3m 40s) PUT /v8/artifacts/{hash} — tarball of the task outputs RUN 2 identical inputs → identical hash GET /v8/artifacts/{hash} 200 + tarball — restored in 1.4s, task never runs

Complete working example

// server/index.ts — a complete Turborepo remote cache over S3-compatible storage.
import express from 'express';
import { S3Client, GetObjectCommand, PutObjectCommand, HeadObjectCommand } from '@aws-sdk/client-s3';

const app = express();
const s3 = new S3Client({ region: process.env.AWS_REGION });
const BUCKET = process.env.CACHE_BUCKET!;

// Tokens are per-team and per-permission. Read-only tokens are what pull requests get.
const TOKENS: Record<string, { team: string; write: boolean }> = {
  [process.env.CI_WRITE_TOKEN!]: { team: 'shop', write: true },
  [process.env.CI_READ_TOKEN!]:  { team: 'shop', write: false },
};

app.use((req, res, next) => {
  const token = (req.headers.authorization ?? '').replace(/^Bearer\s+/i, '');
  const auth = TOKENS[token];
  // Reject unknown tokens, and reject a token used for a team it does not own:
  // team scoping is what stops one repository reading another's artifacts.
  if (!auth || auth.team !== req.query.teamId) return res.status(401).end();
  (req as any).auth = auth;
  next();
});

// Object key includes the team, so two teams can never collide on one hash.
const key = (team: string, hash: string) => `turbo/${team}/${hash}`;

app.get('/v8/artifacts/:hash', async (req, res) => {
  const k = key((req as any).auth.team, req.params.hash);
  try {
    const obj = await s3.send(new GetObjectCommand({ Bucket: BUCKET, Key: k }));
    res.setHeader('Content-Type', 'application/octet-stream');
    (obj.Body as NodeJS.ReadableStream).pipe(res);       // stream: artifacts reach 100s of MB
  } catch {
    res.status(404).end();                                // a miss is not an error
  }
});

app.put('/v8/artifacts/:hash', express.raw({ type: '*/*', limit: '500mb' }), async (req, res) => {
  if (!(req as any).auth.write) return res.status(403).end();   // read-only tokens cannot poison
  const k = key((req as any).auth.team, req.params.hash);
  // Never overwrite an existing hash: content-addressed entries are immutable by definition,
  // so a second PUT for the same hash is either redundant or an attack.
  try {
    await s3.send(new HeadObjectCommand({ Bucket: BUCKET, Key: k }));
    return res.status(200).end();
  } catch { /* not present — proceed */ }
  await s3.send(new PutObjectCommand({ Bucket: BUCKET, Key: k, Body: req.body }));
  res.status(202).end();
});

app.listen(3000);
# .github/workflows/build.yml — pull requests read, main writes.
env:
  TURBO_API:  https://turbo-cache.internal.example.com
  TURBO_TEAM: shop
  # The token differs by trigger: only trusted runs may write to the cache.
  TURBO_TOKEN: ${{ github.event_name == 'pull_request'
                   && secrets.TURBO_READ_TOKEN || secrets.TURBO_WRITE_TOKEN }}

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20, cache: npm }
      - run: npm ci --ignore-scripts
      - run: npx turbo run build --summarize    # --summarize writes a machine-readable run report

Step-by-step walkthrough

Two routes is genuinely the whole protocol. GET returns the artifact or 404; PUT stores it. There is no index, no invalidation call, and no metadata negotiation, because the hash is the identity. Anything more you build is your own operational addition.

Team scoping belongs in the object key. Deriving the key from the authenticated team rather than from a request parameter means a token cannot read outside its team even if the client sends a different teamId. The middleware also rejects the mismatch, so it fails twice — which is what you want in an authorization path.

Read-only tokens for pull requests. This is the single most important control. A pull request runs untrusted code with your cache credentials in the environment; if that token can write, an attacker can store a malicious artifact under the hash a later trusted build will request. The cache-poisoning mechanics are covered in fixing cache poisoning issues in distributed CI runners.

Read-Only and Write Tokens Against One Bucket Pull-request runs carry a read token: their GET succeeds and their PUT is rejected with 403. Main-branch runs carry a write token and may do both. Both tokens resolve to the same team prefix in the bucket, so neither can read another team's objects. pull-request run untrusted code · read token main-branch run reviewed code · write token GET PUT → 403 GET PUT → 202 cache server team from token, never from the query object store turbo/shop/9f3c2ab7e1… turbo/shop/4b81ce07fa… another team's prefix is unreachable with this token The read/write split is the control that makes a shared cache safe to expose to pull-request runs at all.

Refuse to overwrite an existing hash. Content-addressed entries are immutable. A PUT for a hash that already exists is either a harmless duplicate or an overwrite attempt, and treating it as a no-op handles both correctly.

Stream, do not buffer. Task outputs for a large application routinely exceed 200 MB. Buffering them into memory turns a cache server into an out-of-memory incident under concurrency.

Verification

# 1. The routes answer correctly, including the 404-on-miss path.
curl -s -o /dev/null -w '%{http_code}\n' \
  -H "Authorization: Bearer $CI_READ_TOKEN" \
  "$TURBO_API/v8/artifacts/0000000000000000?teamId=shop"        # → 404

# 2. A read-only token cannot write.
curl -s -o /dev/null -w '%{http_code}\n' -X PUT --data-binary @/dev/null \
  -H "Authorization: Bearer $CI_READ_TOKEN" \
  "$TURBO_API/v8/artifacts/deadbeefdeadbeef?teamId=shop"        # → 403

# 3. End to end: build, wipe local state, build again.
npx turbo run build --summarize
rm -rf node_modules/.cache/turbo .turbo
npx turbo run build --summarize
jq -r '.tasks[] | "\(.taskId)\t\(.cache.status)\t\(.cache.source // "-")"' \
  .turbo/runs/*.json | tail -5

Expected from the second build — every task restored from the remote:

web#build        HIT     REMOTE
api#build        HIT     REMOTE
emails#build     HIT     REMOTE

If the status is MISS on an unchanged tree, the hash inputs differ between runs. Compare npx turbo run build --dry=json | jq '.tasks[].hash' across the two environments — the usual culprits are an unlisted environment variable and a different Node minor version on the runner.

Operating it once it works

A cache server is easy to build and easy to neglect, and the two failure modes it develops are quiet ones.

The first is unbounded growth. Every unique hash writes an object, and hashes change on every dependency bump, so a busy monorepo can add tens of gigabytes a month. Apply a lifecycle rule expiring objects after 30 days: an artifact older than that will almost never be requested, because the inputs that produced it have moved on. The second is silent degradation — a hit rate that drifts from 85% to 30% because a task started reading an unlisted environment variable, so every run hashes differently. Nothing breaks; builds simply get slower, and by the time anyone investigates the change that caused it is months old.

Both are addressed by exporting two numbers per run: the hit rate, and the cache’s total object count. A hit-rate alert at a threshold you pick from a fortnight of baseline data will catch the hashing regression within a day of it landing, which is the difference between a one-line fix and an archaeology exercise.

A Hashing Regression Seen From the Hit-Rate Metric Cache hit rate holds between eighty and ninety per cent for four weeks, then falls to around thirty per cent in week five when a task begins reading an unlisted environment variable. An alert threshold at seventy per cent marks where the regression would have been detected. 0% 33% 66% 100% alert threshold 70% unlisted env var enters the hash W1 W4 W7 Nothing fails — builds just take four minutes longer. Only the metric makes it visible.

Common pitfalls

Giving pull requests a write token. The convenience of one token is not worth handing cache-write access to untrusted code.

Putting the cache behind a proxy with a small body limit. A default 10 MB limit silently rejects large artifact uploads with a 413 that Turborepo reports as a generic upload failure. Raise it to at least 500 MB.

Assuming a hit means correctness. A hit means the inputs hashed the same, which is only as trustworthy as your input list. Task outputs and env declarations in turbo.json are part of the correctness argument, as covered in cache-key strategies for deterministic CI builds.

← Back to Implementing Remote Build Caching with Turborepo