Setting up Redis-backed remote caching for GitLab CI

Replacing ephemeral local runner caches with a shared Redis backend eliminates the runner-specific cache fragmentation that causes cold-start latency spikes across distributed GitLab CI fleet deployments.

When to use this pattern

• Your pipeline fleet spans more than one GitLab Runner host and jobs fail to reuse each other’s caches.
• node_modules/.cache or node_modules/.vite cache misses are adding more than 90 seconds per run, and S3 transfer overhead is not acceptable for sub-second read latency requirements.
• You need centralized TTL management and ACL-controlled access across multiple projects without provisioning per-project object storage buckets.

Prerequisites

• GitLab Runner 13.0 or later installed on every runner host (gitlab-runner --version)
• Redis 6.2+ reachable from all runners, ACL and TLS configured (see topology section below)
• CI/CD masked variables REDIS_PASSWORD and REDIS_TLS_CA defined at the project or group level
• Network latency below 50 ms RTT between runners and Redis (measure with redis-cli --latency)
• gitlab-runner process owner has read access to the TLS CA bundle path

Architecture: how the Redis backend fits in

The diagram below shows the data path from a developer push through GitLab CI to Redis and back. The runner daemon intercepts every cache: directive in .gitlab-ci.yml, compresses the listed paths into a tarball, and reads or writes that tarball via Redis — the pipeline YAML never references Redis directly.

Complete working example

The two files below form a self-contained configuration. Drop them onto an existing GitLab Runner host and project — no other changes are required.

# /etc/gitlab-runner/config.toml
# Restart the runner daemon after editing: sudo gitlab-runner restart

[[runners]]
  name = "frontend-runner-01"
  url  = "https://gitlab.example.com"
  token = "__RUNNER_TOKEN__"
  executor = "docker"

  [runners.cache]
    Type   = "redis"
    Shared = true          # all runners sharing this config share the same cache namespace

    [runners.cache.redis]
      # Use the rediss:// scheme (double-s) for TLS — plain redis:// sends credentials in clear text
      URL = "rediss://:${REDIS_PASSWORD}@redis.internal:6379/0"

      TLSInsecureSkipVerify = false   # never disable TLS verification in production
      PasswordFromFile      = ""      # set this to a file path if you prefer secrets on disk

# .gitlab-ci.yml
# The cache: directive selects paths and the key; the daemon resolves the Redis backend transparently.

variables:
  VITE_CACHE_DIR:    "$CI_PROJECT_DIR/node_modules/.vite"
  WEBPACK_CACHE_DIR: "$CI_PROJECT_DIR/node_modules/.cache/webpack"

build:
  image: node:22-alpine
  cache:
    # Key locks to the branch slug + lockfile hash so parallel feature branches never collide
    key:
      prefix: "${CI_PROJECT_PATH_SLUG}-${CI_COMMIT_REF_SLUG}-v2"
      files:
        - package-lock.json          # invalidate automatically when deps change
    paths:
      - node_modules/.vite/          # Vite's persistent module resolution cache
      - node_modules/.cache/webpack/ # Webpack 5 filesystem cache
    policy: pull-push                # restore at start, write back on success
  before_script:
    # Ensure cache directories exist even on a first-run (cache miss) scenario
    - mkdir -p "$VITE_CACHE_DIR" "$WEBPACK_CACHE_DIR"
  script:
    - npm ci
    - npm run build

Step-by-step walkthrough

1 — Redis topology and ACL hardening

Allocate at least 4 GB RAM per Redis node; LRU eviction below that threshold causes cache thrash during peak pipeline concurrency. Set maxmemory-policy allkeys-lru so Redis evicts the least-recently-used keys when memory pressure peaks rather than returning errors.

Restrict the runner service account to the minimum required ACL command set:

# Run once on the Redis host as an admin user
redis-cli ACL SETUSER gitlab-runner on >"${REDIS_PASSWORD}" \
  ~* +get +set +del +exists +expire

~* grants key-space access across all keys; tighten to a project-specific pattern (e.g. ~my-org-*) on multi-tenant clusters.

2 — config.toml: daemon-level backend switch

The [runners.cache] block in config.toml is the only place GitLab Runner learns about Redis. The URL field accepts the rediss:// scheme (TLS) or redis:// (plain). Inject the password via the REDIS_PASSWORD CI/CD variable so it never appears in the file on disk:

# Validate the TLS handshake before restarting the runner
openssl s_client -connect redis.internal:6379 \
  -CAfile "$REDIS_TLS_CA" < /dev/null 2>&1 | grep "Verify return code"
# Expected: Verify return code: 0 (ok)

sudo gitlab-runner verify   # confirms runner can reach GitLab and resolves the config
sudo gitlab-runner restart

Setting Shared = true means any runner host with the same config.toml block shares the same Redis namespace. Set Shared = false to silo caches per runner host — useful during a phased migration.

3 — .gitlab-ci.yml: path mapping for Vite and Webpack

The cache:key:files syntax introduced in GitLab 12.5 generates a SHA-256 over the listed files and appends it to the prefix. When package-lock.json changes the key rotates automatically, triggering a cold start — no manual version bump required.

For Vite’s persistent module cache, set cacheDir explicitly in vite.config.ts to align with the path declared in .gitlab-ci.yml:

// vite.config.ts
import { defineConfig } from 'vite'

export default defineConfig({
  cacheDir: 'node_modules/.vite',   // must match the path in .gitlab-ci.yml cache.paths
})

For Webpack 5, confirm cache.type is 'filesystem' and that cacheDirectory matches:

// webpack.config.js (relevant excerpt)
module.exports = {
  cache: {
    type: 'filesystem',
    cacheDirectory: path.resolve(__dirname, 'node_modules/.cache/webpack'),
  },
}

4 — TTL and eviction policy

Set expire_in in the runner’s cache configuration to bound Redis storage consumption. The runner writes this TTL on every successful push operation:

[runners.cache]
  Type      = "redis"
  Shared    = true

  [runners.cache.redis]
    URL      = "rediss://:${REDIS_PASSWORD}@redis.internal:6379/0"
    # TTL written to Redis on every successful cache push (seconds)
    # 172800 = 48 hours; tune per artifact volatility

GitLab Runner does not expose a direct expire_in TOML key for Redis TTL — set it in the .gitlab-ci.yml cache: block instead:

cache:
  key: "${CI_PROJECT_PATH_SLUG}-${CI_COMMIT_REF_SLUG}-v2"
  paths:
    - node_modules/.vite/
    - node_modules/.cache/webpack/
  policy:    pull-push
  expire_in: 48 hours   # GitLab passes this TTL to the configured cache backend

Verification

After restarting the runner and triggering a pipeline, confirm the cache is being written and read:

# On the Redis host — list keys written by the runner
redis-cli --scan --pattern "${CI_PROJECT_PATH_SLUG}-*"
# Expected output: one or more keys per branch slug

# Check hit/miss and eviction counters
redis-cli INFO stats | grep -E "keyspace_hits|keyspace_misses|evicted_keys"
# keyspace_hits:   rising on repeated pipeline runs → cache is working
# evicted_keys:    non-zero and rising → increase maxmemory or lower expire_in

# In GitLab CI — look for this line in the job log
# "Checking cache for frontend-app-main-v2..."
# "Successfully extracted cache"   ← HIT
# "No URL provided, cache will not be downloaded"   ← config error

Compare pipeline durations before and after in CI/CD → Pipelines → Charts. A warm Redis cache typically cuts npm ci + build time by 60–80% for projects with stable node_modules.

Common pitfalls

TLS handshake failures after runner restart

Symptom: tls: failed to verify certificate or connection refused in runner logs immediately after updating config.toml.

Fix: The REDIS_TLS_CA variable must be available to the runner daemon process, not only to CI jobs. Export it in the runner’s systemd unit or environment file:

# /etc/systemd/system/gitlab-runner.service.d/override.conf
[Service]
Environment="REDIS_TLS_CA=/etc/ssl/certs/redis-ca.pem"

Then systemctl daemon-reload && systemctl restart gitlab-runner.

High miss rate despite identical dependency versions

Symptom: Cache key matches expected pattern but job log shows No URL provided or every run is a cold start.

Fix: Confirm that cache:key:files lists the correct lockfile path relative to the project root, and that the file is committed (unstaged changes are invisible to the CI file hash). Also verify all runner hosts point to the same Redis endpoint — a mis-matched URL in one config.toml creates a silent silo:

grep -r "URL" /etc/gitlab-runner/config.toml

Redis memory eviction breaking mid-pipeline restores

Symptom: tar: Error is not recoverable or partial extraction warnings in the cache-restore step.

Fix: A cache key evicted between job dispatch and restore causes a corrupted partial read. Set maxmemory-policy allkeys-lru (not noeviction) so Redis evicts cleanly rather than returning errors. Also reduce expire_in to match your actual pipeline cadence — a 7-day TTL on a project that deploys ten times a day wastes memory holding keys that will never be restored:

redis-cli CONFIG SET maxmemory-policy allkeys-lru
redis-cli CONFIG SET maxmemory 4gb
redis-cli CONFIG REWRITE   # persist to redis.conf

For an emergency cache flush during a suspected corruption event:

# Flush all keys for one project — replace prefix with your CI_PROJECT_PATH_SLUG
redis-cli --scan --pattern "my-project-*" | xargs -r redis-cli DEL \
  && echo "Cache cleared. Next pipeline run will be a cold start."

This approach aligns with the broader cache poisoning prevention strategies applicable to any distributed runner fleet.

Frequently Asked Questions

Related

• Optimizing Webpack and Vite for CI Environments — parent guide covering Webpack 5 filesystem cache and Vite cacheDir configuration for all CI platforms.
• Build Optimization & Caching Strategies — architectural overview of caching layers, toolchain trade-offs, and remote cache topology for frontend and full-stack pipelines.
• Implementing Remote Build Caching with Turborepo — complements Redis-backed GitLab caching for monorepos by adding task-graph-aware artifact deduplication.
• Fixing Cache Poisoning Issues in Distributed CI Runners — covers integrity validation and emergency flush procedures applicable across Redis and S3 cache backends.