From 45 Minutes to Under 5: CI/CD Build Acceleration

A 45-minute build doesn't feel like an emergency, which is why it survives so long. But run the math: 45 minutes, multiplied by every PR, multiplied by every push-to-fix-CI retry, multiplied by a team of a dozen engineers. That's easily 15-20 engineer-hours burned per week on nothing but waiting. It also directly caps your deployment frequency, since nobody ships more often than their pipeline allows.

The changes below are listed in the order we applied them, because sequencing matters: fix the wrong stage first and you waste effort re-measuring later. This sequence took a monolith pipeline from 45 minutes to 4 minutes 30 seconds.

Where the time went

No single cache accounts for most of the time saved, the bigger shift is structural: turning a fully serial pipeline into one where independent work runs concurrently.

1. Dependency caching, keyed correctly

The most common caching mistake: keying the cache on the branch name instead of the lockfile hash. That means the cache almost never hits on a fresh branch, right when you need it most.

# .github/workflows/ci.yml
- name: Cache node_modules
  uses: actions/cache@v4
  with:
    path: |
      ~/.npm
      node_modules
    key: npm-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
    restore-keys: |
      npm-${{ runner.os }}-

restore-keys matters as much as key; it lets a near-miss (a slightly different lockfile) still restore most of the cache instead of starting from zero.

2. Docker layer caching via BuildKit registry cache

Rebuilding the full image on every push is the second-largest time sink after dependency installs. BuildKit's registry-backed cache persists layers across separate CI runs:

- name: Build and push with layer cache
  uses: docker/build-push-action@v6
  with:
    context: .
    push: true
    tags: registry.example.com/app:${{ github.sha }}
    cache-from: type=registry,ref=registry.example.com/app:buildcache
    cache-to: type=registry,ref=registry.example.com/app:buildcache,mode=max

Pair this with a Dockerfile ordered so rarely-changing layers (system packages, dependency installs) come before frequently-changing ones (application source):

FROM node:20-slim
WORKDIR /app
 
# Dependencies change far less often than source, install them first
# so this layer stays cached across most commits.
COPY package.json package-lock.json ./
RUN npm ci --omit=dev
 
COPY . .
RUN npm run build

3. Splitting the test suite across parallel shards

A single-process test run doesn't get faster just because your CI runner has 8 cores. Something has to tell the test runner to use them, or the job has to be split across multiple runners:

jobs:
  test:
    strategy:
      matrix:
        shard: [1, 2, 3, 4]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run test shard ${{ matrix.shard }}
        run: npx jest --shard=${{ matrix.shard }}/4

Four shards on four parallel runners turn a 15-minute serial test run into roughly 4 minutes. Not exactly 15/4 (there's always a slowest shard), but close enough to matter.

4. Runner sizing: bigger isn't always the fix

Before reaching for self-hosted or larger runners, exhaust caching and parallelization first: they're close to one-time investments, while larger runners scale your bill linearly with usage. We typically reach for bigger runners only on the Docker build step, where more CPU cores meaningfully speeds up multi-stage builds.

The GitLab CI equivalent

The same three levers apply almost 1:1 on GitLab, with needs used to break the default stage-by-stage serialization into a true dependency graph:

test:
  parallel: 4
  script:
    - npx jest --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL
  cache:
    key:
      files:
        - package-lock.json
    paths:
      - node_modules/
 
deploy:
  needs: ["test", "docker-build"]  # runs as soon as both finish, not after every prior stage
  script:
    - ./deploy.sh

Stage-by-stage result

| Stage | Before | After | |---|---|---| | Install dependencies | 8m 00s | 0m 45s | | Build | 12m 00s | included in cached build | | Unit tests | 15m 00s | 2m 10s (4-way parallel) | | Docker build | 7m 00s | 1m 10s (layer cache) | | Push + deploy | 3m 00s | 1m 05s | | Total | 45m 00s | 4m 30s |