How we cut CI from 45 minutes to 8 minutes — a practical breakdown
Ajitesh MohantaAmbassador
Mar 18, 2026 5,243 0
Six months ago our test pipeline blocked PRs for 45 minutes. Today it's 8 minutes. Here's every change we made, in order of impact:
**1. Fixed test isolation (freed us to parallelise)**
Tests had hidden ordering dependencies. We ran `pytest-randomly` for 2 weeks to find and fix them. Time saved: this was the prerequisite, not a speedup itself.
**2. Sharding across 6 GitHub Actions runners** → 38 min → 12 min
Plateau here because our slowest shard (Stripe integration tests) dominated.
**3. Moved integration tests to a nightly pipeline** → 12 min → 9 min
Fast feedback on PRs, full suite every night. Acceptance compromise we debated a lot.
**4. Proper pip caching with hash on requirements.txt** → 9 min → 8 min
Small but consistent saving on every run.
**5. Didn't help: switching to faster test hardware**
We tried GitHub's larger runners. Minimal impact because our bottleneck was test logic, not CPU.