Skip to main content
All posts Use cases Services Contact

The deployment checklist every backend we ship passes

Web Development Operations

The demo works on the laptop. The difference between that and week three in production is a list — and the list is short enough that there is no excuse for skipping items, only for skipping them silently.

Secrets never live in the repo

API keys belong in a secret store, one per environment, rotatable without a deploy. The repository carries .env.example with the key names and no values. If a key has ever been committed, rotating it is part of the migration, not a cleanup task for later — git history is public sooner or later.

Migrations are expand-contract

Every schema change ships in three steps across deploys: expand (additive, old code still works), migrate (backfill and dual-write), contract (drop the old shape, days later). Rollback then means redeploying the previous code against an expanded schema — which works precisely because the migration never assumed it was the only version running.

-- expand: additive, old code unaffected
ALTER TABLE orders ADD COLUMN status_v2 text;
-- backfill, dual-write, verify counts

-- contract (days later, once nothing reads it)
ALTER TABLE orders DROP COLUMN status;

Health checks that reflect real work

A process being alive is not a health check. Readiness should verify the things a request actually needs — the database answers, the queue accepts, the dependency latency is inside budget — and liveness should only restart a wedged process. A container that is "healthy" while every request returns a 500 will pass every orchestrator on earth.

The rollback has been exercised

One command, one documented procedure, run at least once in staging against real traffic shape. A rollback that exists only in a wiki is a hypothesis. Preview deployments per pull request catch most of what would need a rollback in the first place.

Observability on day one

Every request carries an id that threads through logs, errors and any queue hop. Alerts go to a person, not a dashboard — a dashboard nobody watches is a museum, and the first alert of an incident should arrive before the first customer complaint. This is the same rule we hold agents to; backends just fail more slowly.

DNS and email are part of the launch

The most common launch-week delay is not code: it is a domain still propagating, a certificate waiting on DNS, or email landing in spam because SPF, DKIM and DMARC were never verified from an outside mailbox. Start those a week early and verify them with a real send to a real external account, not with a settings page screenshot.

The list, on one screen

  • Secrets in a store, one environment each, rotation rehearsed.
  • Migration expand-contract, safe to run twice, rehearsed on production-sized data.
  • Readiness and liveness probes wired to real dependencies.
  • Rollback exercised once, not merely documented.
  • Error tracking and alert routes with a named human owner.
  • SPF, DKIM and DMARC verified by sending to an external mailbox.
  • Perf budget measured on a mid-range phone, not office wifi.
  • A backup restored once — a backup nobody restored is a rumour.