Midnight. A DELETE without WHERE on the orders table. The admin opens the "restore backup" ticket. At 2 a.m., the MySQL dump refuses to import—incompatible charset. At 4 a.m., media files are missing: they lived on an unsnapshotted volume. At 6 a.m., the client learns nobody ever ran the drill.
Testing a restore is not paranoia. It is the only proof your backup strategy works outside PowerPoint.
Two minimum scenarios to simulate
Scenario A — Targeted human error. One table or directory deleted. Goal: partial restore under your RTO without rebuilding everything.
Scenario B — Compromise or ransomware. Production suspect. Goal: rebuild on clean infra from immutable copy dated before intrusion—not the latest backup.
| Scenario | Question tested | Measurable success |
|---|---|---|
| Table DROP | Partial dump / PITR | Orders OK, data consistent |
| Ransomware | Immutable copy D-5 | No malware, secrets rotated |
| Bad deploy | Code rollback + aligned DB | Version N-1 works |
| Datacenter loss | Off-site restore | DNS switched, RTO met |
If you test only one scenario, test the worst—not the most comfortable.
Sandbox: reproduce without risk
Isolated machine — disposable VPS, local Docker compose, offline VM.
Anonymized data if GDPR applies—mask emails and PII for frequent tests; keep one annual drill on a faithful copy in a closed environment.
Ordered checklist — test DNS, DB import, file sync, env vars, workers, cache flush, smoke test login + payment.
Time each step. A theoretical 2h RTO that becomes 8h in practice is the drill's most valuable finding.
What the drill almost always reveals
- Expired backup credentials or 2FA blocking automated restore.
- Empty or zero-byte dumps unnoticed.
- MySQL restore version incompatible with dump source.
- Missing
.env—app starts but sends no mail. - Wrong order: site up, images 404 because object storage was not resynced.
Log each gap in a dated runbook. That is your capital for the real crisis.
Measure and improve
Set simple KPIs:
- Total time vs target RTO.
- Simulated data loss vs target RPO.
- Manual step count—candidate for automation.
- Tickets opened during drill—missing training?
Share a short report with the team. Failure in sandbox is gold; failure in production is a lost client.
The climax: the host's "verified" backup is not your restore
This is the line marketing SLAs erase. Run internal drills; do not outsource restore competence.
Decide and move forward without blind spots
- Schedule the next test (fixed date, not "when we have time").
- Prepare sandbox + representative dataset.
- Run scenarios A and B with a stopwatch.
- Fix the backup pipeline before closing.
- Compare hosts on restore ease via directory.
See also restore proof to document the drill for clients or auditors.
Frequently asked questions
How often should I test a restore?
At least quarterly for critical sites; semiannually for brochure sites. Retest after major infra or backup changes.
Should I restore to production?
No. Isolated sandbox. The drill measures process, not live uptime.
What if restore fails?
Document cause, fix pipeline, retest within the week with proof.
Partial or full restore?
Both by scenario: deleted table = partial; ransomware = full rebuild from older clean copy.
A backup not restored in sandbox is not insurance. It is an assumption—and assumptions always break on a Tuesday night.