Your accounting export cron runs "every hour". One morning, finance receives two identical files with duplicated entries. Digging in: the 08:00 job had not finished when 09:00 started — same script, same temp table, no mutex.
On a single VPS, this is classic. On scaling infra, it is worse: two servers run the same crontab because nobody centralised the scheduler.
Why crons overlap
| Cause | Scenario | Symptom |
|---|---|---|
| Job too long | Import > interval | Duplicates, DB deadlocks |
| Manual restart | Admin + cron | Two parallel processes |
| Multi-server | crontab copied on 2 VPS | Double client billing |
| Host retry | PHP timeout kill + retry | Partial commit × 2 |
A cron is not "idempotent" by default — you must prove it or lock it.
flock: file lock on a single host
* * * * * flock -n /var/lock/sync.lock /usr/local/bin/sync.sh >> /var/log/sync.log 2>&1
-n: non-blocking — skip if lock held (preferable to infinite queue).- Lock on local disk shared if multiple users — not unstable NFS for critical locks.
PHP alternative:
$fp = fopen('/var/lock/job.lock', 'c');
if (!flock($fp, LOCK_EX | LOCK_NB)) {
exit(0);
}
// work...
Distributed lock: Redis and PostgreSQL
Redis (short pattern, TTL mandatory):
SET lock:export $(uuid) NX EX 3600
Release with Lua compare-and-del script to avoid deleting another process's lock.
PostgreSQL advisory lock — practical if the job already touches the DB:
SELECT pg_try_advisory_lock(12345);
Lock releases at session end — handy for long SQL jobs.
Architecture: cron trigger vs single worker
For heavy jobs, cron only enqueues:
- Cron minute → push job ID into Redis queue.
- Single worker consumer processes the queue.
- Visibility timeout on message if crash.
Laravel Horizon, Sidekiq, Celery beat — same principle. Cron becomes a wake-up, not execution itself.
Monitoring and logs
Log start and end with PID, duration and status. Correlate with database metrics — deadlocks and duplicate rows are often the first signal. On shared hosting, check server timezone and PHP limits before blaming code.
The peak: idempotence does not replace the lock — it complements it
Both together survive Black Friday night.
Decide and move forward without blind spots
Audit every scheduled cron: compare observed maximum duration to configured interval. Add flock immediately on any single-node VPS. Centralise the scheduler if you have multiple servers — one cron node, or shared Redis lock. Log start and end with PID to correlate duplicates. See our hosting guide for cron and PHP limits by offer.
Frequently asked questions
Is flock enough to lock a bash cron?
Yes on a single server: flock -n /var/lock/myjob.lock /path/script.sh prevents a second concurrent instance. The lock disappears if the process dies — verify the script does not fork without releasing the lock.
How do you lock a cron across multiple servers?
Use a distributed lock: Redis SET NX EX, PostgreSQL advisory lock, or DynamoDB lock. flock does not work across machines.
What if the previous job runs longer than the cron interval?
Either increase the interval, allow skip (flock -n exiting 0), or queue the work (single worker). Never let two massive imports overlap.
Do shared-hosting crons have specific traps?
Yes: server time vs local time, different PHP paths, silent timeout kill, and multiple nodes without coordination if the host scales infrastructure without telling you.
A cron without a lock is roulette: one day the task finishes twice — the day you are not watching the logs.