← deemwar engineering examples Runnable · Node · zero deps · free

The process that's "helping" is the one blocking every write.

A shared session or database store needs an exclusive lock — two processes racing to write it at once corrupts it. That's correct. It also means whichever process holds it, for whatever reason, blocks every other write for as long as it runs.

The crux

This shows up in any system with a per-store exclusive lock: messaging automation, database migrations, deploy scripts. A maintenance job — a re-sync, a health check that opens the store, an interactive debug session — is legitimate and needed. Left running past the point anyone is watching it, it holds the lock the whole time, and everything else that touches the store silently queues or fails. The fix isn't a retry or a longer timeout. It's stopping the process that looks like it's helping.

function acquireLock(lockDir) {
  fs.mkdirSync(lockDir); // atomic: fails with EEXIST if already held
}

That's correct locking. The failure isn't a bug in the lock — it's a second process that legitimately needed it, left running past the point anyone was watching.

Run it

node check.js --demo

Zero dependencies, no network — spawns a real child process that acquires the lock and holds it for 2 seconds (simulating a maintenance job that outlived its purpose), then races a second write attempt against it.

$ node check.js --demo Store: /tmp/lock-demo-4gpjzA Spawning a real child process to simulate a maintenance/re-sync job holding the store lock for 2s... Attempting a send (lock wait 400ms) while the holder is still running... BLOCKED after 400ms. Lock held by: {"role":"maintenance-job","pid":64969,...} DIAGNOSIS: the write did not fail because anything is broken -- it failed because a DIFFERENT process, started to help (a check, a migration, a re-sync), is still holding the exclusive store lock. Waiting longer does not fix it if that process is left running indefinitely. Stopping IT does. Waiting for the holder (pid 64969) to release naturally... Holder exited (lock released). Retrying the send now... Send acquired the lock in 0ms. Works fine once the holder is gone.

Against your own store directory (any lock implementation using a LOCK dir/file with a PID recorded inside):

node check.js --store-dir <path> [--wait-ms 1000]

If a lock is present, it reports how old it is, who acquired it, and — if a PID was recorded — whether that process is still alive. Dead PID: a stale lock, safe to investigate removing. Live PID: it genuinely still owns the store.

The fix is a habit, not code

Anything that acquires a store-wide write lock (a migration, a full resync, an interactive debug session) should run in the foreground, watched, and stopped the moment it's done — not launched and forgotten.
Monitoring for the write path should distinguish "the lock is held by something else" from a generic failure, so the response is "go stop the holder," not "retry harder."
Keep lock-wait timeouts short in production paths — a long wait just delays the same diagnosis.

Check your own system

Does your store lock record who holds it (PID, role, start time), or just that it's held?
Do your runbooks for "fix the store" say to run the fix command and watch it, or leave it as a background step someone might forget?
Can your monitoring say "writes are blocked because X is running," apart from "writes are failing for an unknown reason"?

Get the code

Source on GitHub