Serialization Failure Try Restarting Transaction, and Why Retrying Is Correct

The application logged ERROR: could not serialize access due to concurrent update, SQLSTATE 40001, and the on call engineer treated it as data corruption, because that is what a transaction being killed from the outside feels like. It was not corruption. It was the strongest isolation level in the database correctly refusing to let two transactions disagree about history.

The unsettling part of 40001 is that the database is not malfunctioning. It has detected that completing this transaction would produce a history that no serial execution could have produced, and rather than quietly return a wrong answer, it aborts and asks you to retry. The error is the safety feature.

This was Postgres 16.3 running serializable isolation on a hot accounting table. The mechanics apply to any version using SSI.

The short answer

Serializable isolation guarantees that concurrent transactions behave as if they ran one after another. Postgres implements this with serializable snapshot isolation, tracking read and write dependencies. When the dependency graph shows a cycle, meaning two transactions each read something the other wrote, no serial order exists, and Postgres aborts one of them with 40001.

The aborted transaction did nothing wrong and lost no data. It is simply the one chosen to go again. The message's own advice, "The transaction might succeed on rerun", is the entire contract.

Why it feels like a bug

Most codebases run at read committed, where 40001 never appears, so the first encounter is usually after someone deliberately raised the isolation level to fix a real anomaly. The team then sees the new error and assumes the fix broke something. In fact the isolation level is now catching the anomaly that read committed was silently allowing, and reporting it as a retryable abort instead of as wrong data.

So 40001 arriving after an isolation upgrade is evidence the upgrade is working, not that it failed. The anomaly moved from invisible wrong results to visible retryable errors, which is a strict improvement, provided the client retries.

The only correct client behaviour

The transaction must be retried, and the retry must restart the whole transaction, not the failed statement. The abort invalidates everything the transaction did, because its reads were part of the dependency cycle. Retrying a single statement inside a now aborted transaction is meaningless.

The retry loop belongs at the boundary where the transaction is defined:

await db.transaction(async (tx) => { ... }, {
  isolationLevel: "Serializable",
  maxAttempts: 5,
});

If your ORM does not have that, write the loop yourself around a function that opens a fresh transaction each attempt. The loop should be small, bounded, and add a little jitter so two colliding transactions do not retry in lockstep, which is the same jitter argument as reviewing retry and backoff logic.

When retrying is the wrong answer

Retry is correct when the collision is genuinely concurrent and rare. If 40001 is constant, the design is the problem, and retrying turns the database into a spin lock. A few patterns cause chronic serialization failures.

Long transactions over hot rows. The longer a serializable transaction lives, the more dependencies it accumulates and the more cycles it forms. Keep them short and narrow.

Read then write over a whole table. A transaction that aggregates a table and then writes a summary depends on everything it read, so any concurrent writer collides. This shape often belongs outside serializable isolation, in a single writer or an atomic upsert.

Using serializable as a default out of caution. Serializable is a correctness tool for specific races, not a global setting. Applying it everywhere converts every concurrency into a potential abort.

Choosing the isolation level honestly

The honest ladder is to start at read committed, which is the default and is fine for most work, and raise only the transactions with a named anomaly to prevent. Repeatable read removes non repeatable reads but not write skew. Serializable removes write skew and the rest, at the cost of retryable aborts.

The decision is a trade between silently allowing a class of anomaly and paying for it in retries. Name the anomaly you are preventing, and if you cannot name it, you probably do not need the level. This is the same discipline as postgres deadlock detected, where the error is the system resolving contention rather than a defect.

Observing it

Count 40001s per transaction type, and alert on the rate, not the presence. A low steady rate with successful retries is the system working. A rising rate means collisions are increasing, usually from a new hot writer or a longer transaction, and it will show up as retry latency before it shows up as failures.

Also log the number of attempts each transaction needed. A transaction that routinely needs three attempts is a design smell even when it eventually succeeds, because it is burning the retries the rare case will need.

The rule

40001 is the database choosing visible correctness over silent wrongness. Retry the whole transaction, bounded and with jitter, and treat a chronic rate as a design signal to shorten or narrow the transaction, not as a database fault.

It is worth naming the cultural half of this. Teams that treat 40001 as an error to be eliminated will weaken the isolation level to make it disappear, and the anomaly it was preventing will return as silent wrong data, which is strictly worse and much harder to find. The mature posture is the opposite: keep the strongest isolation the correctness case justifies, make the retry boring and automatic, and let the error rate be a monitored signal of contention. The error is not the disease. It is the immune response, and suppressing it does not cure anything.

The error you should fear is the one that never arrives, the silent anomaly at a weaker isolation level, because that one ships wrong numbers instead of asking for a retry. The silent truncation cousin of that lesson is the export that always returned 999 rows.