Logged In Users Saw Somebody Else's Search Results

The ticket said search was broken. The user searched for a product they had bought before and got results for industrial fasteners. They do not sell industrial fasteners.

Two hours later there were eleven tickets. Then a merchant called. The common thread was that every affected user was logged in, and every one of them saw the same wrong results, which were correct for exactly one account.

This was a Node 22.14 storefront with a Redis 7.2 cache in front of a Postgres 16.3 product search, behind nginx 1.27.

The symptom, stated precisely

Authenticated users hitting /search?q=bearing received a result set belonging to a different customer account. Anonymous users got correct results every time. Reloading the page sometimes fixed it and sometimes made it worse.

That last detail should have told me everything. Nondeterminism in a cache means key collision, and I did not follow it for another ninety minutes.

The hypotheses that died

It is a search index problem

My first instinct was the full text index. Two accounts with overlapping product slugs, a misconfigured search_vector, maybe a stale materialised view. I dumped the index, ran the query directly against Postgres with the affected account's filter applied, and got the right answer in four milliseconds.

It died because the database was never wrong. Every single time I bypassed the application and asked Postgres directly, it answered correctly. The corruption was happening above it.

It is a session bug

Next I assumed the session was leaking. Maybe a user object was being shared across requests, a middleware assigning to a module level variable, the classic Node mistake of writing to something that outlives the request.

I logged the session ID and the user ID at the top of every request handler for twenty minutes of production traffic. Every request had the correct user attached. The session was fine.

It died because the identity was right at the point of entry. Whatever was wrong happened after the user was correctly identified.

It is a race condition

Then I went after concurrency. Two requests in flight, a shared query builder, a promise resolving into the wrong continuation. I had written about this exact shape before in race conditions that disappear when you log them, so I went looking for the same pattern.

I added a request ID to every log line and traced a single bad response end to end. The request ID was consistent from entry to response. One request, one response, no interleaving.

It died because there was no concurrency involved. A single request, handled in isolation, with a cold application process, would still return the wrong results if the cache was warm.

That was the test that mattered, and I should have run it first.

The breakthrough

I restarted the application, cleared nothing, and made one authenticated request as the affected user. Correct results. I made the same request as a different user. Wrong results, showing the first user's data.

The cache was keyed on the request path and query string:

const key = `search:${req.path}:${req.url.split("?")[1] ?? ""}`;

Nothing about the customer. Not the account ID, not the customer group, not the warehouse the request should resolve against. Two merchants on the same platform, both with a product category called "bearing", both hitting /search?q=bearing, and whoever populated the cache first won for everyone.

The reason anonymous users were unaffected is that their results are genuinely account independent. The cache was correct for them by accident.

What I changed

The immediate fix was one line:

const key = `search:${req.accountId}:${req.customerGroup}:${req.path}:${qs}`;

But a one line fix to a cache key is not a fix, it is a patch on the specific key you noticed. So I did three more things.

Every cache key now goes through one function. No string templates at call sites. A single cacheKey(scope, parts) helper that takes an explicit scope argument and throws if the request has a customer attached and the scope does not include the account.

I added a test that asserts isolation, not correctness. Two accounts, same path, same query, different expected results. Seed the cache as account A, request as account B, assert B does not see A's data. That test fails against the old code in about forty milliseconds.

I logged the key on every cache hit at debug level for a week. Not because I expected to find more, but because seeing the keys printed together made two more collisions obvious that nobody had reported yet. One of them was on the pricing endpoint, which is considerably worse than search.

What I would do differently

I would have cleared the cache first. Not as a fix, as a diagnostic. If a bug disappears when you clear a cache and comes back when the cache warms, you have a cache key bug, and every other hypothesis is a waste of time.

I also spent too long trusting the shape of the symptom. "Search returns wrong products" reads like a data problem, so I looked at data. The detail that made it a caching problem was that multiple unrelated users saw the same wrong answer. Shared wrong output is a shared state signature every time.

And I would have written the isolation test before the cache. A test that asks "can account A see account B's data through this path" is a test worth having for every cached endpoint in a multi tenant system, and it is the kind of test that catches this class of bug at review time rather than at two in the afternoon on a Tuesday.

If your cache is serving the wrong thing but always the same wrong thing, the key is too small. Go and look at it. Related reading: cache invalidation and stale data bugs covers what happens when the key is right but the lifetime is not.