How Cloudflare Freed 100 TB by Deleting 64 Bytes at a Time

Start at the cache and you miss it. Start at the cost and you miss how it arrived. The bug lived in the hand-off.

I have lost on-call weeks to this exact shape of slow burn, so the order below is the order I fix it in.

Cloudflare published a write-up on 27 August describing how it redesigned the in-memory representation of the DNS cache inside Big Pineapple, the platform behind 1.1.1.1, and it has been circulating again this week. The result was a 56 per cent reduction in per-entry footprint, from 953 bytes to 420, which across the fleet freed roughly 100 terabytes of working-set memory.

The reason it is worth reading is not the headline number. It is that the optimisation made the cache faster at the same time, which contradicts the trade most engineers assume they are making.

What actually changed

There were five successive changes, and the order matters because each one exposed the next.

One: stop paying for growth that never happens. A Rust Vec<T> holds a pointer, a length and a capacity, which is 24 bytes on a 64-bit system, because the capacity exists so the vector can grow without reallocating. A DNS cache entry never grows after insertion. Replacing Vec and String with Box<[T]> and Box<str> drops the capacity word to 16 bytes, and each entry had eight such fields, so that is 64 bytes per entry before counting the heap space a Vec over-reserves. Cloudflare attributes more than 15 terabytes fleet-wide to this change alone.

Two: fewer lists, fewer pointers. DNS responses have answer, authority and additional sections, and each was stored as its own list with an 8-byte pointer and an 8-byte length. Since the record count always fits in a u16, Cloudflare merged the three into one list with two 2-byte offsets marking the boundaries, saving 28 bytes per entry.

Three: pack the booleans. Loose boolean fields plus the padding Rust inserts for alignment became a single bitflag byte.

Four: stop storing what you can reconstruct. Every record stored its owner name, which almost always matches the queried domain. The owner is now omitted in the common case and reconstructed from the cache key, retained only when it genuinely differs, such as after following a CNAME. Separately, the RecordData enum was sized for rare large variants like NAPTR at around 136 bytes inline, so common A and AAAA records were paying a roughly 144-byte tax for types they would never use. Boxing the large variants removed it, saving about 120 bytes per A or AAAA record, and those two types are more than 80 per cent of traffic.

Five: store the bytes as they travel. Moving data into boxes scattered it across the heap, so the final change stored only the record data, in DNS wire format, as a 2-byte length plus raw bytes, packed contiguously into a single Box<[u8]>. That eliminated both the wide enum and the per-variant allocations, and put the data where the CPU could read it in bulk.

Why it got faster instead of slower

The instinct is that memory optimisation costs speed, and it often does. Here it did not, because the two problems had the same cause.

Fewer allocations per insert means less work per insert: throughput went from 625,000 to 893,000 entries per second, up 43 per cent. Contiguous storage means better locality, so lookup latency fell from 828 to 670 nanoseconds, down 19 per cent. The wire-format change alone accounted for a 5 per cent latency reduction, and reusable temporary buffers added another 13 per cent on inserts.

In production, p99 resident memory per instance dropped from 9.3 GB to 5.3 GB and p90 from 6.5 to 3.8 GB, rolled out in stages between 18 May and 6 July 2026.

There is a real trade-off, and Cloudflare names it. Storing records as a contiguous buffer means losing random access by index, so the code walks the buffer sequentially. Records that contain domain names, such as CNAME, NS, MX and SOA, still need parsing to apply DNS name compression. The win is concentrated in the types that dominate traffic, which is exactly why it worked.

Two caveats worth carrying

The 100 terabytes is a reduction in the working set of production processes, not 130 servers removed from a rack, and multiplying 250 billion entries by 533 bytes does not give you the production figure because the servers use memory for other things. Cloudflare is reinvesting the freed space into cache capacity, which is the actual point: a larger cache has a better hit rate, and a better hit rate means fewer queries going upstream.

The other caveat is that this only pays off at a specific scale. Sixty-four bytes means nothing in an application with a few thousand objects. It means 15 terabytes when you have 250 billion of them. The general lesson is not to go and replace your vectors, it is to notice which of your data structures carry state that your usage pattern never needs.

That is the transferable idea. Vec was not a mistake. It was the correct general-purpose choice for mutable data, applied to data that is immutable the moment it is stored, at a scale where the generality became the dominant cost.

The scale detail is also why the sharding matters. Big Pineapple routes queries for the same registered domain to specific nodes using consistent hashing, which is what makes a per-entry saving multiply into a fleet-wide one instead of being smeared across nodes that each hold a fraction. Data centres with EDNS Client Subnet enabled benefit most, since they cache several versions of the same query, which is the same reasoning as why adding one server moved everyone's cache: the placement strategy decides what a per-key optimisation is actually worth.

The rule

A general-purpose container carries state for a usage pattern you may not have, and at sufficient scale that state becomes the dominant cost. Find the structures holding capacity, pointers or padding your access pattern never uses, and check whether removing them also improves locality before assuming memory and speed trade against each other.

Cloudflare saved 100 terabytes by noticing that a DNS cache entry never grows after insertion, so the capacity field it was paying for on eight fields per entry across 250 billion entries was pure waste, and the fix made lookups faster rather than slower because the memory problem and the locality problem turned out to be the same problem.

Published on debugly.dev by Rohit Bhadani | Founder of Krova Cloud