Cloudflare saves 100 terabytes of memory in 1.1.1.1's DNS cache

Cloudflare runs Big Pineapple, the platform behind the 1.1.1.1 DNS service, as well as Gateway DNS, DNS Firewall and other company services. The system holds over 250 billion entries in its cache at any given time, the temporary memory that allows it to answer requests to translate an internet address into an IP address more quickly. At that scale, even a single wasted byte per entry costs more than 250 gigabytes of memory across the whole infrastructure.

Cloudflare's engineers made five changes to how cache entries are stored in memory. The result is a reduction of over 50% in the space taken up by each individual entry. Across the company's entire infrastructure, the changes freed up roughly 100 terabytes of memory, an amount equal to the RAM contained in 130 of the company's newest, generation 13, servers.

The changes did not only concern the space used. The speed of inserting new entries into the cache rose by 43%, while the time needed to retrieve one dropped by 19%.

How the cache works

Each cache entry consists of a key, which identifies the request, and a value, which contains the DNS response along with its data and metadata such as the creation time and validity period. Several fields in these structures used data types with unnecessary space overhead once the entry was stored.

Among the most significant changes is the replacement of some standard data structures of the Rust programming language with more compact versions, which do not reserve space for future growth, since an entry, once stored, is never modified again. Another change concerns how domain names are stored: in most cases, the domain name in the response matches that of the original request, so it does not need to be stored a second time.

Finally, the engineers changed how individual DNS records are stored, moving from a format with overly generous fixed margins to one that uses only the space actually needed. This also reduced the time needed to read the data, thanks to more direct memory access.

Cloudflare verified the effect of the changes by measuring the memory actually used by its own servers during the system's rollout in production, confirming the results obtained in testing.