Caching Strategies for Production: CDN, HTTP Cache, Redis, and In-Memory

Part 8 ended on the database: every query costs a connection, a lock, and some latency, and most of that cost is paid again each time two requests ask for the same thing. Caching is the obvious answer, and the one with the most ways to go wrong.

A cache is a copy of an answer kept closer to whoever is asking. The speed-up is real. So is the bill: the copy can be wrong, someone has to decide when it stops being trusted, and a copy kept in the wrong place can be shown to the wrong person. Most production caching bugs are not performance bugs. They are correctness and security bugs that look like performance work.

Part 2 already covered the HTTP caching headers (Cache-Control, ETag, Vary). This post does not repeat the reference. It looks at the whole stack of places a response can be cached, how they differ, what happens when a hot entry expires, and how to invalidate without serving stale data forever. As in the previous posts, I tested what I could: browser caching in a real browser, and the in-process cache behaviors in Node. The Redis and CDN details come from their official documentation, and I mark them that way, because I did not run those systems.

Quick answer

• Cache in layers, closest to the user first. Browser, then CDN, then your application's memory, then a shared store such as Redis, then the database. Each layer trades freshness for speed.
• Decide the staleness you can tolerate before you choose the layer. "Up to 30 seconds old" and "must be current" are different systems.
• The cache key is a security boundary. Anything that changes the response (the tenant, the user's permissions, the language) must be part of the key, or one person will receive another's data.
• Expiry is when caches hurt you. A hot entry that expires can send every waiting request to the database at once. Coalesce those requests, serve stale data while refreshing, and spread out expiry times.
• Invalidation has a race. Deleting a key after a write is not enough on its own. A slow reader can write the old value back.
• Cache the data, not the permission decision. Part 5's authorization has to run for every caller.
• Measure the hit ratio. A cache you do not measure is a cache you assume works.

browser cache  -->  CDN / shared proxy  -->  app memory  -->  Redis (shared)  -->  database
 (one user)        (many users, by URL)    (one instance)    (all instances)       (the truth)
   fastest ......................................................................... slowest
   most stale  ..................................................................... freshest

When not to cache

Caching is not free, so start by asking whether you need it.

• Does it fix a measured problem? If a page is slow because a query is missing an index (Part 8), caching hides the problem and adds a second one. Fix the query first.
• Is the data cheap to produce? A cached value that takes 2 ms to recompute saves nothing and costs a network hop.
• Is it personalized or changing every second? A low hit ratio means you pay for the cache and still hit the database.
• Would a stale answer be harmful? Account balances, stock reservations and permission checks usually should not come from a cache that can lag.

Good candidates are read-heavy, expensive to compute, shared across many requests, and tolerant of some delay: public pages, configuration, reference data, expensive aggregates, and responses from slow third-party APIs.

The HTTP layer: what the browser really does

Browsers and CDNs follow Cache-Control, so this layer is configured with response headers. I ran a small server that counted how many times each URL actually reached it, and a page that fetched each URL twice using the default cache behavior. The browser was a Chromium-based one; behavior can differ between browsers.

The last row is a trap worth knowing. The response had no Cache-Control header at all, only a Last-Modified date ten days in the past, and the browser still reused it without asking the server. That is heuristic freshness: when a response has no explicit lifetime, caches are allowed to estimate one from how old the content is. If you want a response revalidated or not stored, say so explicitly; never rely on the absence of headers.

Two more results from the same test:

• Vary is honored. With Vary: Accept-Language, fetches with en, fr, fr caused 2 server requests: the second fr was served from the cache entry for fr, and the first fr was a miss even though an en entry existed. Without a correct Vary, a cache can serve one language to everyone.
• A request can force revalidation. fetch(url, { cache: 'no-cache' }) on a URL with max-age=60 went back to the server (2 requests total for that URL). That is forced revalidation; a hard reload goes further and bypasses the cache.

Stale-while-revalidate

RFC 5861 defines two extensions that matter for availability. With stale-while-revalidate=N, a cache may serve a stale response for up to N seconds after it expires, while it revalidates in the background. With stale-if-error=N, a cache may serve a stale response when the origin returns a 500, 502, 503 or 504. The RFC's example is Cache-Control: max-age=600, stale-while-revalidate=30: fresh for 600 seconds, then served stale for up to 30 more seconds while a revalidation runs.

I tested the first in the browser against an origin that takes 300 ms to respond, with max-age=1, stale-while-revalidate=10:

first fetch                                  315 ms   (cold, origin responds)
fetch after expiry, inside the SWR window      8 ms   (served stale instantly)
server requests afterwards                     2      (the initial one + a background revalidation)

The user did not wait for the origin, and the cache refreshed itself quietly. This is one of the highest-value, lowest-effort headers for public content, and the same idea reappears below for application caches.

Private and shared caches

A browser's cache belongs to one user. A CDN or proxy cache is shared by everyone, so the rules for what it may store are stricter. private tells shared caches not to store a response, while still letting the user's own browser keep it (my private, max-age=60 test above was reused by the browser). Cloudflare's documentation, as an example of CDN behavior, says it does not cache a response when Cache-Control is private, no-store, no-cache or max-age=0, and its list of non-cached cases also includes responses with a Set-Cookie header and requests whose method is not GET. It also says Cloudflare decides what to cache by file extension rather than content type, so HTML and JSON are not cached by default; caching them takes an explicit rule. Check your own CDN's documentation, because defaults differ, and test with real responses.

CDN caching and the danger of a shared cache

A CDN stores responses and serves them to many users from a location near them. For static assets that is pure benefit. For anything dynamic it is where cache bugs become security incidents.

Web cache deception is the best-known example. PortSwigger describes it as tricking a web cache into storing sensitive, dynamic content, through a discrepancy between how the cache and the origin interpret a URL. Their example: a request for /user/123/profile/wcd.css. The origin may ignore the trailing wcd.css and return the profile page, while the cache sees a path ending in .css, treats it as a static file, and stores the response. The attacker then requests the same URL and receives the victim's cached page. The recommended defenses are the ones you would hope for:

• Mark dynamic, user-specific responses with Cache-Control: no-store and private.
• Configure the CDN so its own rules do not override the origin's Cache-Control.
• Use any CDN protection that checks that the response content type matches the file extension in the URL.
• Make sure the origin and cache interpret URL paths the same way.

This is a direct consequence of earlier parts: a response that depends on who is asking (Part 5) must never be stored under a key that does not include who is asking.

In-process caches: fast, local, and forgetful

The fastest cache is a variable in your application's memory: no network, no serialization. It is a good fit for small, hot, read-mostly data: configuration, feature flags, a compiled permission set, reference tables.

It has four limits:

1. Each instance has its own copy. With three instances, you have three caches that expire at different times, so two requests can see different values. Acceptable for configuration, not for anything users compare.
2. It disappears on restart or deploy. Every deploy starts cold, which matters for the stampede problem below.
3. It needs a size bound, or it will grow until the process is killed. An LRU (least recently used) policy evicts the entry that has gone unused longest. In JavaScript, a Map iterates in insertion order, which makes a compact LRU possible: re-insert on read, delete the first key when over capacity. I verified the behavior: with capacity 3, after setting a, b, c, reading a, then setting d, the cache kept c, a, d; b was evicted.
4. Do not assume it persists. In serverless and edge runtimes, an instance may be short-lived or run in many places at once, so treat in-memory state there as an optimization, never as storage.

A shared cache: Redis

A shared store such as Redis sits between your instances and the database, so every instance sees the same cached value. The standard way to use it is cache-aside: the application checks the cache, and on a miss loads from the database and fills the cache.

async function getProduct(id: string) {
  const key = `product:v2:${id}`            // namespaced and versioned (see "Designing keys")
  const cached = await redis.get(key)
  if (cached) return JSON.parse(cached)

  const row = await db.query.products.findFirst({ where: eq(products.id, id) })
  if (row) await redis.set(key, JSON.stringify(row), { EX: 300 })  // expire in 300 s
  return row
}

The following details come from Redis's own documentation, not from my testing:

• SET takes expiry and condition options. EX seconds sets a time to live, NX sets the key only if it does not exist, and XX only if it does. Overwriting a key with SET discards its previous TTL unless you pass KEEPTTL. Forgetting this is a classic way to create entries that never expire.
• Memory is bounded by maxmemory, and an eviction policy decides what goes. On 64-bit systems the default is no limit. The policies include noeviction (refuse writes when full), allkeys-lru, allkeys-lfu (least frequently used), allkeys-random, and volatile-* variants that only evict keys that have a TTL, plus volatile-ttl. The docs suggest allkeys-lru as a good default when a subset of keys is accessed far more often than the rest. They also warn that volatile-* policies behave like noeviction when no keys have an expiry, which means a cache full of keys without TTLs stops accepting writes.
• Measure the hit ratio from the server. INFO stats reports keyspace_hits and keyspace_misses; the docs give the hit percentage as keyspace_hits / (keyspace_hits + keyspace_misses) * 100, and suggest checking evicted_keys and expired_keys when it is lower than expected.
• Locks with SET key value NX EX are described in the docs as a simple pattern, but they now discourage it in favor of the Redlock algorithm, which offers better guarantees. If you are tempted to use a cache as a lock, read that page first.

Operationally, a shared cache adds a network hop to every request and a new way to fail. Decide what happens when it is unreachable: fall back to the database, and make sure the database can survive the full load, because the cache was probably the reason it could. A short timeout on cache calls, and a limit on how many requests may fall through, prevent a cache outage from becoming a database outage.

The expiry problem: stampedes, stale data and jitter

Everything above is the easy part. The hard part is the moment an entry is missing: it expired, was evicted, or the instance just started. I tested the behaviors in Node with a backend that takes 200 ms.

A cold key under concurrency

Fifty concurrent requests for the same missing key:

naive cache-aside   -> 50 backend calls
single-flight       ->  1 backend call

With a naive cache, all fifty requests miss at the same moment and all fifty go to the backend. This is a cache stampede (or thundering herd). Single-flight (request coalescing) fixes it: the first request starts the load and stores its promise; the rest await the same promise.

const inflight = new Map<string, Promise<unknown>>()

function singleFlight<T>(key: string, load: () => Promise<T>): Promise<T> {
  const running = inflight.get(key)
  if (running) return running as Promise<T>
  const p = load().finally(() => inflight.delete(key))
  inflight.set(key, p)
  return p
}

Remember this one works within a process. With several instances, each coalesces its own requests, so the backend sees up to one call per instance, which is still far better than one per request. Coordinating across instances needs a shared lock or a request-coalescing layer.

I also tested the same collision at expiry: an entry that had just expired, 30 concurrent requests, naive cache: 30 backend calls.

Serve stale while refreshing

Coalescing stops the duplicate work but not the wait: every caller still waits for the one load. Stale-while-revalidate, the same idea as the HTTP header, removes the wait. After expiry, return the old value immediately and refresh in the background:

first request after expiry:
  single-flight           -> 201 ms  (waited for the load)
  stale-while-revalidate  ->   0 ms  (served stale, refreshed in the background)
next request                ->   0 ms  (fresh again)

A related pattern, stale-if-error, serves the last known value when the backend fails. I tested a cache that falls back to the expired entry when the loader throws, and it returned the old value instead of an error. Whether that is right depends on the data: it is a good trade for a product catalogue and a bad one for a balance.

Spread out expiry with jitter

If you fill a cache with many keys at once and give them all the same TTL, they all expire together, and the refill load arrives as one spike. Adding random jitter to the TTL spreads it. In my simulation, 1,000 keys cached with a fixed 60-second TTL had all 1,000 expiring in the same 100 ms bucket. With the TTL randomized by ±10%, the busiest bucket held 16 (the exact figure varies per run). A simple helper:

const ttlWithJitter = (seconds: number, spread = 0.1) =>
  Math.round(seconds * (1 - spread + Math.random() * 2 * spread))

Negative caching

A lookup for something that does not exist is also expensive if you repeat it. If your cache only stores found values, a missing id goes to the database every time. I ran 40 sequential lookups for a missing id:

null results not cached  -> 40 database calls
null results cached      ->  1 database call

Cache the "not found" result too, but give it a short TTL, because the record may be created a moment later, and use a representation that distinguishes "cached as missing" from "not in the cache" (in my test, Map.has() rather than checking the value). This also blunts a simple attack: requesting random nonexistent ids to bypass the cache and hammer the database.

Invalidation: the race nobody sees

The usual cache-aside write path is: update the database, then delete the cache key, so the next read repopulates it. It looks right and has a race. I reproduced it with controlled timing (a simulation of the sequence, not a benchmark):

1. Reader misses the cache and reads the database          -> gets the OLD value
2. Writer updates the database to NEW and deletes the key
3. Reader (slow) finishes and writes OLD into the cache
Result: database = NEW, cache = OLD, until the TTL expires

I ran this with the writer deleting the key after the write, and also with the delete before the write. In both orders the cache ended up holding old while the database held new. The stale value then lives until it expires, which for a long TTL can be hours.

Ways to live with this, from simplest to strongest:

• Keep a TTL on everything, even cached data you also invalidate. The TTL is the backstop that bounds how long a bad entry survives.
• Version the data. Give each row a version number; when the reader fills the cache, only accept the value if the version is not older than the newest version the writer announced. I tested this: with the writer raising the minimum accepted version, the slow reader's stale write was rejected and the cache stayed empty, so the next read loaded the new value.
• Use versioned keys for whole-object changes. Put a version or timestamp in the key (product:v2:42:rev9) and let old keys age out, so a change never overwrites an entry in place.
• Prefer short TTLs for data that is hard to invalidate, and cache-busting URLs (a content hash in the filename, served with max-age and immutable) for static assets.

There is no setting that makes invalidation free. The practical goal is to know how stale an entry can be, and to make that bound short enough that the business is comfortable with it.

Designing keys

The key decides who gets which value, so it needs the same care as a permission check.

• Include everything that changes the response. Tenant, user (or permission scope), language, API version, and normalized query parameters.
• Make the tenant mistake impossible, not merely unlikely. I wrote the failure on purpose: a handler that cached by path only. Tenant A requested /invoices/latest and the response was stored; tenant B requested the same path and received {"plan":"A-secret-invoice"}. Using tenant:path as the key returned B's own data. This is the cache version of Part 5's "wrong tenant" bug, and it is easy to introduce because the code works perfectly in a single-tenant test.
• Cache data, then authorize. If a response depends on what the caller may see (Part 5), either key the cache by the caller's permission scope or cache the underlying data and apply field and row filtering per request. Do not cache the output of an authorization decision under a key that ignores who asked.
• Namespace and version keys. A prefix such as product:v2: lets you change the stored shape in a deploy without reading old entries as the new format.
• Keep keys bounded and non-secret. Unbounded keys (raw user input) grow the cache and can be abused, and keys often show up in logs and monitoring.
• Keep cardinality in check for metrics, as Part 7 explained: do not use the cache key as a metric label.

Measuring a cache

A cache that is not measured is assumed to work. Track:

• Hit ratio per cache and per key family, not only overall. Redis gives you the server-wide figure from keyspace_hits and keyspace_misses; for in-process caches, count it yourself.
• Latency of hits versus misses. If misses are not much slower than hits, the cache is not earning its keep.
• Evictions and expirations. A high eviction rate means the cache is too small or the policy is wrong; a high expiry rate means TTLs may be too short.
• Backend load behind the cache, with a close eye on its peaks when a cache is cleared or restarted.
• Staleness incidents: how often users see data that is older than your stated bound, which usually shows up first as a support ticket, so give yourself a way to see it earlier.

Debugging checklist

What headers did the response carry, and did a cache serve it?

curl -sI https://example.com/api/products | grep -i -E 'cache-control|etag|age|vary|expires|last-modified|set-cookie'

An Age header means a shared cache served the response and tells you how old it is. Your CDN will usually add its own cache-status header; check its documentation for the exact name and values.

Is it cached when it should not be? Request the same URL as two different users or with a different cookie. If the second response contains the first user's data, stop and fix it before anything else.

Does the revalidation work?

curl -sI https://example.com/app.js | grep -i etag
curl -s -o /dev/null -w '%{http_code}\n' https://example.com/app.js -H 'If-None-Match: "paste-the-etag"'
# 304 means the validator works

In Redis: INFO stats for hits, misses, evictions and expirations, and TTL key to see how long an entry has left (and to find keys without one).

Production checklist:

• A stated staleness bound for every cached thing.
• Explicit Cache-Control on every response; no-store or private on personalized ones; no reliance on heuristic freshness.
• CDN rules that do not override the origin's headers; no caching of responses with Set-Cookie unless intended.
• Keys include tenant, permission scope, locale and version; no raw secrets.
• A TTL on every entry, with jitter; negative results cached briefly.
• Single-flight on hot keys; stale-while-revalidate where the data tolerates it.
• Invalidation that accounts for the read-repopulate race (version check or short TTL backstop).
• Size bounds and an eviction policy; a plan for the cache being down.
• Hit ratio, evictions and backend load on a dashboard (Part 7).

Interview answer vs production answer

The interview answer: "Cache frequently read data in Redis with a TTL, and invalidate it when the data changes."

The production answer adds the parts that decide whether it is safe:

• Where you cache is a freshness and a security decision. A shared cache needs the caller in the key; a CDN needs explicit headers; a browser cache is private.
• The absence of headers is not "no caching." Browsers estimate a lifetime from Last-Modified.
• Expiry is the dangerous moment. Stampedes turn a cache miss into a database outage unless you coalesce, serve stale, and jitter.
• "Delete after write" has a race. Bound staleness with a TTL, and version cache fills if it matters.
• Authorization is not cacheable by default. Cache data; decide per caller.
• Know your hit ratio. A cache with a low one costs more than it saves.

FAQ

What is the difference between no-cache and no-store?

no-cache means "you may store it, but check with the server before reusing it." no-store means "do not store it." My test showed no-cache with an ETag still sending a request each time (answered by a cheap 304), while no-store sent a full request each time.

Why is my browser not asking the server for a file I did not set headers for?

Probably heuristic freshness. With no Cache-Control but a Last-Modified date, the browser may reuse the response for a period it estimates itself. In my test an old Last-Modified alone was enough to make the browser reuse the response without a second request.

Should I cache in memory or in Redis?

In memory for small, hot, tolerant-of-inconsistency data; Redis when instances must agree or the data is large. Many systems use both: a short in-memory layer in front of Redis.

How long should the TTL be?

Set it from the staleness the business accepts, not from habit. Long TTLs raise the hit ratio and the staleness; short TTLs do the opposite. Add jitter, and use stale-while-revalidate where users should not wait.

What is a cache stampede?

Many requests miss the same key at once, often at expiry, and all go to the backend. In my test, 50 concurrent requests caused 50 backend calls without coalescing and 1 with it.

Can I cache authenticated API responses at a CDN?

Only with great care. Mark per-user responses private or no-store. If you deliberately cache them, the key must include everything that decides what the caller may see, and you should test with two different users.

Is Vary: Cookie or Vary: Authorization a good way to cache per user?

It makes a cache keep separate copies per value, but it fragments the cache to one entry per user and is easy to get subtly wrong. For personalized responses, private or no-store is usually the safer choice (Part 2 makes the same point).

What happens if Redis goes down?

That depends on what you built. Without a fallback, requests fail. With one, load shifts to the database, which may not survive the full traffic. Decide in advance, use short timeouts on cache calls, and test it.

Glossary

The end-to-end mental model

 request
    |
    v
 browser cache ---- fresh? ------------------------------------------> served (private to this user)
    | miss / stale (revalidate or stale-while-revalidate)
    v
 CDN / shared proxy ---- key = URL (+ Vary) ---- fresh? ------------> served (shared: never store personalized data)
    | miss
    v
 app in-memory cache (LRU, per instance) ---- hit? -----------------> served
    | miss
    v
 shared cache (Redis)  key = tenant : scope : locale : version : id
    | miss  --> single-flight: ONE load per key
    v
 database (truth) --> fill caches with TTL + jitter; cache "not found" briefly
                      on write: update DB, delete/version the key, keep a TTL as the backstop

 measure: hit ratio, miss latency, evictions, backend load, staleness

Part 8 made the database correct, and this part made it fast to reach. One post remains, and it covers how all of this gets to production and stays there: deployment and DevOps, from CI/CD and containers to a VPS and Cloudflare Workers.