--- title: "Caching Strategies for Production: CDN, HTTP Cache, Redis, and In-Memory" description: "Where to cache a backend response, how long, and how to keep it correct: HTTP and CDN caches, Redis, in-memory caches, stampedes, and invalidation." canonical: "https://www.suriyaprakash.in/blog/engineering/caching-strategies-cdn-http-redis-memory" category: "Engineering" date: "2026-09-26" updated: "2026-09-26" series: "Production Backend Systems" tags: - caching - http-caching - cdn - redis - performance - cache-invalidation - backend --- # Caching Strategies for Production: CDN, HTTP Cache, Redis, and In-Memory Canonical: https://www.suriyaprakash.in/blog/engineering/caching-strategies-cdn-http-redis-memory Part 8 ended on the database: every query costs a connection, a lock, and some latency, and most of that cost is paid again each time two requests ask for the same thing. Caching is the obvious answer, and the one with the most ways to go wrong. A cache is a copy of an answer kept closer to whoever is asking. The speed-up is real. So is the bill: the copy can be wrong, someone has to decide when it stops being trusted, and a copy kept in the wrong place can be shown to the wrong person. Most production caching bugs are not performance bugs. They are correctness and security bugs that look like performance work. Part 2 already covered the HTTP caching headers (`Cache-Control`, `ETag`, `Vary`). This post does not repeat the reference. It looks at the whole stack of places a response can be cached, how they differ, what happens when a hot entry expires, and how to invalidate without serving stale data forever. As in the previous posts, I tested what I could: browser caching in a real browser, and the in-process cache behaviors in Node. The Redis and CDN details come from their official documentation, and I mark them that way, because I did not run those systems. ## Quick answer - **Cache in layers, closest to the user first.** Browser, then CDN, then your application's memory, then a shared store such as Redis, then the database. Each layer trades freshness for speed. - **Decide the staleness you can tolerate before you choose the layer.** "Up to 30 seconds old" and "must be current" are different systems. - **The cache key is a security boundary.** Anything that changes the response (the tenant, the user's permissions, the language) must be part of the key, or one person will receive another's data. - **Expiry is when caches hurt you.** A hot entry that expires can send every waiting request to the database at once. Coalesce those requests, serve stale data while refreshing, and spread out expiry times. - **Invalidation has a race.** Deleting a key after a write is not enough on its own. A slow reader can write the old value back. - **Cache the data, not the permission decision.** Part 5's authorization has to run for every caller. - **Measure the hit ratio.** A cache you do not measure is a cache you assume works. ```txt browser cache --> CDN / shared proxy --> app memory --> Redis (shared) --> database (one user) (many users, by URL) (one instance) (all instances) (the truth) fastest ......................................................................... slowest most stale ..................................................................... freshest ``` ## When not to cache Caching is not free, so start by asking whether you need it. - **Does it fix a measured problem?** If a page is slow because a query is missing an index (Part 8), caching hides the problem and adds a second one. Fix the query first. - **Is the data cheap to produce?** A cached value that takes 2 ms to recompute saves nothing and costs a network hop. - **Is it personalized or changing every second?** A low hit ratio means you pay for the cache and still hit the database. - **Would a stale answer be harmful?** Account balances, stock reservations and permission checks usually should not come from a cache that can lag. Good candidates are read-heavy, expensive to compute, shared across many requests, and tolerant of some delay: public pages, configuration, reference data, expensive aggregates, and responses from slow third-party APIs. ## The HTTP layer: what the browser really does Browsers and CDNs follow `Cache-Control`, so this layer is configured with response headers. I ran a small server that counted how many times each URL actually reached it, and a page that fetched each URL twice using the default cache behavior. The browser was a Chromium-based one; behavior can differ between browsers. | Response headers | Server requests for 2 fetches | |---|---| | `Cache-Control: max-age=60` | **1** | | `Cache-Control: private, max-age=60` | **1** | | `Cache-Control: max-age=60, immutable` | **1** | | `Cache-Control: no-cache` with an `ETag` | 2 (the second was a conditional request answered with `304`) | | `Cache-Control: no-store` | 2 | | No cache headers at all | 2 | | **No `Cache-Control`, only `Last-Modified` (10 days old)** | **1** | The last row is a trap worth knowing. The response had no `Cache-Control` header at all, only a `Last-Modified` date ten days in the past, and the browser still reused it without asking the server. That is **heuristic freshness**: when a response has no explicit lifetime, caches are allowed to estimate one from how old the content is. If you want a response revalidated or not stored, say so explicitly; never rely on the absence of headers. Two more results from the same test: - **`Vary` is honored.** With `Vary: Accept-Language`, fetches with `en`, `fr`, `fr` caused 2 server requests: the second `fr` was served from the cache entry for `fr`, and the first `fr` was a miss even though an `en` entry existed. Without a correct `Vary`, a cache can serve one language to everyone. - **A request can force revalidation.** `fetch(url, { cache: 'no-cache' })` on a URL with `max-age=60` went back to the server (2 requests total for that URL). That is forced revalidation; a hard reload goes further and bypasses the cache. ### Stale-while-revalidate RFC 5861 defines two extensions that matter for availability. With `stale-while-revalidate=N`, a cache may serve a stale response for up to `N` seconds after it expires, while it revalidates in the background. With `stale-if-error=N`, a cache may serve a stale response when the origin returns a 500, 502, 503 or 504. The RFC's example is `Cache-Control: max-age=600, stale-while-revalidate=30`: fresh for 600 seconds, then served stale for up to 30 more seconds while a revalidation runs. I tested the first in the browser against an origin that takes 300 ms to respond, with `max-age=1, stale-while-revalidate=10`: ```txt first fetch 315 ms (cold, origin responds) fetch after expiry, inside the SWR window 8 ms (served stale instantly) server requests afterwards 2 (the initial one + a background revalidation) ``` The user did not wait for the origin, and the cache refreshed itself quietly. This is one of the highest-value, lowest-effort headers for public content, and the same idea reappears below for application caches. ### Private and shared caches A browser's cache belongs to one user. A CDN or proxy cache is shared by everyone, so the rules for what it may store are stricter. `private` tells shared caches not to store a response, while still letting the user's own browser keep it (my `private, max-age=60` test above was reused by the browser). Cloudflare's documentation, as an example of CDN behavior, says it does not cache a response when `Cache-Control` is `private`, `no-store`, `no-cache` or `max-age=0`, and its list of non-cached cases also includes responses with a `Set-Cookie` header and requests whose method is not `GET`. It also says Cloudflare decides what to cache by file extension rather than content type, so HTML and JSON are not cached by default; caching them takes an explicit rule. Check your own CDN's documentation, because defaults differ, and test with real responses. ## CDN caching and the danger of a shared cache A CDN stores responses and serves them to many users from a location near them. For static assets that is pure benefit. For anything dynamic it is where cache bugs become security incidents. **Web cache deception** is the best-known example. PortSwigger describes it as tricking a web cache into storing sensitive, dynamic content, through a discrepancy between how the cache and the origin interpret a URL. Their example: a request for `/user/123/profile/wcd.css`. The origin may ignore the trailing `wcd.css` and return the profile page, while the cache sees a path ending in `.css`, treats it as a static file, and stores the response. The attacker then requests the same URL and receives the victim's cached page. The recommended defenses are the ones you would hope for: - Mark dynamic, user-specific responses with `Cache-Control: no-store` and `private`. - Configure the CDN so its own rules do not override the origin's `Cache-Control`. - Use any CDN protection that checks that the response content type matches the file extension in the URL. - Make sure the origin and cache interpret URL paths the same way. This is a direct consequence of earlier parts: a response that depends on who is asking (Part 5) must never be stored under a key that does not include who is asking. ## In-process caches: fast, local, and forgetful The fastest cache is a variable in your application's memory: no network, no serialization. It is a good fit for small, hot, read-mostly data: configuration, feature flags, a compiled permission set, reference tables. It has four limits: 1. **Each instance has its own copy.** With three instances, you have three caches that expire at different times, so two requests can see different values. Acceptable for configuration, not for anything users compare. 2. **It disappears on restart or deploy.** Every deploy starts cold, which matters for the stampede problem below. 3. **It needs a size bound,** or it will grow until the process is killed. An LRU (least recently used) policy evicts the entry that has gone unused longest. In JavaScript, a `Map` iterates in insertion order, which makes a compact LRU possible: re-insert on read, delete the first key when over capacity. I verified the behavior: with capacity 3, after setting `a`, `b`, `c`, reading `a`, then setting `d`, the cache kept `c`, `a`, `d`; `b` was evicted. 4. **Do not assume it persists.** In serverless and edge runtimes, an instance may be short-lived or run in many places at once, so treat in-memory state there as an optimization, never as storage. ## A shared cache: Redis A shared store such as Redis sits between your instances and the database, so every instance sees the same cached value. The standard way to use it is **cache-aside**: the application checks the cache, and on a miss loads from the database and fills the cache. ```ts async function getProduct(id: string) { const key = `product:v2:${id}` // namespaced and versioned (see "Designing keys") const cached = await redis.get(key) if (cached) return JSON.parse(cached) const row = await db.query.products.findFirst({ where: eq(products.id, id) }) if (row) await redis.set(key, JSON.stringify(row), { EX: 300 }) // expire in 300 s return row } ``` The following details come from Redis's own documentation, not from my testing: - **`SET` takes expiry and condition options.** `EX seconds` sets a time to live, `NX` sets the key only if it does not exist, and `XX` only if it does. Overwriting a key with `SET` discards its previous TTL unless you pass `KEEPTTL`. Forgetting this is a classic way to create entries that never expire. - **Memory is bounded by `maxmemory`, and an eviction policy decides what goes.** On 64-bit systems the default is no limit. The policies include `noeviction` (refuse writes when full), `allkeys-lru`, `allkeys-lfu` (least frequently used), `allkeys-random`, and `volatile-*` variants that only evict keys that have a TTL, plus `volatile-ttl`. The docs suggest `allkeys-lru` as a good default when a subset of keys is accessed far more often than the rest. They also warn that `volatile-*` policies behave like `noeviction` when no keys have an expiry, which means a cache full of keys without TTLs stops accepting writes. - **Measure the hit ratio from the server.** `INFO stats` reports `keyspace_hits` and `keyspace_misses`; the docs give the hit percentage as `keyspace_hits / (keyspace_hits + keyspace_misses) * 100`, and suggest checking `evicted_keys` and `expired_keys` when it is lower than expected. - **Locks with `SET key value NX EX`** are described in the docs as a simple pattern, but they now discourage it in favor of the Redlock algorithm, which offers better guarantees. If you are tempted to use a cache as a lock, read that page first. Operationally, a shared cache adds a network hop to every request and a new way to fail. Decide what happens when it is unreachable: fall back to the database, and make sure the database can survive the full load, because the cache was probably the reason it could. A short timeout on cache calls, and a limit on how many requests may fall through, prevent a cache outage from becoming a database outage. ## The expiry problem: stampedes, stale data and jitter Everything above is the easy part. The hard part is the moment an entry is missing: it expired, was evicted, or the instance just started. I tested the behaviors in Node with a backend that takes 200 ms. ### A cold key under concurrency Fifty concurrent requests for the same missing key: ```txt naive cache-aside -> 50 backend calls single-flight -> 1 backend call ``` With a naive cache, all fifty requests miss at the same moment and all fifty go to the backend. This is a **cache stampede** (or thundering herd). **Single-flight** (request coalescing) fixes it: the first request starts the load and stores its promise; the rest await the same promise. ```ts const inflight = new Map>() function singleFlight(key: string, load: () => Promise): Promise { const running = inflight.get(key) if (running) return running as Promise const p = load().finally(() => inflight.delete(key)) inflight.set(key, p) return p } ``` Remember this one works within a process. With several instances, each coalesces its own requests, so the backend sees up to one call per instance, which is still far better than one per request. Coordinating across instances needs a shared lock or a request-coalescing layer. I also tested the same collision at expiry: an entry that had just expired, 30 concurrent requests, naive cache: **30 backend calls**. ### Serve stale while refreshing Coalescing stops the duplicate work but not the wait: every caller still waits for the one load. Stale-while-revalidate, the same idea as the HTTP header, removes the wait. After expiry, return the old value immediately and refresh in the background: ```txt first request after expiry: single-flight -> 201 ms (waited for the load) stale-while-revalidate -> 0 ms (served stale, refreshed in the background) next request -> 0 ms (fresh again) ``` A related pattern, **stale-if-error**, serves the last known value when the backend fails. I tested a cache that falls back to the expired entry when the loader throws, and it returned the old value instead of an error. Whether that is right depends on the data: it is a good trade for a product catalogue and a bad one for a balance. ### Spread out expiry with jitter If you fill a cache with many keys at once and give them all the same TTL, they all expire together, and the refill load arrives as one spike. Adding random jitter to the TTL spreads it. In my simulation, 1,000 keys cached with a fixed 60-second TTL had all 1,000 expiring in the same 100 ms bucket. With the TTL randomized by ±10%, the busiest bucket held 16 (the exact figure varies per run). A simple helper: ```ts const ttlWithJitter = (seconds: number, spread = 0.1) => Math.round(seconds * (1 - spread + Math.random() * 2 * spread)) ``` ### Negative caching A lookup for something that does not exist is also expensive if you repeat it. If your cache only stores found values, a missing id goes to the database every time. I ran 40 sequential lookups for a missing id: ```txt null results not cached -> 40 database calls null results cached -> 1 database call ``` Cache the "not found" result too, but give it a short TTL, because the record may be created a moment later, and use a representation that distinguishes "cached as missing" from "not in the cache" (in my test, `Map.has()` rather than checking the value). This also blunts a simple attack: requesting random nonexistent ids to bypass the cache and hammer the database. ## Invalidation: the race nobody sees The usual cache-aside write path is: update the database, then delete the cache key, so the next read repopulates it. It looks right and has a race. I reproduced it with controlled timing (a simulation of the sequence, not a benchmark): ```txt 1. Reader misses the cache and reads the database -> gets the OLD value 2. Writer updates the database to NEW and deletes the key 3. Reader (slow) finishes and writes OLD into the cache Result: database = NEW, cache = OLD, until the TTL expires ``` I ran this with the writer deleting the key after the write, and also with the delete before the write. In both orders the cache ended up holding `old` while the database held `new`. The stale value then lives until it expires, which for a long TTL can be hours. Ways to live with this, from simplest to strongest: - **Keep a TTL on everything,** even cached data you also invalidate. The TTL is the backstop that bounds how long a bad entry survives. - **Version the data.** Give each row a version number; when the reader fills the cache, only accept the value if the version is not older than the newest version the writer announced. I tested this: with the writer raising the minimum accepted version, the slow reader's stale write was rejected and the cache stayed empty, so the next read loaded the new value. - **Use versioned keys for whole-object changes.** Put a version or timestamp in the key (`product:v2:42:rev9`) and let old keys age out, so a change never overwrites an entry in place. - **Prefer short TTLs for data that is hard to invalidate,** and cache-busting URLs (a content hash in the filename, served with `max-age` and `immutable`) for static assets. There is no setting that makes invalidation free. The practical goal is to know how stale an entry can be, and to make that bound short enough that the business is comfortable with it. ## Designing keys The key decides who gets which value, so it needs the same care as a permission check. - **Include everything that changes the response.** Tenant, user (or permission scope), language, API version, and normalized query parameters. - **Make the tenant mistake impossible, not merely unlikely.** I wrote the failure on purpose: a handler that cached by path only. Tenant A requested `/invoices/latest` and the response was stored; tenant B requested the same path and received `{"plan":"A-secret-invoice"}`. Using `tenant:path` as the key returned B's own data. This is the cache version of Part 5's "wrong tenant" bug, and it is easy to introduce because the code works perfectly in a single-tenant test. - **Cache data, then authorize.** If a response depends on what the caller may see (Part 5), either key the cache by the caller's permission scope or cache the underlying data and apply field and row filtering per request. Do not cache the output of an authorization decision under a key that ignores who asked. - **Namespace and version keys.** A prefix such as `product:v2:` lets you change the stored shape in a deploy without reading old entries as the new format. - **Keep keys bounded and non-secret.** Unbounded keys (raw user input) grow the cache and can be abused, and keys often show up in logs and monitoring. - **Keep cardinality in check for metrics,** as Part 7 explained: do not use the cache key as a metric label. ## Measuring a cache A cache that is not measured is assumed to work. Track: - **Hit ratio** per cache and per key family, not only overall. Redis gives you the server-wide figure from `keyspace_hits` and `keyspace_misses`; for in-process caches, count it yourself. - **Latency of hits versus misses.** If misses are not much slower than hits, the cache is not earning its keep. - **Evictions and expirations.** A high eviction rate means the cache is too small or the policy is wrong; a high expiry rate means TTLs may be too short. - **Backend load behind the cache,** with a close eye on its peaks when a cache is cleared or restarted. - **Staleness incidents:** how often users see data that is older than your stated bound, which usually shows up first as a support ticket, so give yourself a way to see it earlier. ## Debugging checklist **What headers did the response carry, and did a cache serve it?** ```bash curl -sI https://example.com/api/products | grep -i -E 'cache-control|etag|age|vary|expires|last-modified|set-cookie' ``` An `Age` header means a shared cache served the response and tells you how old it is. Your CDN will usually add its own cache-status header; check its documentation for the exact name and values. **Is it cached when it should not be?** Request the same URL as two different users or with a different cookie. If the second response contains the first user's data, stop and fix it before anything else. **Does the revalidation work?** ```bash curl -sI https://example.com/app.js | grep -i etag curl -s -o /dev/null -w '%{http_code}\n' https://example.com/app.js -H 'If-None-Match: "paste-the-etag"' # 304 means the validator works ``` **In Redis:** `INFO stats` for hits, misses, evictions and expirations, and `TTL key` to see how long an entry has left (and to find keys without one). **Production checklist:** - A stated staleness bound for every cached thing. - Explicit `Cache-Control` on every response; `no-store` or `private` on personalized ones; no reliance on heuristic freshness. - CDN rules that do not override the origin's headers; no caching of responses with `Set-Cookie` unless intended. - Keys include tenant, permission scope, locale and version; no raw secrets. - A TTL on every entry, with jitter; negative results cached briefly. - Single-flight on hot keys; stale-while-revalidate where the data tolerates it. - Invalidation that accounts for the read-repopulate race (version check or short TTL backstop). - Size bounds and an eviction policy; a plan for the cache being down. - Hit ratio, evictions and backend load on a dashboard (Part 7). ## Interview answer vs production answer The interview answer: *"Cache frequently read data in Redis with a TTL, and invalidate it when the data changes."* The production answer adds the parts that decide whether it is safe: - **Where you cache is a freshness and a security decision.** A shared cache needs the caller in the key; a CDN needs explicit headers; a browser cache is private. - **The absence of headers is not "no caching."** Browsers estimate a lifetime from `Last-Modified`. - **Expiry is the dangerous moment.** Stampedes turn a cache miss into a database outage unless you coalesce, serve stale, and jitter. - **"Delete after write" has a race.** Bound staleness with a TTL, and version cache fills if it matters. - **Authorization is not cacheable by default.** Cache data; decide per caller. - **Know your hit ratio.** A cache with a low one costs more than it saves. ## FAQ ### What is the difference between `no-cache` and `no-store`? `no-cache` means "you may store it, but check with the server before reusing it." `no-store` means "do not store it." My test showed `no-cache` with an `ETag` still sending a request each time (answered by a cheap `304`), while `no-store` sent a full request each time. ### Why is my browser not asking the server for a file I did not set headers for? Probably heuristic freshness. With no `Cache-Control` but a `Last-Modified` date, the browser may reuse the response for a period it estimates itself. In my test an old `Last-Modified` alone was enough to make the browser reuse the response without a second request. ### Should I cache in memory or in Redis? In memory for small, hot, tolerant-of-inconsistency data; Redis when instances must agree or the data is large. Many systems use both: a short in-memory layer in front of Redis. ### How long should the TTL be? Set it from the staleness the business accepts, not from habit. Long TTLs raise the hit ratio and the staleness; short TTLs do the opposite. Add jitter, and use stale-while-revalidate where users should not wait. ### What is a cache stampede? Many requests miss the same key at once, often at expiry, and all go to the backend. In my test, 50 concurrent requests caused 50 backend calls without coalescing and 1 with it. ### Can I cache authenticated API responses at a CDN? Only with great care. Mark per-user responses `private` or `no-store`. If you deliberately cache them, the key must include everything that decides what the caller may see, and you should test with two different users. ### Is `Vary: Cookie` or `Vary: Authorization` a good way to cache per user? It makes a cache keep separate copies per value, but it fragments the cache to one entry per user and is easy to get subtly wrong. For personalized responses, `private` or `no-store` is usually the safer choice (Part 2 makes the same point). ### What happens if Redis goes down? That depends on what you built. Without a fallback, requests fail. With one, load shifts to the database, which may not survive the full traffic. Decide in advance, use short timeouts on cache calls, and test it. ## Glossary | Term | Simple meaning | |---|---| | Cache hit / miss | The value was / was not found in the cache | | TTL | Time to live: how long an entry is kept | | Cache-aside | The app checks the cache, loads on a miss, and fills the cache | | Stampede | Many simultaneous misses on one key hitting the backend at once | | Single-flight | Letting one request load a missing key while others wait for its result | | Stale-while-revalidate | Serving an expired value while refreshing it in the background | | Stale-if-error | Serving an expired value when the origin fails | | Heuristic freshness | A cache estimating a lifetime when no explicit one is given | | Jitter | Random variation added to TTLs to avoid synchronized expiry | | Negative caching | Caching "not found" results | | LRU / LFU | Evict the least recently / least frequently used entry | | Private / shared cache | One user's cache (browser) / a cache used by many users (CDN) | | Web cache deception | Tricking a shared cache into storing someone's private page | | Hit ratio | Hits divided by hits plus misses | ## The end-to-end mental model ```txt request | v browser cache ---- fresh? ------------------------------------------> served (private to this user) | miss / stale (revalidate or stale-while-revalidate) v CDN / shared proxy ---- key = URL (+ Vary) ---- fresh? ------------> served (shared: never store personalized data) | miss v app in-memory cache (LRU, per instance) ---- hit? -----------------> served | miss v shared cache (Redis) key = tenant : scope : locale : version : id | miss --> single-flight: ONE load per key v database (truth) --> fill caches with TTL + jitter; cache "not found" briefly on write: update DB, delete/version the key, keep a TTL as the backstop measure: hit ratio, miss latency, evictions, backend load, staleness ``` Part 8 made the database correct, and this part made it fast to reach. One post remains, and it covers how all of this gets to production and stays there: deployment and DevOps, from CI/CD and containers to a VPS and Cloudflare Workers.