Caching Strategies: Redis, CDN, and Application-Level Caching

A few years back, a service we owned went from 40ms p99 to timing out entirely, in about ninety seconds, with zero deploys and zero traffic spikes. The cause: a Redis node failed over, the cache emptied, and every one of our ~600 application pods went straight to the primary database at once. The database, which had been happily serving cache misses at a trickle all week, saw fifteen thousand simultaneous queries for the same twenty "hot" product IDs and fell over. The incident had nothing to do with load. It had everything to do with the fact that we treated caching as a performance optimization instead of what it actually is: a second source of truth that can drift, stampede, and fail in ways your primary datastore never will.

That's really what this post is about. Not "add a cache to make things faster" — that part is easy. The hard part is deciding where each cache layer lives, what invalidates it, and what happens to your system in the ten seconds after it goes cold.

The Layers, and What Each One Actually Protects

People talk about "caching" as if it's one thing. In a real request path there are usually four distinct layers, and conflating them is where most of the confusing incidents come from.

Browser cache. Controlled by Cache-Control, ETag, and Expires headers, this layer protects your network entirely — a cache hit here never leaves the user's machine. It's the cheapest cache you'll ever operate because you operate none of it; the browser does the work. The catch is you have zero visibility into hit rates and no way to invalidate it early except by changing the URL (cache-busting via content hashes in filenames is the standard trick for static assets).

CDN / edge cache. This sits between the user and your origin — Cloudflare, Fastly, CloudFront, Akamai. It protects your origin infrastructure from geographic latency and from raw request volume. A well-configured CDN can absorb 95%+ of traffic for content that doesn't change per-user: images, JS bundles, marketing pages, and — increasingly — API responses for public, non-personalized data. Netflix's edge architecture is the canonical example here: they push cache nodes (Open Connect) as close to ISPs as physically possible because at their scale, shaving milliseconds off round-trip time for video segment requests matters more than almost anything else in the stack. Cloudflare's whole business, similarly, is built on the premise that most read traffic on the internet is cacheable and most origins are overprovisioned to compensate for not caching it.

Application-level cache. This is where most of the interesting engineering happens, and where the rest of this post lives. It's either in-process (Caffeine, Guava, a plain ConcurrentHashMap with eviction) or distributed (Redis, Memcached). It protects your database and downstream services from repeated computation and repeated I/O for the same logical query.

Database-level cache. Your database already caches for you — buffer pools, query plan caches, materialized views. This is the layer people forget exists, which is why "just add Redis in front of Postgres" sometimes barely moves the needle: Postgres was already serving that query from shared_buffers in under a millisecond, and now you've added a network hop and a second system to keep consistent for no real gain.

The mental model that's saved me the most grief: each layer should protect the layer behind it from a specific kind of load, not just "load" in the abstract. CDN protects origin bandwidth. App cache protects database query volume. DB cache protects disk I/O. If you can't articulate which layer a given cache is protecting and from what, you probably don't need it yet.

Invalidation: The Part Everyone Underestimates

There's an old joke about the two hard problems in computer science being cache invalidation, naming things, and off-by-one errors. It's a joke because it's true, and it's true because invalidation is where correctness and performance stop being separate concerns.

TTL (time-to-live) is the default and, honestly, the right starting point for most caches. You set an expiry — five seconds, five minutes, an hour — and accept that data can be stale for up to that window. The appeal is that it's operationally trivial: no invalidation logic to write, no events to wire up, no risk of a stale entry living forever because someone forgot to fire an invalidation event. The cost is that you're explicitly choosing a staleness window, and you need to be honest with your product owners about what that window means. A five-minute TTL on a product price is a five-minute window where a customer can see a price that no longer exists.

Event-driven invalidation ties cache eviction to the write that made the data stale — a ProductPriceChanged event fires, a consumer picks it up, and it evicts (or updates) the corresponding cache key. This gets you much tighter consistency, but it adds a dependency: your cache correctness now depends on every writer reliably publishing invalidation events, and on your event pipeline never silently dropping messages. In practice, teams that do this well combine it with a TTL as a safety net — event-driven for the common case, TTL as the "eventually this heals itself even if the event never arrived" backstop. Airbnb's caching layer for listing data works roughly this way: search results and listing details use fairly aggressive TTLs, but price and availability changes fire explicit invalidation because guests seeing a stale "available" date is a direct revenue and trust problem, not a cosmetic one.

Separately from when you invalidate, there's how you keep the cache and database in sync on writes, and this is really three different patterns:

• Cache-aside (lazy loading). The application checks the cache; on a miss, it reads from the database and populates the cache. Writes go to the database, and the cache entry is either invalidated or updated afterward. This is the most common pattern because the cache is a pure derived view — if it disappears entirely, the system keeps working, just slower. It's what we used at the API layer in the incident I opened with.

• Write-through. Writes go to the cache first, and the cache synchronously writes through to the database before acknowledging. This keeps cache and database always in sync, at the cost of write latency, since every write pays for both operations before returning.

• Write-behind (write-back). Writes land in the cache and are acknowledged immediately; the cache asynchronously flushes to the database in the background. This gives you the best write latency, but it means there's a window where data exists only in the cache — if the cache dies before flushing, you lose writes. This pattern shows up in high-throughput counters and analytics pipelines where losing a small percentage of writes during a rare failure is an acceptable trade for consistently low latency, but it's rarely the right choice for anything resembling a financial transaction.

Cache-aside is the right default for the overwhelming majority of read-heavy services. Reach for write-through when correctness matters more than write latency and reads vastly outnumber writes. Reach for write-behind only when you've explicitly decided that some data loss on cache failure is tolerable — and when you do, make sure someone signed off on that trade-off in a design doc, not in a Slack thread after the fact.

The Stampede: When Your Cache Becomes a Denial-of-Service Attack on Yourself

The thundering herd problem is what happened in the incident I described earlier, and it's worth understanding mechanically because the fix is almost always cheaper than the outage.

Here's the sequence: a popular cache key — a trending product, a celebrity's profile, the homepage feed — expires. In the next instant, hundreds or thousands of concurrent requests for that same key all miss the cache simultaneously. Every one of those requests, having found nothing in the cache, goes to the database to recompute the value. The database, which was previously serving that value to the cache and nothing else, now receives N simultaneous identical queries where N is your concurrent request rate for that key. If N is large enough, the database falls over, which makes every other request slower too, which causes more cache misses elsewhere, and you get a cascading failure that looks like a traffic spike but was actually caused by a single key expiring.

There are three complementary defenses, and mature systems use more than one:

Locks / mutex on recompute. The first request to miss the cache acquires a distributed lock (a Redis SETNX with a short TTL works fine) and is the only one allowed to query the database and repopulate the cache. Every other concurrent request for that key either waits briefly and retries the cache, or is served a slightly stale value if one exists. This turns N simultaneous database queries into one.

Probabilistic early expiry. Instead of a hard TTL, you let requests probabilistically decide to refresh the cache before it expires, with the probability increasing as the entry approaches its actual expiry time. The effect is that refreshes get spread out across the minutes leading up to expiry instead of all landing in the same instant. This is the approach described in the well-known "XFetch" algorithm and it's elegant because it needs no locks at all — just a formula applied at read time.

Request coalescing. Structurally similar to locking, but implemented at the application layer rather than via the cache: if five requests for the same uncached key arrive within the same tens-of-milliseconds window, the application recognizes they're identical in-flight requests and lets one database query serve all five callers, rather than firing five queries. Libraries like singleflight in Go formalize this pattern; in Java, you can approximate it with a ConcurrentHashMap<String, CompletableFuture<T>> keyed by cache key, where concurrent callers await the same future instead of issuing separate queries.

Which of these you need depends on how "hot" your hottest keys get. If your traffic is fairly evenly distributed across millions of keys, stampedes are rare and a plain TTL is fine. If you have a Pareto distribution where 1% of keys serve 60% of traffic — which describes almost every consumer product I've worked on — you need at least the lock-based approach before you go to production, not after the first outage.

Redis vs. Local Caching: Two Different Problems Wearing the Same Name

"Should we use Redis or an in-process cache?" is usually the wrong question, because they solve different problems and the right architecture often uses both.

A local (in-process) cache — Caffeine on the JVM is the standard choice today — lives in the application's own heap. Reads are nanoseconds, there's no network hop, no serialization cost, and no external dependency to keep available. The catch is that it's per-instance: with 50 pods behind a load balancer, you have 50 independent copies of the cache, each potentially stale relative to the others, and each cold on startup. Local caching is the right call when data is read extremely frequently, changes rarely, and modest staleness across instances is fine — feature flags, configuration, reference data like currency codes or country lists.

A distributed cache like Redis is shared across every instance of your service. A cache warm in one pod is warm for all of them, invalidation applied once is applied everywhere, and you get a single, consistent view of cached state. The cost is a network round trip (still much faster than a database query, but not free), serialization overhead, and now Redis itself is a dependency your service can't function without — which is exactly what bit us in the stampede incident. Distributed caching is the right call for anything where consistency across instances actually matters, or where the underlying computation is expensive enough that duplicating it 50 times in-process would be wasteful.

Multi-level caching — local cache as L1, Redis as L2, database as L3 — is where you end up when you've outgrown either extreme on its own. The pattern: check the local cache first (nanoseconds); on a miss, check Redis (single-digit milliseconds); on a miss there, hit the database and populate both layers on the way back. This gets you the latency of local caching for the hottest subset of keys while keeping cross-instance consistency for everything else, since Redis acts as the shared source of truth that repopulates each instance's L1 independently.

The complexity tax is real, though: you now have two invalidation paths to keep in sync, two sets of metrics to watch, and a subtler failure mode where a local cache serves data that Redis has already invalidated. We've found multi-level caching worth it exactly once in the last few years — a read path serving on the order of 50,000 requests per second for a small set of extremely hot keys (feature flag evaluations on every request), where even a Redis round-trip at scale was a meaningful chunk of our latency budget. For anything under maybe 5,000 requests per second on a given path, a single well-tuned Redis layer is simpler to operate and almost as fast in practice.

Warm-Up: Don't Let Your Cache Learn on Production Traffic

Every cache starts empty, and if the first thing that touches it is unthrottled production traffic, you've just recreated the stampede problem on every deploy and every failover. Cache warm-up is the practice of populating the cache with known-hot keys before traffic hits it.

The simplest version: on service startup, query your analytics for the top N most-requested keys over the last 24 hours and pre-populate the cache with them before the health check passes and the load balancer starts routing traffic. A more sophisticated version — useful for a full regional failover — replicates the cache state itself (Redis supports this natively via replication, or you can snapshot and restore) so a new region comes online already warm rather than starting from zero.

The trade-off is usually just engineering time: warm-up scripts need to be maintained, and they need their own monitoring, because a warm-up job that silently stops running is worse than not having one — you'll assume protection you don't actually have. But for any service where a cold cache means database overload, warm-up isn't optional polish, it's the thing that makes deploys and failovers boring instead of terrifying.

When a Cache Miss Isn't Really a Miss — It's a Database DDoS

It's worth stating plainly, because it's easy to lose sight of amid all the invalidation and stampede mechanics: a cache is a bet that your database can handle the traffic that doesn't get served from cache. Every design decision above is really about managing what happens when that bet goes wrong — cache expires, cache node dies, deploy flushes the cache, a marketing email drives ten times the normal traffic to a page that was never cache-warmed.

The guardrail that's independent of everything else in this post is a circuit breaker or a hard concurrency limit between your application and your database, so that when the cache fails to protect it, the database degrades instead of collapsing. A database that returns 503s under overload and recovers in thirty seconds is an incident. A database that falls over completely and needs a manual restart is an outage that takes down every other service sharing that database, cached or not. We'll get into this properly in the next post in this series, but the short version: caching reduces average load, it doesn't cap worst-case load, and you need something else in the stack that does.

Coherency Across Instances: The Problem Multi-Level Caching Reintroduces

Once you have more than one instance of your application, and especially once you introduce local caching alongside Redis, you inherit a coherency problem: instance A invalidates a key in Redis after a write, but instance B's local L1 cache still has the old value, and it'll keep serving it until its own TTL expires.

There's no way to make this fully consistent without giving up the latency benefit of local caching in the first place — if you need every instance's local cache to reflect a write instantly, you've reinvented a distributed cache with extra steps. The practical answer is to bound the blast radius: keep local cache TTLs short (seconds, not minutes) for anything where staleness across instances matters, and use pub/sub invalidation (Redis supports this via keyspace notifications, or you can run your own lightweight invalidation channel) to proactively evict local entries across instances rather than waiting out the TTL. This gets you "consistent within a few seconds" rather than "consistent within the local TTL window," which is usually the difference between acceptable and not.

Code: Cache-Aside with Stampede Protection

Here's a fairly representative implementation of cache-aside reads backed by Redis, with a distributed lock to prevent stampedes on recompute. This is deliberately manual rather than annotation-based, because the lock logic doesn't map cleanly onto @Cacheable.

@Service
public class ProductCacheService {

    private final StringRedisTemplate redis;
    private final ProductRepository repository;
    private final ObjectMapper mapper;

    private static final Duration TTL = Duration.ofMinutes(5);
    private static final Duration LOCK_TTL = Duration.ofSeconds(3);

    public ProductCacheService(StringRedisTemplate redis, ProductRepository repository, ObjectMapper mapper) {
        this.redis = redis;
        this.repository = repository;
        this.mapper = mapper;
    }

    public Product getProduct(String productId) {
        String cacheKey = "product:" + productId;
        String cached = redis.opsForValue().get(cacheKey);
        if (cached != null) {
            return deserialize(cached);
        }

        String lockKey = "lock:" + cacheKey;
        boolean acquired = Boolean.TRUE.equals(
            redis.opsForValue().setIfAbsent(lockKey, "1", LOCK_TTL));

        if (!acquired) {
            return waitForCacheEntry(cacheKey);
        }

        try {
            Product product = repository.findById(productId)
                .orElseThrow(() -> new NotFoundException(productId));
            redis.opsForValue().set(cacheKey, serialize(product), TTL);
            return product;
        } finally {
            redis.delete(lockKey);
        }
    }

    private Product waitForCacheEntry(String cacheKey) {
        for (int attempt = 0; attempt < 10; attempt++) {
            try {
                Thread.sleep(50);
            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
                break;
            }
            String value = redis.opsForValue().get(cacheKey);
            if (value != null) {
                return deserialize(value);
            }
        }
        return repository.findById(cacheKey).map(this::asProduct)
            .orElseThrow();
    }

    private String serialize(Product product) {
        try {
            return mapper.writeValueAsString(product);
        } catch (JsonProcessingException e) {
            throw new IllegalStateException(e);
        }
    }

    private Product deserialize(String json) {
        try {
            return mapper.readValue(json, Product.class);
        } catch (JsonProcessingException e) {
            throw new IllegalStateException(e);
        }
    }
}

For simpler cases without stampede risk — reference data, low-traffic lookups — Spring's caching annotations get you most of the way there with far less code:

@Service
public class PricingService {

    private final PricingRepository repository;

    public PricingService(PricingRepository repository) {
        this.repository = repository;
    }

    @Cacheable(value = "pricing", key = "#skuId")
    public Price getPrice(String skuId) {
        return repository.findPriceBySku(skuId);
    }

    @CacheEvict(value = "pricing", key = "#skuId")
    public void invalidatePrice(String skuId) {
        // called from the price-change event consumer
    }

    @CachePut(value = "pricing", key = "#price.skuId")
    public Price updatePrice(Price price) {
        return repository.save(price);
    }
}

@Cacheable handles the cache-aside read path, @CacheEvict handles event-driven invalidation triggered from a message consumer, and @CachePut keeps the cache in sync on writes without invalidating and forcing a recompute. It's not stampede-safe on its own — Spring's cache abstraction doesn't include locking — so we reserve the annotation-based approach for keys where the traffic pattern doesn't make stampedes a realistic risk.

When Staleness Is Fine, and When It's Dangerous

Not all cached data carries the same risk when it's wrong, and being explicit about this early saves a lot of debate later.

Product descriptions, blog content, user avatars, and most reference data can tolerate minutes of staleness without anyone noticing or caring. Cache these aggressively, with long TTLs, and don't lose sleep over it.

Pricing and inventory sit in a different category. A stale price that's too low means you honor a lower price than you meant to, or you don't — and either way, you've created a customer trust problem or a support ticket. Stale inventory that says "in stock" when it isn't means an order you can't fulfill. These need either short TTLs, event-driven invalidation, or — for the final check before an order is placed — reading straight from the source of truth and skipping the cache entirely. The cache is fine for browsing; it's the wrong tool for the last read before money changes hands.

Authentication and authorization data is the most dangerous category to get wrong, because staleness here doesn't just mean an inconvenience, it means a security bug. A cached permission set that says a user still has admin access after it was revoked is a live vulnerability for as long as the cache entry lives. Session and token validation should either use very short TTLs (seconds) or be invalidated synchronously and immediately on revocation — never left to a TTL to "eventually" catch up.

The rule of thumb we've settled on: if being wrong for N seconds costs you a slightly stale product description, N can be measured in minutes. If being wrong for N seconds costs you money or a security boundary, N needs to be measured in single-digit seconds, or you shouldn't be caching that value on that path at all.

Common Mistakes

Caching without a defined invalidation strategy. Someone adds a Redis lookup with a set() call and no TTL, no eviction policy, no plan for how stale data ever leaves. Six months later the cache is one of the largest sources of "why is this data wrong in production" tickets, because nobody can say with confidence when any given key was last refreshed.

Treating the cache as durable storage. We've seen services that write a value to Redis and never persist it anywhere else, because "it's basically a database, right?" Redis eviction policies, restarts, and failovers can and will drop data that isn't backed by a real source of truth. If losing a key would be a real problem, it needs to live somewhere durable, with the cache as a derived, disposable copy.

No stampede protection on hot keys. This is the mistake that opened this post. If you haven't checked your access-pattern distribution and you have any reason to believe a small number of keys account for a large share of your traffic, you have stampede exposure whether or not you've been paged for it yet.

Cache key collisions from insufficiently specific keys. Using "product:" + id for both a public catalog view and an authenticated view with personalized pricing, for instance, will eventually leak one user's discounted price to another user, or serve stale personalized data across sessions. Cache keys need to fully encode everything that makes the cached value distinct — tenant ID, locale, user segment, API version — not just the primary entity ID.

Never load-testing cache failure. Teams load-test their happy path constantly and almost never simulate "what happens if Redis disappears for sixty seconds under full production traffic." That's precisely the scenario that takes services down, and it's the cheapest failure mode to test deliberately, in a staging environment, before it happens to you for the first time in production.

Final Takeaway

Caching is easy to add and easy to get wrong in ways that don't show up until the one day your traffic pattern or your infrastructure doesn't cooperate. The layers matter — browser, CDN, application, database — because each one protects something specific, and stacking them without understanding what each protects just adds complexity without adding resilience. Invalidation strategy matters more than cache technology choice; a well-reasoned TTL will outperform a poorly-reasoned event-driven system every time. And stampede protection isn't an optimization for later — it's the difference between a cache that occasionally serves stale data and a cache that occasionally takes your database down with it.

If there's one habit worth adopting from all of this, it's asking "what happens when this cache is empty, under full production load, right now" before you ship it — not after you've paged the on-call team to find out the hard way.

Next in this system design series: Building Resilient Systems: Circuit Breakers, Timeouts, and Bulkheads, where we'll look at what happens when the dependency you just cached against becomes the thing that's failing.

Subscribe to get system design and backend engineering posts in your inbox every two weeks.