Caching Strategies: Memcached, Redis, and CDNs

Your application is slow. A user clicks a button, and the page takes eight seconds to respond. You check your database and realize the query hitting your main table is running a million times a day, even though the data rarely changes. This is a caching problem, and every engineer faces it eventually.

Caching strategies sit at the heart of scalable system design. They let you serve hot data from memory in milliseconds instead of querying a database in tens or hundreds of milliseconds. But caching is not a single solution. It is a layered discipline involving your browser, your CDN, your application server, and your database. Get the layers wrong, and you trade speed for consistency. Get them right, and you ship fast and reliable systems.

Quick answer: Caching strategies operate at multiple layers (browser cache, CDN, application memory, database query results) and follow patterns like cache-aside (lazy loading), read-through, write-through, write-back, and write-around. Cache-aside is simplest and most common for web applications. The hard problems are invalidation, stale data, thundering herd (cache stampede), and eviction policy choice (LRU, LFU, TTL). Redis is a rich, persistent in-memory database supporting data structures and replication; Memcached is a dumb, fast distributed hash. CDNs cache static and edge-computed content globally for latency-critical UX. Choose based on your data mutability, read-to-write ratio, consistency tolerance, and geography.

Why Cache At All?

Caching exists because hardware has a speed hierarchy. A CPU cache hit is nanoseconds. RAM access is tens to hundreds of nanoseconds. Disk I/O is milliseconds. A network round-trip to a database is tens of milliseconds or more. If your hot data lives in the database, you are paying that latency cost repeatedly. Caching moves hot data up the hierarchy.

The trade-off is consistency. A cache is a copy of the truth. When the original changes, the copy may become stale. Your system must decide how stale it tolerates and what freshness guarantees it needs. A cache that is always perfectly in sync with its source is not a cache; it is a redundant copy. Useful caching always accepts some staleness.

The real cost of caching is complexity. You now have to think about invalidation (when to discard a stale copy), eviction (what to throw out when memory fills up), and cache coherency (keeping multiple caches consistent). Most engineers underestimate this cost upfront.

The Cache Layers

Modern systems have multiple cache layers, each with different characteristics and purposes.

Browser cache is the cheapest. The browser stores responses in memory and on disk. A user opens a page, the browser serves it from its local storage without a network round-trip. This layer is almost free, but you do not control when the user's browser invalidates it. You use HTTP cache headers (Cache-Control, ETag, Last-Modified) to suggest how long an asset should be cached.

CDN caches live at the edge, geographically distributed around the world. When a user requests a static file (image, CSS, JavaScript), the CDN serves it from a nearby edge location rather than your origin server. CDNs are excellent for latency-sensitive content and high traffic because they distribute load and reduce origin bandwidth. You push cache invalidation by versioning files, setting appropriate TTLs, or using CDN purge APIs.

Application caches (in-memory stores like Redis or Memcached) live inside your data center or application layer. These are fast, in-process or network-adjacent lookups. An application cache stores computed results, query results, or session data. This is where most engineers focus their caching effort because the latency savings are enormous and the control is local.

Database query caches are built into the database itself (MySQL query cache, for example) or layered in front via a database proxy. These are less common because they are hard to invalidate. Most modern applications skip database query caches and rely on application-level caching instead.

Caching Patterns and Their Tradeoffs

Different patterns suit different problems. Understanding the tradeoffs is critical.

Cache-Aside (Lazy Loading) is the most popular pattern in web applications. Your application code checks the cache first. If the data is not there, the application fetches it from the database, stores it in the cache, and returns it. On subsequent requests, the cache hits and the database is never touched.

The advantage is simplicity. The application is in full control of cache logic. You load only what you actually use. The disadvantage is complexity on the write path. Every write must invalidate the cache, or stale reads will happen. Also, on the first request after invalidation, you pay the database latency cost (this is called a cache miss). If many requests hit during the miss, all of them may query the database (thundering herd problem, discussed later).

Read-Through is less common but valuable in certain systems. You configure a cache that knows how to fetch data from the source (the database). Your application always queries the cache, never the database directly. If the cache misses, the cache itself loads from the database and returns the value. This centralizes the fetch logic but requires the cache to be aware of your data layer. Some in-memory stores and cache libraries support this pattern.

Write-Through ensures consistency. Every write goes to both the cache and the database, synchronously. On a read, the cache is always fresh relative to the database. The tradeoff is write latency. Every write is now two operations, and you wait for both. This pattern is used when strong consistency is critical, like in financial systems, but it is too slow for most web applications.

Write-Back (Write-Behind) is the opposite. Writes go to the cache immediately, and the cache asynchronously flushes to the database (often in batches). This dramatically speeds up writes because the application returns immediately. The risk is data loss if the cache crashes before flushing. Write-back is common in session stores and logging systems where losing some data is acceptable.

Write-Around is a hybrid. Writes go directly to the database, bypassing the cache. Reads use the cache and fall back to the database on miss. This works well when the same data is rarely both written and read frequently (high churn). It avoids polluting the cache with data no one reads.

Here is a comparison table:

Pattern       | Consistency | Write Latency | Complexity
--------------+-------------+---------------+----------
Cache-Aside   | Eventual    | Low           | Medium
Read-Through  | Eventual    | Low           | High
Write-Through | Strong      | High          | Low
Write-Back    | Eventual    | Very Low      | High
Write-Around  | Eventual    | Low           | Medium

Cache Invalidation and the Hard Problems

Phil Karlton famously said: "There are only two hard things in Computer Science: cache invalidation and naming things." He was right.

Invalidation means deciding when a cached value is stale and must be discarded. Three approaches exist.

Time-based invalidation (TTL, Time To Live) sets an expiration time. After X seconds, the cache entry dies, and the next request recomputes. This is simple and requires no application logic. The downside is you do not know if the underlying data changed. If the TTL is short, you check the source often and waste cache hits. If the TTL is long, stale data persists.

Event-based invalidation means explicitly deleting cache entries when the source data changes. This is precise and keeps data fresh. But it is complex to implement correctly. You must know all the cache keys that depend on a data change, and you must invalidate them all. In a large system, this dependency graph can be deep and fragile. Miss one key, and you have a silent stale-data bug.

Dependency-based invalidation (also called tagged caching) tags related cache entries with a common label. When the source changes, you invalidate all entries with that tag in one operation. This is more maintainable than tracking individual keys but requires cache support for tagging.

Cache stampede (also called thundering herd) is a performance disaster. Imagine 10,000 users request the same data right after the cache entry expires. All 10,000 miss the cache and query the database at once, hammering it with load spikes. The database slows down, the queries take longer, and the cache never refills quickly because the database is bottlenecked.

Mitigation strategies include probabilistic early expiration (refresh the cache slightly before it truly expires), using locks (only one thread recomputes, others wait), and accepting that the cache may serve slightly stale data during refresh.

Stale data is a business decision, not a technical one. How old can data be before your users notice or care? A product catalog can be hours stale. A user's account balance must be minutes stale at most. A stock price during trading hours must be seconds or even sub-second stale. Define your staleness tolerance upfront.

Cache Eviction Policies

When a cache fills up, something must be removed to make room. The eviction policy determines what.

LRU (Least Recently Used) is the default. The cache tracks when each item was last accessed. When space is needed, it evicts the item that was least recently used. This works well for temporal locality (recent data is more likely to be needed again). Most systems use LRU for this reason.

LFU (Least Frequently Used) evicts items that are accessed infrequently. This works better for skewed access patterns where a few items are hot and most are cold. An item accessed once per month is evicted before an item accessed once per second, even if the monthly item was accessed more recently.

FIFO (First In, First Out) is simple: evict the oldest item. This is rarely optimal but is easy to implement and useful for scenarios with strong recency (access patterns move forward in time, old data is never needed again).

TTL (Time To Live) is not really an eviction policy; it is a time-based expiration. Items are removed when their TTL expires, regardless of access or frequency. TTL is often combined with LRU or LFU.

Redis and Memcached both support LRU eviction by default. Redis also supports LFU and TTL.

Memcached vs Redis

These two are the dominant in-memory stores in modern web applications. They are often confused, but they have different strengths.

Memcached is a distributed hash table. It stores key-value pairs, and that is all. Values are opaque byte strings. Memcached is extremely simple, extremely fast, and scales to millions of keys. It is stateless and horizontally scalable without any cluster coordination. If a Memcached server fails, you lose the data on that server, but other servers keep working. Memcached has no persistence. On restart, all data is gone. Memcached is single-threaded per connection but multi-threaded overall, and it uses consistent hashing for distribution.

Redis is an in-memory database. It supports rich data structures: strings, lists, sets, sorted sets, hashes, streams, bitmaps, and more. You can store entire objects, run transactions, and execute Lua scripts. Redis is slower than Memcached for simple key-value gets (higher per-operation overhead) but far more expressive. Redis supports persistence to disk (snapshots or append-only logs). Redis supports replication, clustering, and Pub-Sub messaging. Redis is single-threaded, which simplifies concurrency but limits throughput on a single server. Redis 7.0+ supports multi-threading for I/O.

Here is a comparison table:

Feature          | Memcached  | Redis
-----------------+------------+-----------
Data Structures  | Strings    | 8+ types
Persistence      | No         | Yes
Replication      | No         | Yes
Clustering       | Consistent | Sentinel/
                 | Hashing    | Cluster
Transactions     | No         | Yes
Scripting        | No         | Lua
Pub-Sub          | No         | Yes
Eviction Policies| LRU        | LRU/LFU/
                 |            | Random/TTL
Operations/sec   | Very High  | High

Choose Memcached if you need a simple, ultra-fast, stateless cache for session data or computed values where losing data is acceptable. Choose Redis if you need durability, data structure richness, transactions, or replication. For most modern systems, Redis is the default because its flexibility is worth the slight latency trade-off.

CDNs and Edge Caching

Content Delivery Networks (CDNs) cache static assets and edge-computed content geographically. When a user in Mumbai requests an image, the CDN serves it from a Mumbai edge location instead of from your origin server in Virginia. Latency drops from 200+ ms to 20+ ms.

CDNs are essential for any globally distributed application. They reduce origin bandwidth, improve user experience, and provide DDoS protection. Popular CDNs include Cloudflare, Fastly, Akamai, and AWS CloudFront.

CDN caching is controlled via HTTP cache headers (Cache-Control, Surrogate-Control) and origin directives. You can vary caches by user (via Vary headers), purge by URL or tag, and use Stale-While-Revalidate to serve old content while refreshing in the background.

Modern CDNs also support edge computing (serverless functions at edge locations) and request routing based on geography, performance, or custom logic. This enables personalization and dynamic content delivery, not just static file caching.

A Typical Request Path

Here is how a request flows through cache layers in a typical web application:

User Browser
    |
    v (1. Check browser cache)
  [Browser Cache] --miss--> [CDN]
    |                         |
    |(2. Check CDN)           |
    +------------------------+--miss--> [Load Balancer]
                                         |
                                         v
                                    [App Server]
                                         |
                                         v
                       (3. Check application cache)
                              [Redis/Memcached]
                                    |
                           miss------+------hit
                           |                |
                           v                v
                        [Database]      [Return to user]
                           |
                           v
                      [Compute & cache]
                           |
                           v
                      [Return to user]

A hit at any layer saves the cost of all layers below it.

System Design Interview Tips

In a system design interview, caching is almost always part of the discussion. You will be asked about it directly or indirectly.

Start by identifying what data is read frequently and written infrequently. Those are good caching candidates. For data written frequently, caching is harder and less valuable.

Choose a caching pattern appropriate to your consistency requirements. If you need strong consistency, write-through is safer (but slower). If eventual consistency is acceptable, cache-aside is simpler.

Always discuss the hard problems. Show that you understand cache invalidation, thundering herd, and stale data. Do not just say "add Redis" without thinking through the consequences. Interviewers want to hear your reasoning.

Explain your eviction policy choice. Justify why LRU makes sense for your workload, or why you might use LFU for a skewed distribution.

Finally, talk about observability. How will you know if your cache is working? What metrics matter? A cache that is not hit frequently is not worth maintaining. Know your hit rate, eviction rate, and latency improvements.

Frequently Asked Questions

What is the difference between cache invalidation and cache expiration? Cache expiration is time-based (TTL). After X seconds, the cache entry is automatically discarded. Cache invalidation is event-based. When the source data changes, you explicitly delete the cache entry. Both serve the same goal (keeping the cache fresh) but use different triggers.

When should I use Memcached instead of Redis? Memcached is a good choice if you need ultra-high throughput for simple key-value lookups, do not need persistence, do not need replication, and are comfortable losing data on server failure. In most modern systems, Redis is the better choice because its flexibility and durability are worth the slight overhead.

How do I prevent cache stampede? Use probabilistic early expiration to refresh the cache before it truly expires. Use locks so only one thread recomputes on miss while others wait. Or accept that a brief flash flood of database traffic is acceptable and focus on database resilience. No solution is perfect; it depends on your load and consistency tolerance.

Can I use a cache to improve write performance? Yes, write-back caching does this. Writes go to the cache and are asynchronously flushed to the database. This is fast but risky if the cache fails before flushing. Use write-back for non-critical writes (session data, logs) where losing some data is acceptable.

What is the relationship between caching and CDNs? CDNs are a specific type of cache at the network edge, optimized for serving static assets globally. An application cache (Redis/Memcached) is at the application layer and handles dynamic, personalized, or computed data. They solve different problems and are often used together.

How do I measure cache effectiveness? Monitor cache hit rate (hits divided by total requests), eviction rate (items evicted per second), and latency improvement (cache hits vs misses). A hit rate below 80% usually indicates the cache is not being used effectively. Adjust TTL, eviction policy, or cache size based on these metrics.

Practising This with Rehurz

System design is a spoken technical round at Rehurz where you design and defend tradeoffs under cross-questioning. When you tackle a system design problem involving caching (and most do), you will need to discuss layers, patterns, and trade-offs clearly.

Rehurz's system design rounds use adaptive cross-questioning that forces you to think deeply. You will explain why you chose cache-aside over write-through, discuss how you would handle cache stampede under load, and defend your choice of Redis vs Memcached for a specific use case. The interviewer will probe your edge-case thinking and push back on oversimplifications. This builds the exact skills you need for real system design conversations.

Try a Rehurz system design round to practice defending a caching strategy under realistic cross-questioning. Your first interview is free, no card required.

Caching is not something you learn once. It is a discipline you refine through practice and experience. Build systems, measure them, break them, and fix them. Understand the costs and benefits of each layer and pattern. Over time, caching intuition becomes second nature, and you will spot caching opportunities and pitfalls before they become problems.