Sizing and eviction
Memcached is an in-memory cache with a fixed memory budget. Understanding how that budget is used, what happens when it fills up, and how to tell when you need more, is the key to running it well. The short version: pick a memory size, let Memcached evict old entries automatically when it runs out, and resize when your hit rate drops.
Memcached is an in-memory cache with a fixed memory budget. Understanding how that budget is used, what happens when it fills up, and how to tell when you need more, is the key to running it well. The short version: pick a memory size, let Memcached evict old entries automatically when it runs out, and resize when your hit rate drops.
How memory sizing works
When you create a Memcached instance you choose a memory allocation — the amount of memory available for cached items. This is the primary sizing control. You also choose the number of nodes; each node receives the full memory allocation, so adding a second node roughly doubles the total capacity of the instance. See the overview for the available memory and node options.
The platform provisions the CPU and system memory each node needs automatically, with headroom for connection handling and internal overhead. You only decide how much memory is available for caching and how many nodes to run.
A useful way to estimate a starting size:
working_set = avg_item_size_kb * number_of_hot_keys
target_size = working_set * 1.5 # headroom for metadata and churnIf you cannot estimate it, start small, watch the hit rate under real traffic for a day, and resize.
LRU eviction when full
Memcached does not grow beyond the memory you allocate. When the cache is full and a new item needs space, Memcached evicts the least recently used entries to make room. This is normal, expected behaviour — the cache is designed to run full and continually recycle its coldest entries.
Two things follow from this:
- Nothing in the cache is guaranteed to stay there. An item can be evicted before its TTL expires if the cache is under memory pressure. Always treat a cache read as something that may return nothing, and fall back to your source of truth (database, upstream service, recomputation) on a miss.
- Setting a TTL still matters. A TTL bounds how long an item is considered fresh; eviction removes items early under pressure. Use both — a TTL for correctness, eviction to keep memory bounded.
Signs you need more memory
Watch for these symptoms, which usually mean the working set no longer fits:
- Falling hit rate. More requests are missing the cache and hitting your origin. This is the clearest signal.
- High eviction rate. Items are being pushed out faster than they expire. If freshly written entries disappear within seconds, the cache is too small for your traffic.
- Rising database or upstream load that tracks cache misses rather than genuine traffic growth.
The fix is to increase the memory allocation, or add a node to spread keys across more memory. Resize gradually — bump the size, watch the hit rate recover, and stop when it plateaus. There is no benefit to over-provisioning; you pay for what you allocate.
Cache is ephemeral, not storage
Memcached is a cache, not a database. Its contents can vanish for several reasons — LRU eviction under pressure, TTL expiry, a node restart, or the instance being redeployed. Each region runs its own independent cache and they are not synchronised, so a miss in one region never checks another.
Never treat Memcached as the only copy of anything you cannot recreate. It is the right place for data you can always rebuild from an authoritative source and wrong for anything you need to survive a restart. For durable, restart-surviving data, use persistent storage instead.
Related
From the blog