Why our p99 got worse after we added a cache
We put Redis in front of the hot query and p50 dropped 40 ms. Then p99 climbed. The cache was fine — the problem was that every miss now paid for two round trips instead of one, and the misses arrived together.
Adding jitter to the expiry spread the refetches out. Same cache, same hit rate, p99 back under budget.
replied just now
Fixed TTLs are fine when the key space is huge — misses never line up. It's the small hot sets that bite.