There is no absolutely "better" algorithm in caching. Depends on the usage pattern of the user. We believe LRU is very simple, predictable and works well for most use cases, so it's a good default . It is unrelated to the fact that the Redis LFU implementation is good or not. Your argument is not valid to justify you misrepresented Redis abilities in the post.
I did not claim that DrashCache is "the absolutely best" algorithm.
But some caching algorithms could be better than others on real-world data.
Would you want me to run something like https://github.com/twitter/cache-trace on both Redis/(LRU,LFU) and Dragonfly and see which one has higher hit-rate for the same memory requirements? Would this be a good test to decide if Dragonfly improves on Redis caching quality?
Aside for anything else, backwards compatibility is good (repeatability for a certain behaviour). And it doesn't seem relevant as to why you'd completely ignore it when discussing cache eviction compared to Redis.
Backward compatibility is good? I disagree with you regarding this specific case. Redis philosophy is to add more and more settings, requiring from an end-user to understand internals and low-level trade-offs. And I am sorry, but in the case of caching use-case, a user wants it to "just work" and it's a reasonable expectation. I've run Redis myself in a startup I worked at - I tried LRU, LFU and it did not work very well due to mixed traffic patterns. And I even did not know if it did not work well due to LFU/LRU heuristic or due to the fact that Redis does random sampling and evicts a small number of items it tests. Eventually, I gave up and moved on. I am sure most users do the same or try increasing its cache capacity to ridiculous sizes.
So yeah, I do not completely ignore Redis LFU, I just did not find useful to mention it
because from my experience, Redis LFU is not being widely used, and does not have significant advantage over Redis LRU (if at all) and I wanted to maintain the focus and keep my post succinct and clear.
I also have not provided any benchmarks in the post, not because I have something to hide, just because I do not have an easy setup to benchmark cache efficiency vs Redis and I do not have time to invest into it right now.
Back then when I developed DashCache, I used Caffeine simulator to compare it to dozens other heuristics implemented in Caffeine using real-world data (twitter traces) and it was better than most of then (including LFU). It lost to algorithms that keep some information about evicted items (like TinyLFU or its extensions).
please note that I ran the bench with 30 threads with 1 connection per thread. you're not doing the same but doing 28 threads with 20 connections on each thread, effectively using 560 connections whereas I only used 30 connections. rerun the same with -t 30 and -c 1
if I'm doing my math right: redis is doing 99892 sets per sec, so 25X should be 2,497,300. the video however shows 1,023,030 which is more like 10X and NOT 25X
You can make incredibly fast pseudo dynamic webpages by combining webdis and NGINX SSI. http://nginx.org/en/docs/http/ngx_http_ssi_module.html The way we did this, when the user logged in, we generated their customized content which was the same on all pages -- think of it, this is very common, say, their avatar, the links to their profile etc -- and put it in redis. Now the user gets a customized page without any costly SQL queries, without a scripting language running, it's just NGINX processing SSI include virtuals -- in parallel, no less.
Some valid business use-cases allow eventual consistency. Think of social apps, for example. FB and TW have excellent engineering teams and they built huge systems that rely on caching layers. Similarly, with gaming companies.
Once you reach a certain scale, caching becomes basically a necessity. I mean, heck, even DNS is cached. The idea that caching is a failure of design is just wrong.
https://redis.io/docs/manual/eviction/