Redis releases automatic cluster recovery for Kubernetes and RedisInsight GUI tool

"Almost every one of our on-prem customers is shifting to K8s," Redis Labs CTO and co-founder Yiftach Shoolman tells The Register. Redis Labs – home to the enterprise in-memory database platform and open source Redis – was in London for a Redis Day when we spoke to them, and yesterday emitted automated cluster recovery for …

[James 47]

The problem with Redis is its single-threaded nature. It wastes cores, especially if you're running it in the cloud. You could use Twemproxy as a proxy to several redis instances running on separate cores on the same machine, but twemproxy doesn't support all the commands and seems to be dead. There is redis-cluster but that still comes with a use-at-your-own risk warning.

[thames]

Multi-threading wouldn't necessarily give you any better performance, and may in fact reduce overall performance. In-memory databases need locks to maintain data consistency and the lock overhead can be greater than simply running the database on a single thread. This is a very different situation from on-disk databases where the database would otherwise spend considerable time waiting for I/O.

Dr. Stonebraker, who is a noted database researcher and one of the people behind Ingres, Postgres, H-Store, and many others did some detailed research in recent years where he proved the above problems with multi-threading in this type of application. This led to VoltDB also being single-threaded.

The answer is supposedly clustering, but that comes with its own problems as noted in the article.

I used Redis in a application a while ago, and the biggest problem I had was actually the serialising/deserialising overhead inherent in their protocol at the client end. I needed something very fast and with low overhead, and that wasn't it. I'm not sure if they've improved since.

I'm not sure that anyone has yet come up with a really satisfactory solution in this area.

[James 47]

Yes, but that's not what I'm suggesting. It should be possible to run several separate redis instances on each core and have something distribute the load. This is actually possible with Twemproxy, but like I said, that is out of date, if not dead.

I've even been tempted to fork the redis code so that I can run multiple, independent, Redis server threads in a single application.

And yes, the serialisation does seem to be an overhead. Even with twemproxy the main process will use 100% of its core even though there are slaves on the same machine doing the actual redis commands.

[thames]

There's "Redis Cluster Proxy", but that's still in alpha state. I've never tried it, but it's under development on Github.

[GrapeBunch]

"Redis releases automatic cluster recovery for Kubernetes and RedisInsight GUI tool"

Come again?