Redis for .NET Developers – Redis Pipeline Batching

Redis provides a way to use Pipeline Batching to send messages to Redis but first we must understand that Redis uses tcp request response protocol, some may know it as client server model. If you are coming from a web http world, you will have no issue understanding client server, where the browser acts like a client and the web server running (IIS, Nginx or Apache) as the server.

Below is an example of a Client calling the Redis library to make a call to Redis Server to store a value of 10.

In the above diagram as you can see there is a large amount of time where the client is just waiting for a response, the round trip time can have a significant impact if the latency between the network is long.

Redis Pipelining

Redis provides a mechanics called pipelining, which allows a client to send multiple messages to Redis Server without waiting for a reply on each message. Something like the diagram below. Note: Something to consider, Redis does stored these messages into queue such that it can reply to the client and as a result it is using memory on the server.
redis-pipeline

In .NET we are lucky that we have TPL and asycn/await built into our languages, so the design choice that StackExchange.Redis has done is to use the framework itself to handle such situations. One can simply use the Wait() keyword or ContinueWith() for completion of task. There is also another case, which is the CommandFlags.FireAndForget that you may have seen in my previous examples, the FireAndForget allows us to continue immediately in our code since we do not care about the response we get back from Redis Server.

C# code using Redis Pipeliine (TPL)

 class Program
    {
        static void Main(string[] args)
        {
            var redis = RedisStore.RedisCache;

            var redis = RedisStore.RedisCache;
            
            RedisKey alphaKey = "alphaKey";
            RedisKey betaKey = "betaKey";

            redis.KeyDelete(alphaKey, CommandFlags.FireAndForget);
            redis.KeyDelete(betaKey, CommandFlags.FireAndForget);

            var writeTask = redis.StringSetAsync(alphaKey, "abc");
            var writeBetaTask = redis.StringSetAsync(betaKey, "beta");

            var readTask = redis.StringGetAsync(alphaKey);

            redis.Wait(writeTask);

            var readValue = redis.Wait(readTask);

            Console.WriteLine($"Redis Task wait and read {readValue}");

            writeBetaTask.Wait();

            readValue = redis.StringGet(betaKey);

            Console.WriteLine($"Task wait and read {readValue}");

            Console.ReadKey();
        }
    }

class Program

{

static void Main(string[] args)

{

var redis = RedisStore.RedisCache;

RedisKey alphaKey = "alphaKey";

RedisKey betaKey = "betaKey";

redis.KeyDelete(alphaKey, CommandFlags.FireAndForget);

redis.KeyDelete(betaKey, CommandFlags.FireAndForget);

var writeTask = redis.StringSetAsync(alphaKey, "abc");

var writeBetaTask = redis.StringSetAsync(betaKey, "beta");

var readTask = redis.StringGetAsync(alphaKey);

redis.Wait(writeTask);

var readValue = redis.Wait(readTask);

Console.WriteLine($"Redis Task wait and read {readValue}");

writeBetaTask.Wait();

readValue = redis.StringGet(betaKey);

Console.WriteLine($"Task wait and read {readValue}");

Console.ReadKey();

}

Redis Batching

StackExchange.Redis also provides a way to send batch request to Redis. What it allows us to do is to send a block of operations to the server together. The reason for this is that it will help reduce packet fragmentation when the connection to redis is slow. The downside is it will take a longer time to get the first operation complete to process, but overall it can improve the time to get all the operation processed. Below is an example of using Batching.

redis-batch

C# code using Redis batching

 class Program
    {
        static void Main(string[] args)
        {
            var redis = RedisStore.RedisCache;          
            
            RedisKey alphaKey = "alphaKey";
            RedisKey betaKey = "betaKey";

            //Batching
            var list = new List<Task<bool>>();
            var keys = new List<RedisKey> {alphaKey, betaKey};
            IBatch batch = redis.CreateBatch();

            //add the delete into batch
            batch.KeyDeleteAsync(alphaKey);

            foreach (var key in keys)
            {
                var task = batch.StringSetAsync(key, "123");
                list.Add(task);
            }

            batch.Execute();

            Task.WhenAll(list.ToArray());

            readTask = redis.StringGetAsync(alphaKey);
            readValue = redis.Wait(readTask);
            Console.WriteLine($"Alpha read value {readValue}");


            readTask = redis.StringGetAsync(betaKey);
            readValue = redis.Wait(readTask);
            Console.WriteLine($"Beta read value {readValue}");            

            Console.ReadKey();
        }
    }

class Program

{

static void Main(string[] args)

{

var redis = RedisStore.RedisCache;

RedisKey alphaKey = "alphaKey";

RedisKey betaKey = "betaKey";

//Batching

var list = new List<Task<bool>>();

var keys = new List<RedisKey> {alphaKey, betaKey};

IBatch batch = redis.CreateBatch();

//add the delete into batch

batch.KeyDeleteAsync(alphaKey);

foreach (var key in keys)

{

var task = batch.StringSetAsync(key, "123");

list.Add(task);

}

batch.Execute();

Task.WhenAll(list.ToArray());

readTask = redis.StringGetAsync(alphaKey);

readValue = redis.Wait(readTask);

Console.WriteLine($"Alpha read value {readValue}");

readTask = redis.StringGetAsync(betaKey);

readValue = redis.Wait(readTask);

Console.WriteLine($"Beta read value {readValue}");

Console.ReadKey();

}

So this covers the basic usage of Redis Pipeline and Batching, the recommendation for StackExchange.Redis is to use the normal async/await of TPL to handle all the pipeline for you, since StackExchange.Redis does it best under the cover to handle multiplex connections. In the next blog post I will cover how to use Transactions in Redis.

For the code please visit
https://github.com/taswar/RedisForNetDevelopers/tree/master/10.RedisPipeline

For previous Redis topics

1 Comment

Jose Manuel
July 15, 2018 at 7:07 pm

First of all, thanks for this amazing post, I’m new to redis and to read your blog clarify a lot of my doubts. I wonder if you can give me a hand with this problem?

I have created an activity stream, now I need to implement the cache for the A.S . I want to use redis but I’m having a hard time trying to model the data structure

want I need to do is model a master-detail object, the master will contain only one property which is the user id and the detail should be a list of ids (activity ids).

I want to add activities ids to X amount of user, lets said the activity id is ABC and I want to add that activity id to the users with id 1,2,3,4,5…..500, I also want to fire and forget I don’t need to wait for the result of this operation

Would you be so kind to help me put the above problem in a code snippet?

Redis for .NET Developers – Redis Pipeline Batching

Redis Pipelining

C# code using Redis Pipeliine (TPL)

Redis Batching

C# code using Redis batching

1 Comment

Leave a Reply

Opinions are my own and not the views of my employer

LinkedIn

My Book

Archives

Redis for .NET Developers – Redis Pipeline Batching

Redis Pipelining

C# code using Redis Pipeliine (TPL)

Redis Batching

C# code using Redis batching

Related posts:

1 Comment

Leave a Reply

Opinions are my own and not the views of my employer

LinkedIn

My Book

Archives