Redis for .NET Developers – Redis Pipeline Batching

Redis provides a way to use Pipeline Batching to send messages to Redis but first we must understand that Redis uses tcp request response protocol, some may know it as client server model. If you are coming from a web http world, you will have no issue understanding client server, where the browser acts like a client and the web server running (IIS, Nginx or Apache) as the server.

Below is an example of a Client calling the Redis library to make a call to Redis Server to store a value of 10.


In the above diagram as you can see there is a large amount of time where the client is just waiting for a response, the round trip time can have a significant impact if the latency between the network is long.

Redis Pipelining

Redis provides a mechanics called pipelining, which allows a client to send multiple messages to Redis Server without waiting for a reply on each message. Something like the diagram below. Note: Something to consider, Redis does stored these messages into queue such that it can reply to the client and as a result it is using memory on the server.

In .NET we are lucky that we have TPL and asycn/await built into our languages, so the design choice that StackExchange.Redis has done is to use the framework itself to handle such situations. One can simply use the Wait() keyword or ContinueWith() for completion of task. There is also another case, which is the CommandFlags.FireAndForget that you may have seen in my previous examples, the FireAndForget allows us to continue immediately in our code since we do not care about the response we get back from Redis Server.

C# code using Redis Pipeliine (TPL)

 class Program
    {
        static void Main(string[] args)
        {
            var redis = RedisStore.RedisCache;

            var redis = RedisStore.RedisCache;
            
            RedisKey alphaKey = "alphaKey";
            RedisKey betaKey = "betaKey";

            redis.KeyDelete(alphaKey, CommandFlags.FireAndForget);
            redis.KeyDelete(betaKey, CommandFlags.FireAndForget);

            var writeTask = redis.StringSetAsync(alphaKey, "abc");
            var writeBetaTask = redis.StringSetAsync(betaKey, "beta");

            var readTask = redis.StringGetAsync(alphaKey);

            redis.Wait(writeTask);

            var readValue = redis.Wait(readTask);

            Console.WriteLine($"Redis Task wait and read {readValue}");

            writeBetaTask.Wait();

            readValue = redis.StringGet(betaKey);

            Console.WriteLine($"Task wait and read {readValue}");

            Console.ReadKey();
        }
    }

Redis Batching

StackExchange.Redis also provides a way to send batch request to Redis. What it allows us to do is to send a block of operations to the server together. The reason for this is that it will help reduce packet fragmentation when the connection to redis is slow. The downside is it will take a longer time to get the first operation complete to process, but overall it can improve the time to get all the operation processed. Below is an example of using Batching.

C# code using Redis batching

 class Program
    {
        static void Main(string[] args)
        {
            var redis = RedisStore.RedisCache;          
            
            RedisKey alphaKey = "alphaKey";
            RedisKey betaKey = "betaKey";

            //Batching
            var list = new List>();
            var keys = new List {alphaKey, betaKey};
            IBatch batch = redis.CreateBatch();

            //add the delete into batch
            batch.KeyDeleteAsync(alphaKey);

            foreach (var key in keys)
            {
                var task = batch.StringSetAsync(key, "123");
                list.Add(task);
            }

            batch.Execute();

            Task.WhenAll(list.ToArray());

            readTask = redis.StringGetAsync(alphaKey);
            readValue = redis.Wait(readTask);
            Console.WriteLine($"Alpha read value {readValue}");


            readTask = redis.StringGetAsync(betaKey);
            readValue = redis.Wait(readTask);
            Console.WriteLine($"Beta read value {readValue}");            

            Console.ReadKey();
        }
    }

So this covers the basic usage of Redis Pipeline and Batching, the recommendation for StackExchange.Redis is to use the normal async/await of TPL to handle all the pipeline for you, since StackExchange.Redis does it best under the cover to handle multiplex connections. In the next blog post I will cover how to use Transactions in Redis.

For the code please visit
https://github.com/taswar/RedisForNetDevelopers/tree/master/10.RedisPipeline

For previous Redis topics

1. Intro to Redis for .NET Developers
2. Redis for .NET Developer – Connecting with C#
3. Redis for .NET Developer – String Datatype
4. Redis for .NET Developer – String Datatype part 2
5. Redis for .NET Developer – Hash Datatype
6. Redis for .NET Developer – List Datatype
7. Redis for .NET Developer – Redis Sets Datatype
8. Redis for .NET Developer – Redis Sorted Sets Datatype
9. Redis for .NET Developer – Redis Hyperloglog
10. Redis for .NET Developer – Redis Pub Sub