Xamarin Dev Days

Taswar Bhatti May 15, 2017 No Comments

Last week I did a presentation on Xamarin Dev Days here in Ottawa, the presentation was on Introduction to Xamarin, unfortunately my demo did not work, the demo gods were not with me that day 🙁
The demo code uses Microsoft Cognitive Service Bing Search to search for images to display on a mobile phone. Mainly I was doing a demo on the Android platform. The code base is on Github
https://github.com/xamarin/dev-days-labs/tree/master/Demos/app-imagesearch-cogs

The slides for my presentation are located here.

Dev days 1 Introduction to Xamarin Taswar Bhatti from Taswar Bhatti, MVP

I also did another demo for Carleton University Business Program on Xamarin and if you wish to see more Xamarin related article please let me know 🙂

.NET, Android, Xamarin

.net, android, vs 2017, Xamarin

Redis for .NET Developer – Redis Transactions

Taswar Bhatti May 1, 2017 No Comments

Redis Transactions

Redis provides a way to do Transactions, but the transactions are somewhat different than what you know from a SQL Relational Database perspective. In SQL Relation Database world you can executed partially and then rolled back the entire transaction. While Redis uses the MULTI and EXEC commands where every command passed as part of a transaction is executed one after another until they have all completed and only after they have completed then other clients may execute their commands. You can think of it more like a queue that stores all your commands and when you call EXEC it loops through the queue and execute the task on the queue one after the other while being locked in the loop i,e no other thread could come in.

Below is an example of a Client queuing up the calls to Redis using MULTI in Redis client library to make a call to Redis Server to process the commands in an EXEC.

redis-transaction

Note: You see Client2 also making a call to Redis but the command is only executed after the process of Client commands.

One important thing to note is when using Redis Transaction one cannot make decisions inside the block, but you can use WATCH to prevent a key to be changed in a transaction. E.g We can WATCH the key “A” in our sample and when we try to use it in MULTI and EXEC it will fail to change A. A will be discarded as shown below, when A was set as 4, even inside of the block we were not able to make the change the EXEC was in some way ROLLBACKED i.e DISCARD

127.0.0.1:6379> WATCH A
OK
127.0.0.1:6379> INCR A
(integer) 4
127.0.0.1:6379> GET A
"4"
127.0.0.1:6379> MULTI
OK
127.0.0.1:6379> INCR A
QUEUED
127.0.0.1:6379> EXEC
(nil)
127.0.0.1:6379>
127.0.0.1:6379> GET A
"4"

127.0.0.1:6379> WATCH A

127.0.0.1:6379> INCR A

(integer) 4

127.0.0.1:6379> GET A

"4"

127.0.0.1:6379> MULTI

127.0.0.1:6379> INCR A

QUEUED

127.0.0.1:6379> EXEC

(nil)

127.0.0.1:6379>

127.0.0.1:6379> GET A

"4"

In Redis StackExchange library uses Conditions to mimic Watch since Redis StackExchange uses Multiplex. Commands can come in different stages but conditions can be used to detect changes.

C# code using Redis Transactions

 class Program
    {
        static void Main(string[] args)
        {
            var redis = RedisStore.RedisCache;

            RedisKey alphaKey = "alphaKey";
            RedisKey betaKey = "betaKey";
            RedisKey gammaKey = "gammaKey";

            redis.KeyDelete(alphaKey, CommandFlags.FireAndForget);
            redis.KeyDelete(betaKey, CommandFlags.FireAndForget);
            redis.KeyDelete(gammaKey, CommandFlags.FireAndForget);

            var trans = redis.CreateTransaction();

            var incr = trans.StringSetAsync(alphaKey, "abc");
            var exec = trans.ExecuteAsync();

            var result = redis.Wait(exec);
            var alphaValue = redis.StringGet(alphaKey);

            Console.WriteLine($"Alpha key is {alphaValue} and result is {result}");
            
            //using conditions to watch keys
            var condition = trans.AddCondition(Condition.KeyNotExists(gammaKey));
            var keyIncrement = trans.StringIncrementAsync(gammaKey);
            exec = trans.ExecuteAsync();
            result = redis.Wait(exec);

            var gammaValue = redis.StringGet(gammaKey);

            Console.WriteLine($"Gamma key is {gammaValue} and result is {result}");

            //fail condition
            condition = trans.AddCondition(Condition.KeyNotExists(gammaKey));
            keyIncrement = trans.StringIncrementAsync(gammaKey);
            exec = trans.ExecuteAsync();

            //resultis false 
            result = redis.Wait(exec);

            gammaValue = redis.StringGet(gammaKey);

            //value is still 1 and result is false
            Console.WriteLine($"Gamma key is {gammaValue} and result is {result}");

            Console.ReadKey();
        }
    }

class Program

{

static void Main(string[] args)

{

var redis = RedisStore.RedisCache;

RedisKey alphaKey = "alphaKey";

RedisKey betaKey = "betaKey";

RedisKey gammaKey = "gammaKey";

redis.KeyDelete(alphaKey, CommandFlags.FireAndForget);

redis.KeyDelete(betaKey, CommandFlags.FireAndForget);

redis.KeyDelete(gammaKey, CommandFlags.FireAndForget);

var trans = redis.CreateTransaction();

var incr = trans.StringSetAsync(alphaKey, "abc");

var exec = trans.ExecuteAsync();

var result = redis.Wait(exec);

var alphaValue = redis.StringGet(alphaKey);

Console.WriteLine($"Alpha key is {alphaValue} and result is {result}");

//using conditions to watch keys

var condition = trans.AddCondition(Condition.KeyNotExists(gammaKey));

var keyIncrement = trans.StringIncrementAsync(gammaKey);

exec = trans.ExecuteAsync();

result = redis.Wait(exec);

var gammaValue = redis.StringGet(gammaKey);

Console.WriteLine($"Gamma key is {gammaValue} and result is {result}");

//fail condition

condition = trans.AddCondition(Condition.KeyNotExists(gammaKey));

keyIncrement = trans.StringIncrementAsync(gammaKey);

exec = trans.ExecuteAsync();

//resultis false

result = redis.Wait(exec);

gammaValue = redis.StringGet(gammaKey);

//value is still 1 and result is false

Console.WriteLine($"Gamma key is {gammaValue} and result is {result}");

Console.ReadKey();

}

So this covers the basic usage of Redis Transactions, in the next blog post I will cover how to use Lua in Redis.

For the code please visit
https://github.com/taswar/RedisForNetDevelopers/tree/master/11.RedisTransaction

For previous Redis topics

C#, Redis

C#, redis, StackExchange.Redis

Redis for .NET Developers – Redis Pipeline Batching

Taswar Bhatti April 25, 2017 1 Comment

Redis provides a way to use Pipeline Batching to send messages to Redis but first we must understand that Redis uses tcp request response protocol, some may know it as client server model. If you are coming from a web http world, you will have no issue understanding client server, where the browser acts like a client and the web server running (IIS, Nginx or Apache) as the server.

Below is an example of a Client calling the Redis library to make a call to Redis Server to store a value of 10.

In the above diagram as you can see there is a large amount of time where the client is just waiting for a response, the round trip time can have a significant impact if the latency between the network is long.

Redis Pipelining

Redis provides a mechanics called pipelining, which allows a client to send multiple messages to Redis Server without waiting for a reply on each message. Something like the diagram below. Note: Something to consider, Redis does stored these messages into queue such that it can reply to the client and as a result it is using memory on the server.
redis-pipeline

In .NET we are lucky that we have TPL and asycn/await built into our languages, so the design choice that StackExchange.Redis has done is to use the framework itself to handle such situations. One can simply use the Wait() keyword or ContinueWith() for completion of task. There is also another case, which is the CommandFlags.FireAndForget that you may have seen in my previous examples, the FireAndForget allows us to continue immediately in our code since we do not care about the response we get back from Redis Server.

C# code using Redis Pipeliine (TPL)

 class Program
    {
        static void Main(string[] args)
        {
            var redis = RedisStore.RedisCache;

            var redis = RedisStore.RedisCache;
            
            RedisKey alphaKey = "alphaKey";
            RedisKey betaKey = "betaKey";

            redis.KeyDelete(alphaKey, CommandFlags.FireAndForget);
            redis.KeyDelete(betaKey, CommandFlags.FireAndForget);

            var writeTask = redis.StringSetAsync(alphaKey, "abc");
            var writeBetaTask = redis.StringSetAsync(betaKey, "beta");

            var readTask = redis.StringGetAsync(alphaKey);

            redis.Wait(writeTask);

            var readValue = redis.Wait(readTask);

            Console.WriteLine($"Redis Task wait and read {readValue}");

            writeBetaTask.Wait();

            readValue = redis.StringGet(betaKey);

            Console.WriteLine($"Task wait and read {readValue}");

            Console.ReadKey();
        }
    }

class Program

{

static void Main(string[] args)

{

var redis = RedisStore.RedisCache;

RedisKey alphaKey = "alphaKey";

RedisKey betaKey = "betaKey";

redis.KeyDelete(alphaKey, CommandFlags.FireAndForget);

redis.KeyDelete(betaKey, CommandFlags.FireAndForget);

var writeTask = redis.StringSetAsync(alphaKey, "abc");

var writeBetaTask = redis.StringSetAsync(betaKey, "beta");

var readTask = redis.StringGetAsync(alphaKey);

redis.Wait(writeTask);

var readValue = redis.Wait(readTask);

Console.WriteLine($"Redis Task wait and read {readValue}");

writeBetaTask.Wait();

readValue = redis.StringGet(betaKey);

Console.WriteLine($"Task wait and read {readValue}");

Console.ReadKey();

}

Redis Batching

StackExchange.Redis also provides a way to send batch request to Redis. What it allows us to do is to send a block of operations to the server together. The reason for this is that it will help reduce packet fragmentation when the connection to redis is slow. The downside is it will take a longer time to get the first operation complete to process, but overall it can improve the time to get all the operation processed. Below is an example of using Batching.

redis-batch

C# code using Redis batching

 class Program
    {
        static void Main(string[] args)
        {
            var redis = RedisStore.RedisCache;          
            
            RedisKey alphaKey = "alphaKey";
            RedisKey betaKey = "betaKey";

            //Batching
            var list = new List<Task<bool>>();
            var keys = new List<RedisKey> {alphaKey, betaKey};
            IBatch batch = redis.CreateBatch();

            //add the delete into batch
            batch.KeyDeleteAsync(alphaKey);

            foreach (var key in keys)
            {
                var task = batch.StringSetAsync(key, "123");
                list.Add(task);
            }

            batch.Execute();

            Task.WhenAll(list.ToArray());

            readTask = redis.StringGetAsync(alphaKey);
            readValue = redis.Wait(readTask);
            Console.WriteLine($"Alpha read value {readValue}");


            readTask = redis.StringGetAsync(betaKey);
            readValue = redis.Wait(readTask);
            Console.WriteLine($"Beta read value {readValue}");            

            Console.ReadKey();
        }
    }

class Program

{

static void Main(string[] args)

{

var redis = RedisStore.RedisCache;

RedisKey alphaKey = "alphaKey";

RedisKey betaKey = "betaKey";

//Batching

var list = new List<Task<bool>>();

var keys = new List<RedisKey> {alphaKey, betaKey};

IBatch batch = redis.CreateBatch();

//add the delete into batch

batch.KeyDeleteAsync(alphaKey);

foreach (var key in keys)

{

var task = batch.StringSetAsync(key, "123");

list.Add(task);

}

batch.Execute();

Task.WhenAll(list.ToArray());

readTask = redis.StringGetAsync(alphaKey);

readValue = redis.Wait(readTask);

Console.WriteLine($"Alpha read value {readValue}");

readTask = redis.StringGetAsync(betaKey);

readValue = redis.Wait(readTask);

Console.WriteLine($"Beta read value {readValue}");

Console.ReadKey();

}

So this covers the basic usage of Redis Pipeline and Batching, the recommendation for StackExchange.Redis is to use the normal async/await of TPL to handle all the pipeline for you, since StackExchange.Redis does it best under the cover to handle multiplex connections. In the next blog post I will cover how to use Transactions in Redis.

For the code please visit
https://github.com/taswar/RedisForNetDevelopers/tree/master/10.RedisPipeline

For previous Redis topics

.NET, Redis

.net, redis, StackExchange.Redis

Redis for .NET Developers – Redis Pub Sub

Taswar Bhatti March 21, 2017 3 Comments

Redis Pub Sub, somehow may sound weird that it provides such a functionality since when you think of Redis the first thing that comes to mind is of a cache key value store. But indeed Redis does allow us to use the messaging paradigm by using channels to publish messages and for subscribers to listen for notification of the message. Redis Pub Sub allows multiple subscribers to listen to one or more channels, and to only receive messages that are of interest. It decouples the subscriber from the publisher since the subscriber has no knowledge of whom the publishers are and vice versa there could be multiple publisher not knowing whom the subscribers are.

Sample Diagram of Redis Pub Sub

Redis Pub Sub

There are definitely certain restrictions of using Redis Pub Sub as a Messaging System, it will not be like RabbitMQ, Kafka or Azure MessageBus etc. Those message bus are able to store the message for durability or even replay of an old message for consumption. Redis uses a listener model where there are no listeners (subscribers) it will not receive those messages. But if you wish to have a simple pub sub without the heavy tools then Redis does quite a good job at it.

Redis Pub Sub – Operations

PUBLISH channels message: Posts a message to the given channel O(N+M)
SUBSCRIBE [channel]: Subscribe to a given channel for message, O(N) where N is the number of channels to subscribe to
PSUBSCRIBE [channel]: Subscribes the client to the given patterns, O(N) where N is the number of patterns the client is already subscribed to.
PUBSUB CHANNELS pattern: Currently active channels, O(N)
PUBSUB NUMSUB channel: Number of subscribers to the channels provided, O(N)
PUBSUB NUMPAT: Number of subscriptions to all the patterns O(N)
PUNSUBSCRIBE: Unsubscribes the client from a pattern, O(N+M)
UNSUBSCRIBE: Unsubscribes the client from a channel, O(N) where N is the number of clients already subscribed to a channel.

C# code using Redis PubSub

 class Program
    {
        static void Main(string[] args)
        {
            var redis = RedisStore.RedisCache;

            var sub = redis.Multiplexer.GetSubscriber();            

            //first subscribe, until we publish
            //subscribe to a test message
            sub.Subscribe("test", (channel, message) => {
                Console.WriteLine("Got notification: " + (string)message);
            });

            //create a publisher
            var pub = redis.Multiplexer.GetSubscriber();

            //pubish to test channel a message
            var count = pub.Publish("test", "Hello there I am a test message");
            Console.WriteLine($"Number of listeners for test {count}");


            //pattern match with a message
            sub.Subscribe(new RedisChannel("a*c", RedisChannel.PatternMode.Pattern), (channel, message) => {
                Console.WriteLine($"Got pattern a*c notification: {message}");
            });


            count = pub.Publish("a*c", "Hello there I am a a*c message");
            Console.WriteLine($"Number of listeners for a*c {count}");

            pub.Publish("abc", "Hello there I am a abc message");
            pub.Publish("a1234567890c", "Hello there I am a a1234567890c message");
            pub.Publish("ab", "Hello I am a lost message"); //this mesage is never printed


            //Never a pattern match with a message
            sub.Subscribe(new RedisChannel("*123", RedisChannel.PatternMode.Literal), (channel, message) => {
                Console.WriteLine($"Got Literal pattern *123 notification: {message}");
            });


            pub.Publish("*123", "Hello there I am a *123 message");
            pub.Publish("a123", "Hello there I am a a123 message"); //message is never received due to literal pattern


            //Auto pattern match with a message
            sub.Subscribe(new RedisChannel("zyx*", RedisChannel.PatternMode.Auto), (channel, message) => {
                Console.WriteLine($"Got Literal pattern zyx* notification: {message}");
            });


            pub.Publish("zyxabc", "Hello there I am a zyxabc message");
            pub.Publish("zyx1234", "Hello there I am a zyxabc message");

            //no message being published to it so it will not receive any previous messages
            sub.Subscribe("test", (channel, message) => {
                Console.WriteLine($"I am a late subscriber Got notification: {message}");
            });


            sub.Unsubscribe("a*c");
            count = pub.Publish("abc", "Hello there I am a abc message"); //no one listening anymore
            Console.WriteLine($"Number of listeners for a*c {count}");

            Console.ReadKey();
        }
    }

class Program

{

static void Main(string[] args)

{

var redis = RedisStore.RedisCache;

var sub = redis.Multiplexer.GetSubscriber();

//first subscribe, until we publish

//subscribe to a test message

sub.Subscribe("test", (channel, message) => {

Console.WriteLine("Got notification: " + (string)message);

});

//create a publisher

var pub = redis.Multiplexer.GetSubscriber();

//pubish to test channel a message

var count = pub.Publish("test", "Hello there I am a test message");

Console.WriteLine($"Number of listeners for test {count}");

//pattern match with a message

sub.Subscribe(new RedisChannel("a*c", RedisChannel.PatternMode.Pattern), (channel, message) => {

Console.WriteLine($"Got pattern a*c notification: {message}");

});

count = pub.Publish("a*c", "Hello there I am a a*c message");

Console.WriteLine($"Number of listeners for a*c {count}");

pub.Publish("abc", "Hello there I am a abc message");

pub.Publish("a1234567890c", "Hello there I am a a1234567890c message");

pub.Publish("ab", "Hello I am a lost message"); //this mesage is never printed

//Never a pattern match with a message

sub.Subscribe(new RedisChannel("*123", RedisChannel.PatternMode.Literal), (channel, message) => {

Console.WriteLine($"Got Literal pattern *123 notification: {message}");

});

pub.Publish("*123", "Hello there I am a *123 message");

pub.Publish("a123", "Hello there I am a a123 message"); //message is never received due to literal pattern

//Auto pattern match with a message

sub.Subscribe(new RedisChannel("zyx*", RedisChannel.PatternMode.Auto), (channel, message) => {

Console.WriteLine($"Got Literal pattern zyx* notification: {message}");

});

pub.Publish("zyxabc", "Hello there I am a zyxabc message");

pub.Publish("zyx1234", "Hello there I am a zyxabc message");

//no message being published to it so it will not receive any previous messages

sub.Subscribe("test", (channel, message) => {

Console.WriteLine($"I am a late subscriber Got notification: {message}");

});

sub.Unsubscribe("a*c");

count = pub.Publish("abc", "Hello there I am a abc message"); //no one listening anymore

Console.WriteLine($"Number of listeners for a*c {count}");

Console.ReadKey();

}

So this covers the basic usage of Redis PubSub, in the next blog post I will cover how to use Pipelines in Redis.

For the code please visit
https://github.com/taswar/RedisForNetDevelopers/tree/master/9.RedisPubSub

For previous Redis topics

.NET, C#, Redis

.net, C#, redis, StackExchange.Redis

Upcoming Talks at DevTeach Montreal 2017

Taswar Bhatti March 20, 2017 No Comments

Yeahhhh I am really exited that I have been selected to speak at DevTeach Montreal 2017 this year, July 3rd onward to July 7th in the lovely Montreal Canada.

I remember my first visit to DevTeach was in 2007 as an attendee, now 10 years later I am presenting in the conference. Looking back it has been one amazing ride of where things are for me. In 2007 when I was attending Roy Osherove presented on Agile Development and ended with singing a song at the end of the session, which was quite memorable moment. To JP Boodhoo session on refactoring code and Rules for Agile Design by Jeremy Miller, Oren Eini on MonoRails and Rhino Mocks. Those were quite the early days of ALT .NET and who would have known that .NET would now be open source and runs on Linux with dotnet core.

I wonder what the next 10 years would be like for .net community?

My two abstract which has been selected by DevTeach
– Store 2 million of audit a day into Elastic Search
– OAuth2 and OpenId Connect Demystified

You can read more on the abstract on the DevTeach site.

Come and join us at DevTeach Montreal, it would be an awesome event with the Jazz festival happening at the same time and one day you never know you may be speaking in DevTeach too….

ElasticSearch, OAuth2, Public Speaking

devteach, elastic search, oauth, oauth2, public speaking

Introduction to Xamarin – speaking at Carleton

Taswar Bhatti March 7, 2017 No Comments

I was invited to Sprott School of Business – Carleton University for an introduction to the class on Xamarin what Microsoft platform offers for mobile development.

Below you will see my sides that were present by me and Ahmed Al-Assad a fellow MVP.

A special thanks to Alex Ramirez for the invitation to speak to the students.

Xamarin forms introduction by Taswar Bhatti and Ahmed Assad from Taswar Bhatti, MVP

.NET, Xamarin

Xamarin

Docker for .NET Developers

Taswar Bhatti February 28, 2017 No Comments

Docker for .NET Developers

Last week I was presenting at the Ottawa IT User Group for .NET Developers on Docker for .NET Developers. We covered what docker is, how it fits into microservices, overview of using docker as a .NET Developer etc

Topics that we covered were

What is Docker
Images and Containers
Docker vs VM
Tooling around Docker (Visual Studio Tools for Docker)
Docker linking with other containers
How to use docker compose to link containers

Sample code for the demo on using Postgres, Dotnet core and docker-compose file are located at
https://github.com/taswar/postgre-dotnetcore

If you would like to see me write more on docker, feel free to get in touch or comment.

Here are the presentation slides. Enjoy and feel free to share or clip them 🙂

Docker for .NET Developers from Taswar Bhatti, MVP

.NET, ASP .NET, docker, visual studio

.net, docker, dotnet core

Redis for .NET Developers – Redis HyperLogLog Datatype

Taswar Bhatti January 12, 2017 3 Comments

Redis HyperLogLog is used for calculating the cardinality of a set or non mathematically speaking it is a probabilistic data structure used to count unique values. Basically it is an algorithm that use randomization such that it can provide an approximation of the number of unique elements in a set by just using a constant complexity, and a fixed/small amount of memory footprint. Each key stored in hyperloglog is only about 12 kbytes per key and with a standard error of 0.81%. There is no limit to the number of items you can count in the HyperLogLog Datatype, unless you approach 2^⁶⁴ items. If you are planning to use some form of unique count, e.g ipadress

Redis HyperLogLog Datatype – Operations

PFADD: Adds or updates one or more members to HyperLogLog, Returns 1 if at least 1 HyperLogLog internal register was altered, 0 otherwise O(1)
PFCOUNT: Returns the approximated cardinality computed by the HyperLogLog, 0 if the variable does not exist. O(1).
PFMERGE: Merge multiple HyperLogLog values into an unique value O(N)

C# code using Redis HyperLogLog Datatype

  class Program
    {
        static void Main(string[] args)
        {
            var redis = RedisStore.RedisCache;

            RedisKey key = "setKey";
            RedisKey alphaKey = "alphaKey";            
            RedisKey destinationKey = "destKey";

            redis.KeyDelete(key, CommandFlags.FireAndForget);
            redis.KeyDelete(alphaKey, CommandFlags.FireAndForget);
            redis.KeyDelete(destinationKey, CommandFlags.FireAndForget);

            redis.HyperLogLogAdd(key, "a");
            redis.HyperLogLogAdd(key, "b");
            redis.HyperLogLogAdd(key, "c");

            Console.WriteLine(redis.HyperLogLogLength(key)); //output 3


            redis.HyperLogLogAdd(alphaKey, "1");
            redis.HyperLogLogAdd(alphaKey, "2");
            redis.HyperLogLogAdd(alphaKey, "3");


            Console.WriteLine(redis.HyperLogLogLength(alphaKey)); //output 3


            redis.HyperLogLogAdd(alphaKey, "a");

            Console.WriteLine(redis.HyperLogLogLength(alphaKey)); //output 4


            redis.HyperLogLogMerge(destinationKey, key, alphaKey);


            Console.WriteLine(redis.HyperLogLogLength(destinationKey)); //output 6 which is (a, b, c, 1, 2, 3)

            Console.ReadKey();
        }    
    }

class Program

{

static void Main(string[] args)

{

var redis = RedisStore.RedisCache;

RedisKey key = "setKey";

RedisKey alphaKey = "alphaKey";

RedisKey destinationKey = "destKey";

redis.KeyDelete(key, CommandFlags.FireAndForget);

redis.KeyDelete(alphaKey, CommandFlags.FireAndForget);

redis.KeyDelete(destinationKey, CommandFlags.FireAndForget);

redis.HyperLogLogAdd(key, "a");

redis.HyperLogLogAdd(key, "b");

redis.HyperLogLogAdd(key, "c");

Console.WriteLine(redis.HyperLogLogLength(key)); //output 3

redis.HyperLogLogAdd(alphaKey, "1");

redis.HyperLogLogAdd(alphaKey, "2");

redis.HyperLogLogAdd(alphaKey, "3");

Console.WriteLine(redis.HyperLogLogLength(alphaKey)); //output 3

redis.HyperLogLogAdd(alphaKey, "a");

Console.WriteLine(redis.HyperLogLogLength(alphaKey)); //output 4

redis.HyperLogLogMerge(destinationKey, key, alphaKey);

Console.WriteLine(redis.HyperLogLogLength(destinationKey)); //output 6 which is (a, b, c, 1, 2, 3)

Console.ReadKey();

}

So this covers the basic usage of Redish HyperLogLog Datatype, in the next blog post I will cover how to use Pub/Sub in Redis.

For the code please visit
https://github.com/taswar/RedisForNetDevelopers/tree/master/8.RedisHyperLogLog

For previous Redis topics

.NET, Learn Series, Redis

.net, redis, StackExchange.Redis

Redis for .NET Developers – Redis Sorted Sets Datatype

Taswar Bhatti December 29, 2016 No Comments

Redis Sorted Sets are similar to Sets, with unique feature of values stored in Set called scores. The score is used in order to take the sorted set ordered, from the smallest to the greatest score. Just like in Sets, members are unique but scores can be repeated. Sorted Sets are ideal for storing index data in Redis, or using some of the Sorted Set commands to find out how many users have authenticated to a system, or top users using the system etc.

Redis Sorted Sets Datatype – Operations

ZADD: Adds or updates one or more members to a Sorted Set O(log (N))
ZRANGE: Gets the specified range by rank of elements in the Sorted Set O(log (N) +M).
ZRANGEBYSCORE: Gets elements from the Sorted Sets within the range by score given values are in ascending order O(log (N) +M)
ZREVRANGEBYSCORE: Gets elements from the Sorted Sets within the score given O(log (N) +M)
ZREVRANK: The rank of the member in the Sorted Set O (log (N))
ZREVRANGE: Returns the specified range of elements in the Sorted Set O(log (N) + M)
ZREM: Removes the specified members in the Sorted Set O(M*log (N))
ZREMRANGEBYRANK: Removes the members in a Sorted Set within the given indexes O(log (N) * M)
ZREMRANGEBYSCORE: Removes the members in a Sorted Set within the given scores O(log (N) * M)
ZCARD: Gets the number of members in a Sorted Set O(1)
ZCOUNT: Gets the number of members in a Sorted Set within the score boundaries O(log (N) * M)
ZINCRBY: Increases the score of an element in the Sorted Set O(log (N))
ZINTERSTORE: Calculates the common elements in the Sorted Sets given by the specified keys, and stores the result in destination Sorted Set O(N*K) + O (M*log (M))
ZRANK: Gets the index of the element in the Sorted Set O(log (N))
ZSCORE: Returns the score of the member O(1)
ZUNIONSTORE: Computes the union of keys in the given Sorted Set and stores the result in the resultant Sorted Set O(N) + O(M log (M))

C# code using Redis Sorted Set Datatype

  class Program
    {
        static void Main(string[] args)
        {
            var redis = RedisStore.RedisCache;

            RedisKey topHackerKeys = "hackers";
            RedisKey alphaKey = "alphaKey";
            RedisKey destinationKey = "destKey";
            RedisKey intersectKey = "intersectKey";


            redis.KeyDelete(topHackerKeys, CommandFlags.FireAndForget);
            redis.KeyDelete(alphaKey, CommandFlags.FireAndForget);
            redis.KeyDelete(intersectKey, CommandFlags.FireAndForget);
            redis.KeyDelete(destinationKey, CommandFlags.FireAndForget);

            //According to http://www.arkhitech.com/12-greatest-programmers-of-all-time/
            var topProgrammers = new[] {
                "Dennis Ritchie",
                "Linus Torvalds",
                "Bjarne Stroustrup",
                "Tim Berners-Lee",
                "Brian Kernighan",
                "Donald Knuth",
                "Ken Thompson",
                "Guido van Rossum",
                "James Gosling",
                "Bill Gates",
                "Niklaus Wirth",
                "Ada Lovelace"
            };

            //add 12 items to the sorted set
            for (int i = 0, j = 1; i < topProgrammers.Length; i++, j++)
                redis.SortedSetAdd(topHackerKeys, topProgrammers[i], j);

            var members = redis.SortedSetScan(topHackerKeys);

            Console.WriteLine(string.Join(",\n", members)); 
            /* output 
             * Dennis Ritchie: 1, 
             * Linus Torvalds: 2,
             * Bjarne Stroustrup: 3,
             * Tim Berners-Lee: 4,
             * Brian Kernighan: 5,
             * Donald Knuth: 6,
             * Ken Thompson: 7,
             * Guido van Rossum: 8,
             * James Gosling: 9,
             * Bill Gates: 10,
             * Niklaus Wirth: 11,
             * Ada Lovelace: 12
            */


            Console.WriteLine(redis.SortedSetLength(topHackerKeys)); //output 12

            var byRanks = redis.SortedSetRangeByRank(topHackerKeys);
            Console.WriteLine(string.Join(",\n", byRanks));
            /* output 
            * Dennis Ritchie, 
            * Linus Torvalds,
            * Bjarne Stroustrup,
            * Tim Berners-Lee,
            * Brian Kernighan,
            * Donald Knuth,
            * Ken Thompson,
            * Guido van Rossum,
            * James Gosling,
            * Bill Gates,
            * Niklaus Wirth,
            * Ada Lovelace
           */


            Console.WriteLine(redis.SortedSetRank(topHackerKeys, "Linus Torvalds")); //output 1


            var byScore = redis.SortedSetRangeByScore(topHackerKeys, 1, topProgrammers.Length,Exclude.None, Order.Descending);
            Console.WriteLine(string.Join(",\n", byScore));
            /*output
             * Ada Lovelace,
             * Niklaus Wirth,
             * Bill Gates,
             * James Gosling,
             * Guido van Rossum,
             * Ken Thompson,
             * Donald Knuth,
             * Brian Kernighan,
             * Tim Berners-Lee,
             * Bjarne Stroustrup,
             * Linus Torvalds,
             * Dennis Ritchie
             */

            redis.SortedSetIncrement(topHackerKeys, "Linus Torvalds", 100);

            Console.WriteLine(redis.SortedSetScore(topHackerKeys, "Linus Torvalds")); //output 102, since it was 2 to being with

            redis.SortedSetDecrement(topHackerKeys, "Linus Torvalds", 100);

            Console.WriteLine(redis.SortedSetScore(topHackerKeys, "Linus Torvalds")); //output 2 back to original value

            redis.SortedSetAdd(alphaKey, "a", 1);
            redis.SortedSetAdd(alphaKey, "b", 1);
            redis.SortedSetAdd(alphaKey, "c", 1);

            redis.SortedSetCombineAndStore(SetOperation.Union, destinationKey, topHackerKeys, alphaKey);

            members = redis.SortedSetScan(destinationKey);
            Console.WriteLine("**********UNION**************");
            Console.WriteLine(string.Join(",\n", members));
            /* output
             * Dennis Ritchie: 1,
             * a: 1,
             * b: 1,
             * c: 1,
             * Linus Torvalds: 2,
             * Bjarne Stroustrup: 3,
             * Tim Berners-Lee: 4,
             * Brian Kernighan: 5,
             * Donald Knuth: 6,
             * Ken Thompson: 7,
             * Guido van Rossum: 8,
             * James Gosling: 9,
             * Bill Gates: 10,
             * Niklaus Wirth: 11,
             * Ada Lovelace: 12
             */
            
            redis.SortedSetCombineAndStore(SetOperation.Intersect, intersectKey, topHackerKeys, destinationKey);
            members = redis.SortedSetScan(intersectKey);

            //note it double the key scores
            Console.WriteLine("**********INTERSECT**************");
            Console.WriteLine(string.Join(",\n", members));
            /*output
             * Dennis Ritchie: 2,
             * Linus Torvalds: 4,
             * Bjarne Stroustrup: 6,
             * Tim Berners-Lee: 8,
             * Brian Kernighan: 10,
             * Donald Knuth: 12,
             * Ken Thompson: 14,
             * Guido van Rossum: 16,
             * James Gosling: 18,
             * Bill Gates: 20,
             * Niklaus Wirth: 22,
             * Ada Lovelace: 24
             */

            members = redis.SortedSetRangeByScoreWithScores(topHackerKeys, 2, 4);
            Console.WriteLine("**********RANGE BY SCORE WITH SCORES**************");
            Console.WriteLine(string.Join(",\n", members));
            /*output
             * Linus Torvalds: 2,
             * Bjarne Stroustrup: 3,
             * Tim Berners-Lee: 4
             */


            redis.SortedSetRemove(alphaKey, "a");
            members = redis.SortedSetScan(alphaKey);
            Console.WriteLine("**********REMOVE**************");
            Console.WriteLine(string.Join(",\n", members));
            /*output
             * b: 1,
             * c: 1
             */

            redis.SortedSetRemoveRangeByScore(destinationKey, 0, 11);
            members = redis.SortedSetScan(destinationKey);
            Console.WriteLine("**********REMOVE BY SCORE**************");
            Console.WriteLine(string.Join(",\n", members));
            /* output
             * Ada Lovelace: 12
             */
                        

            Console.ReadKey();
        }    
    }

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

class Program

{

static void Main(string[] args)

{

var redis = RedisStore.RedisCache;

RedisKey topHackerKeys = "hackers";

RedisKey alphaKey = "alphaKey";

RedisKey destinationKey = "destKey";

RedisKey intersectKey = "intersectKey";

redis.KeyDelete(topHackerKeys, CommandFlags.FireAndForget);

redis.KeyDelete(alphaKey, CommandFlags.FireAndForget);

redis.KeyDelete(intersectKey, CommandFlags.FireAndForget);

redis.KeyDelete(destinationKey, CommandFlags.FireAndForget);

//According to http://www.arkhitech.com/12-greatest-programmers-of-all-time/

var topProgrammers = new[] {

"Dennis Ritchie",

"Linus Torvalds",

"Bjarne Stroustrup",

"Tim Berners-Lee",

"Brian Kernighan",

"Donald Knuth",

"Ken Thompson",

"Guido van Rossum",

"James Gosling",

"Bill Gates",

"Niklaus Wirth",

"Ada Lovelace"

};

//add 12 items to the sorted set

for (int i = 0, j = 1; i < topProgrammers.Length; i++, j++)

redis.SortedSetAdd(topHackerKeys, topProgrammers[i], j);

var members = redis.SortedSetScan(topHackerKeys);

Console.WriteLine(string.Join(",\n", members));

/* output

* Dennis Ritchie: 1,

* Linus Torvalds: 2,

* Bjarne Stroustrup: 3,

* Tim Berners-Lee: 4,

* Brian Kernighan: 5,

* Donald Knuth: 6,

* Ken Thompson: 7,

* Guido van Rossum: 8,

* James Gosling: 9,

* Bill Gates: 10,

* Niklaus Wirth: 11,

* Ada Lovelace: 12

Console.WriteLine(redis.SortedSetLength(topHackerKeys)); //output 12

var byRanks = redis.SortedSetRangeByRank(topHackerKeys);

Console.WriteLine(string.Join(",\n", byRanks));

/* output

* Dennis Ritchie,

* Linus Torvalds,

* Bjarne Stroustrup,

* Tim Berners-Lee,

* Brian Kernighan,

* Donald Knuth,

* Ken Thompson,

* Guido van Rossum,

* James Gosling,

* Bill Gates,

* Niklaus Wirth,

* Ada Lovelace

Console.WriteLine(redis.SortedSetRank(topHackerKeys, "Linus Torvalds")); //output 1

var byScore = redis.SortedSetRangeByScore(topHackerKeys, 1, topProgrammers.Length,Exclude.None, Order.Descending);

Console.WriteLine(string.Join(",\n", byScore));

/*output

* Ada Lovelace,

* Niklaus Wirth,

* Bill Gates,

* James Gosling,

* Guido van Rossum,

* Ken Thompson,

* Donald Knuth,

* Brian Kernighan,

* Tim Berners-Lee,

* Bjarne Stroustrup,

* Linus Torvalds,

* Dennis Ritchie

redis.SortedSetIncrement(topHackerKeys, "Linus Torvalds", 100);

Console.WriteLine(redis.SortedSetScore(topHackerKeys, "Linus Torvalds")); //output 102, since it was 2 to being with

redis.SortedSetDecrement(topHackerKeys, "Linus Torvalds", 100);

Console.WriteLine(redis.SortedSetScore(topHackerKeys, "Linus Torvalds")); //output 2 back to original value

redis.SortedSetAdd(alphaKey, "a", 1);

redis.SortedSetAdd(alphaKey, "b", 1);

redis.SortedSetAdd(alphaKey, "c", 1);

redis.SortedSetCombineAndStore(SetOperation.Union, destinationKey, topHackerKeys, alphaKey);

members = redis.SortedSetScan(destinationKey);

Console.WriteLine("**********UNION**************");

Console.WriteLine(string.Join(",\n", members));

/* output

* Dennis Ritchie: 1,

* a: 1,

* b: 1,

* c: 1,

* Linus Torvalds: 2,

* Bjarne Stroustrup: 3,

* Tim Berners-Lee: 4,

* Brian Kernighan: 5,

* Donald Knuth: 6,

* Ken Thompson: 7,

* Guido van Rossum: 8,

* James Gosling: 9,

* Bill Gates: 10,

* Niklaus Wirth: 11,

* Ada Lovelace: 12

redis.SortedSetCombineAndStore(SetOperation.Intersect, intersectKey, topHackerKeys, destinationKey);

members = redis.SortedSetScan(intersectKey);

//note it double the key scores

Console.WriteLine("**********INTERSECT**************");

Console.WriteLine(string.Join(",\n", members));

/*output

* Dennis Ritchie: 2,

* Linus Torvalds: 4,

* Bjarne Stroustrup: 6,

* Tim Berners-Lee: 8,

* Brian Kernighan: 10,

* Donald Knuth: 12,

* Ken Thompson: 14,

* Guido van Rossum: 16,

* James Gosling: 18,

* Bill Gates: 20,

* Niklaus Wirth: 22,

* Ada Lovelace: 24

members = redis.SortedSetRangeByScoreWithScores(topHackerKeys, 2, 4);

Console.WriteLine("**********RANGE BY SCORE WITH SCORES**************");

Console.WriteLine(string.Join(",\n", members));

/*output

* Linus Torvalds: 2,

* Bjarne Stroustrup: 3,

* Tim Berners-Lee: 4

redis.SortedSetRemove(alphaKey, "a");

members = redis.SortedSetScan(alphaKey);

Console.WriteLine("**********REMOVE**************");

Console.WriteLine(string.Join(",\n", members));

/*output

* b: 1,

* c: 1

redis.SortedSetRemoveRangeByScore(destinationKey, 0, 11);

members = redis.SortedSetScan(destinationKey);

Console.WriteLine("**********REMOVE BY SCORE**************");

Console.WriteLine(string.Join(",\n", members));

/* output

* Ada Lovelace: 12

Console.ReadKey();

}

So this covers the basic usage of Redish Sorted Sets Datatype, in the next blog post I will cover using Hyperloglog Datatype.

For the code please visit
https://github.com/taswar/RedisForNetDevelopers/tree/master/5.RedisList/RedisList

For previous Redis topics

C#, Redis

C#, redis, StackExchange.Redis

Redis for .NET Developers – Redis Sets Datatype

Taswar Bhatti December 28, 2016 No Comments

Redis Sets Datatype are similar in C# world as HashSet, they are an unordered collection used for storing strings. One of the great benefit of Redis Sets is that the operation of add, remove, and testing for existence of items is of constant time of O(1) regardless of the number of items in the Set. An important thing about Sets is when adding the same element multiple times, it will only result in a set having a single copy of the item. In a way, one does not have to check if an item exist or not before adding. The amount of members you can store in a Set is more that 4 billion, thus one can store quite a few things in them. Sets are ideal candidate for set theory related items like unique number of people visiting a site, tagging of items, etc.

Redis Sets Datatype – Operations

SADD: Adds one or more elements to the Set O(N).
SPOP: Removes and returns a random element from the set O(1).
SREM: Removes and returns the specified elements from the set O(N).
SCARD: Gets the number of elements in a Set O(1).
SDIFF: Gets the list of elements from the first set after subtracting its elements from the other mentioned sets O(N).
SDIFFSTORE: Create or override a set, getting the list of elements from the first set after subtracting its elements from the other mentioned sets O(N).
SINTER: Gets the common elements in all the sets mentioned O(N * M).
SINTERSTORE: Store the elements in a set after intersection of all specified sets similar to SINTER but results are stored in the mentioned set O(N * M).
SISMEMBER: Finds if the value is a member of set O(1).
SMOVE: Moves members from one set to another set O(1).
SRANDMEMBER: Gets one or multiple random members from the set O(N).
SUNION: Adds multiple sets O(N).
SUNIONSTORE: Adds multiple sets and stores the result in a set O(N).

C# code using Redis Set Datatype

      class Program
    {
        static void Main(string[] args)
        {
            var redis = RedisStore.RedisCache;

            RedisKey key = "setKey";
            RedisKey alphaKey = "alphaKey";
            RedisKey numKey = "numberKey";
            RedisKey destinationKey = "destKey";


            redis.KeyDelete(key, CommandFlags.FireAndForget);
            redis.KeyDelete(alphaKey, CommandFlags.FireAndForget);
            redis.KeyDelete(numKey, CommandFlags.FireAndForget);
            redis.KeyDelete(destinationKey, CommandFlags.FireAndForget);


            //add 10 items to the set
            for (int i = 1; i <= 10; i++)
                redis.SetAdd(key, i);

            var members = redis.SetMembers(key);

            Console.WriteLine(string.Join(",", members)); //output 1,2,3,4,5,6,7,8,9,10
                        
            //remove 5th element
            redis.SetRemove(key, 5);

            Console.WriteLine(redis.SetContains(key, 5)); //False

            members = redis.SetMembers(key);

            Console.WriteLine(string.Join(",", members)); //output 1,2,3,4,6,7,8,9,10
            
            Console.WriteLine(redis.SetContains(key, 10)); //True

            //number of elements
            Console.WriteLine(redis.SetLength(key)); //output 9
           
            
            //add alphabets to set
            redis.SetAdd(alphaKey, "abc".Select(x => (RedisValue) x.ToString()).ToArray());
            redis.SetAdd(numKey, "123".Select(x => (RedisValue)x.ToString()).ToArray());

            var values = redis.SetCombine(SetOperation.Union, numKey, alphaKey);

            Console.WriteLine(string.Join(",", values)); //unordered list of items (e.g output can be "1,3,2,a,c,b")
            
            values = redis.SetCombine(SetOperation.Difference, key, numKey);

            Console.WriteLine(string.Join(",", values)); //4, 6, 7, 8, 9, 10

            values = redis.SetCombine(SetOperation.Intersect, key, numKey);

            Console.WriteLine(string.Join(",", values)); //1, 2, 3
         
            //move a random from source numKey to aplhaKey 
            redis.SetMove(numKey, alphaKey, 2);

            members = redis.SetMembers(alphaKey);

            Console.WriteLine(string.Join(",", members)); //output can be (b, c, a, 2)

            //Add apple to 
            redis.SetAdd(alphaKey, "apple");

            //look for item that starts with he
            var patternMatchValues = redis.SetScan(alphaKey, "ap*"); //output apple

            Console.WriteLine(string.Join(",", patternMatchValues));

            patternMatchValues = redis.SetScan(alphaKey, "a*"); //output apple, a

            Console.WriteLine(string.Join(",", patternMatchValues));
            

            //store into destinantion key the union of numKey and alphaKey
            redis.SetCombineAndStore(SetOperation.Union, destinationKey, numKey, alphaKey); 

            Console.WriteLine(string.Join(",", redis.SetMembers(destinationKey)));

            //radmon value and removes it from the set
            var randomVal = redis.SetPop(numKey);

            Console.WriteLine(randomVal); //random member in the set

            Console.ReadKey();
        }
    }

class Program

{

static void Main(string[] args)

{

var redis = RedisStore.RedisCache;

RedisKey key = "setKey";

RedisKey alphaKey = "alphaKey";

RedisKey numKey = "numberKey";

RedisKey destinationKey = "destKey";

redis.KeyDelete(key, CommandFlags.FireAndForget);

redis.KeyDelete(alphaKey, CommandFlags.FireAndForget);

redis.KeyDelete(numKey, CommandFlags.FireAndForget);

redis.KeyDelete(destinationKey, CommandFlags.FireAndForget);

//add 10 items to the set

for (int i = 1; i <= 10; i++)

redis.SetAdd(key, i);

var members = redis.SetMembers(key);

Console.WriteLine(string.Join(",", members)); //output 1,2,3,4,5,6,7,8,9,10

//remove 5th element

redis.SetRemove(key, 5);

Console.WriteLine(redis.SetContains(key, 5)); //False

members = redis.SetMembers(key);

Console.WriteLine(string.Join(",", members)); //output 1,2,3,4,6,7,8,9,10

Console.WriteLine(redis.SetContains(key, 10)); //True

//number of elements

Console.WriteLine(redis.SetLength(key)); //output 9

//add alphabets to set

redis.SetAdd(alphaKey, "abc".Select(x => (RedisValue) x.ToString()).ToArray());

redis.SetAdd(numKey, "123".Select(x => (RedisValue)x.ToString()).ToArray());

var values = redis.SetCombine(SetOperation.Union, numKey, alphaKey);

Console.WriteLine(string.Join(",", values)); //unordered list of items (e.g output can be "1,3,2,a,c,b")

values = redis.SetCombine(SetOperation.Difference, key, numKey);

Console.WriteLine(string.Join(",", values)); //4, 6, 7, 8, 9, 10

values = redis.SetCombine(SetOperation.Intersect, key, numKey);

Console.WriteLine(string.Join(",", values)); //1, 2, 3

//move a random from source numKey to aplhaKey

redis.SetMove(numKey, alphaKey, 2);

members = redis.SetMembers(alphaKey);

Console.WriteLine(string.Join(",", members)); //output can be (b, c, a, 2)

//Add apple to

redis.SetAdd(alphaKey, "apple");

//look for item that starts with he

var patternMatchValues = redis.SetScan(alphaKey, "ap*"); //output apple

Console.WriteLine(string.Join(",", patternMatchValues));

patternMatchValues = redis.SetScan(alphaKey, "a*"); //output apple, a

Console.WriteLine(string.Join(",", patternMatchValues));

//store into destinantion key the union of numKey and alphaKey

redis.SetCombineAndStore(SetOperation.Union, destinationKey, numKey, alphaKey);

Console.WriteLine(string.Join(",", redis.SetMembers(destinationKey)));

//radmon value and removes it from the set

var randomVal = redis.SetPop(numKey);

Console.WriteLine(randomVal); //random member in the set

Console.ReadKey();

}

So this covers the basic usage of Redish Set Datatype, in the next blog post I will cover using Sorted Sets Datatype.

For the code please visit
https://github.com/taswar/RedisForNetDevelopers/tree/master/6.RedisSet

For previous Redis topics

Redis

C#, redis, Redis Sets, StackExchange.Redis

Xamarin Dev Days

Redis for .NET Developer – Redis Transactions

Redis Transactions

C# code using Redis Transactions

Redis for .NET Developers – Redis Pipeline Batching

Redis Pipelining

C# code using Redis Pipeliine (TPL)

Redis Batching

C# code using Redis batching

Redis for .NET Developers – Redis Pub Sub

Sample Diagram of Redis Pub Sub

Redis Pub Sub – Operations

C# code using Redis PubSub

Upcoming Talks at DevTeach Montreal 2017

Introduction to Xamarin – speaking at Carleton

Docker for .NET Developers

Docker for .NET Developers

Redis for .NET Developers – Redis HyperLogLog Datatype

Redis HyperLogLog Datatype – Operations

C# code using Redis HyperLogLog Datatype

Redis for .NET Developers – Redis Sorted Sets Datatype

Redis Sorted Sets Datatype – Operations

C# code using Redis Sorted Set Datatype

Redis for .NET Developers – Redis Sets Datatype

Redis Sets Datatype – Operations

C# code using Redis Set Datatype

Opinions are my own and not the views of my employer

LinkedIn

Archives