Get Started with Visual Studio Code with Python Flask

Taswar Bhatti October 6, 2016 6 Comments

Visual Studio Code with Python Flask

I wanted to continue on my python exploration and show how to get started with Visual Studio Code with Python Flask.
What is flask you may ask

Flask is a microframework for Python based on Werkzeug, Jinja 2 and good intentions. And before you ask: It’s BSD licensed!

Basically you can think of Flask somewhat like NancyFx or Sinatra web framework.

First thing first, in python we should set up our virtualenv such that we have a dev environment and don’t pollute our global space with our packages. You should already have pip install with your python if you are using the Python 3.x series.

Open up your powershell or command prompt and type

1

2

3

4

5

6

PS C:\Users\flask-sample> pip install virtualenv

Collecting virtualenv

Downloading virtualenv-15.1.0-py2.py3-none-any.whl (1.8MB)

100% |################################| 1.8MB 325kB/s

Installing collected packages: virtualenv

Successfully installed virtualenv-15.1.0

Once you have virtualenv we will need virtualenvwrapper-win since we are running under windows.

1

2

3

4

5

6

7

8

PS C:\Users\flask-sample> pip install virtualenvwrapper-win

Collecting virtualenvwrapper-win

Downloading virtualenvwrapper-win-1.2.1.zip

Requirement already satisfied: virtualenv in c:\users\taswar\appdata\local\programs\python\python35-32\lib\site-packages

(from virtualenvwrapper-win)

Installing collected packages: virtualenvwrapper-win

Running setup.py install for virtualenvwrapper-win ... done

Successfully installed virtualenvwrapper-win-1.2.1

Now we need to create the virtualenv that we will be using.

1	PS C:\Users\flask-sample> mkvirtualenv FlaskProject

This will create your virtualenv inside your home directory for mine it is under

1	PS C:\Users\taswar\Envs\FlaskProject\

In order to activate the virtualenv we need to use the activate.ps1 script but you must make sure we have the correct execution-policy in your powershell but before we do that lets launch our Vscode

1

2

3

4

PS C:\Users\taswar\Documents\GitHub\>mkdir flaskExample

PS C:\Users\taswar\Documents\GitHub\>cd flaskExample

PS C:\Users\taswar\Documents\GitHub\flaskExample>code .

#Once in your vscode launch your command powershell prompt Ctrl+Shift+`

Lets now set up our directory to use the virtualenv, remember the Set-ExecutionPolicy for powershell and use the terminal inside your vscode with Ctrl+Shift+` (right tick)

1

2

3

4

5

6

7

8

9

10

11

12

13

PS C:\Users\taswar\Envs\FlaskProject\Scripts\Activate.ps1

#one can set the project also

(flaskProject) PS C:\Users\taswar\Documents\GitHub\flaskExample> setprojectdir .

"C:\Users\taswar\Documents\GitHub\flaskExample" is now the project directory for

virtualenv "C:\Users\taswar\Envs\flaskProject"

"C:\Users\taswar\Documents\GitHub\flaskExample" added to

C:\Users\taswar\Envs\flaskProject\Lib\site-packages\virtualenv_path_extensions.pth

#once active you will see your prompt change

(flaskProject) PS C:\Users\taswar\Documents\GitHub\flaskExample>

Lets now install flask

#now lets install flask
(flaskProject) PS C:\Users\taswar\Documents\GitHub\flaskExample>pip install flask
 Downloading Werkzeug-0.12.1-py2.py3-none-any.whl (312kB)
    100% |################################| 317kB 585kB/s
Collecting Jinja2>=2.4 (from flask)
  Downloading Jinja2-2.9.6-py2.py3-none-any.whl (340kB)
    100% |################################| 348kB 926kB/s
Collecting MarkupSafe>=0.23 (from Jinja2>=2.4->flask)
  Downloading MarkupSafe-1.0.tar.gz
Building wheels for collected packages: itsdangerous, MarkupSafe
  Running setup.py bdist_wheel for itsdangerous ... done
  Stored in directory: C:\Users\taswar\AppData\Local\pip\Cache\wheels\fc\a\66\24d655233c757e178d45dea2de22a04c6d92766abfb741129a
  Running setup.py bdist_wheel for MarkupSafe ... done
  Stored in directory: C:\Users\taswar\AppData\Local\pip\Cache\wheels\88\a7\30\e39a54a87bcbe25308fa3ca64e8ddc75d9b3e5afa21ee32d57
Successfully built itsdangerous MarkupSafe
Installing collected packages: click, itsdangerous, Werkzeug, MarkupSafe, Jinja2, flask
Successfully installed Jinja2-2.9.6 MarkupSafe-1.0 Werkzeug-0.12.1 click-6.7 flask-0.12.1 itsdangerous-0.24

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

#now lets install flask

(flaskProject) PS C:\Users\taswar\Documents\GitHub\flaskExample>pip install flask

Downloading Werkzeug-0.12.1-py2.py3-none-any.whl (312kB)

100% |################################| 317kB 585kB/s

Collecting Jinja2>=2.4 (from flask)

Downloading Jinja2-2.9.6-py2.py3-none-any.whl (340kB)

100% |################################| 348kB 926kB/s

Collecting MarkupSafe>=0.23 (from Jinja2>=2.4->flask)

Downloading MarkupSafe-1.0.tar.gz

Building wheels for collected packages: itsdangerous, MarkupSafe

Running setup.py bdist_wheel for itsdangerous ... done

Stored in directory: C:\Users\taswar\AppData\Local\pip\Cache\wheels\fc\a\66\24d655233c757e178d45dea2de22a04c6d92766abfb741129a

Running setup.py bdist_wheel for MarkupSafe ... done

Stored in directory: C:\Users\taswar\AppData\Local\pip\Cache\wheels\88\a7\30\e39a54a87bcbe25308fa3ca64e8ddc75d9b3e5afa21ee32d57

Successfully built itsdangerous MarkupSafe

Installing collected packages: click, itsdangerous, Werkzeug, MarkupSafe, Jinja2, flask

Successfully installed Jinja2-2.9.6 MarkupSafe-1.0 Werkzeug-0.12.1 click-6.7 flask-0.12.1 itsdangerous-0.24

Lets now create a hello.py example of flask application, remember spaces are important for python

1

2

3

4

5

6

from flask import Flask

app = Flask(__name__)

@app.route('/')

def hello_world():

return 'Hello, World!'

Go back to your powershell prompt and enter. (NOTE: you may need to restart vscode for this to work)

1

2

3

4

setx FLASK_APP = hello.py

(flaskProject) PS C:\Users\taswar\Documents\GitHub\flaskExample> python -m flask run

* Serving Flask app "hello"

* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

When everything is up and running you should be able to go to Chrome or your favourite browser and view port 5000 and see Hello world there.

VsCode-Python-Flask-1

Hope this helps you in setting up vscode with development of Python Flask.

To learn about running VSCode with python read my previous post on Getting Started with Visual Studio Code with Python.

VSCode

flask, python, vscode

Get Started with Visual Studio Code with Python

Taswar Bhatti September 20, 2016 No Comments

Visual Studio Code with Python

Visual Studio Code is the new kid on the block and it has lots of bells and whistle including using Visual Studio Code with Python.

Requirements

To install vscode, just go Visual Studio Code website. Its a very simple click through process if you are using Windows.

If you are using Linux, then you will need to download the .deb package and use the command to install it, replace the filename below.

1	$ sudo dpkg -i [vscode-package-version].deb

The other requirement is to download python, again its a simple process if you are using Windows, just click through. I wont go through the python install process. Just go through Python Download section. I personally use the 3.x series, since I like being latest 🙂

Once you have visual studio code installed you can launch vscode and click on extension tab and search for python.

vscode-python-1

I use the Don Jayamanne Python extension, and then click on install.

vscode-python-2

Once the plugin is installed you can reload vscode.

Open up a new file and name it sample.py and type the following code into it. Remember spaces in python are important

1 2	for i in range(0, 10): print("hello" + str(i))

We can then click on the debug tab and debug the python code, lets put a breakpoint in the print statement, one can also add a Python configuration. When you debug you will be able to see your variables values also like below.

vscode-python-3

Hope this will help you in getting python running in VsCode.

VSCode

python, vcode

Generating fake data in Node.js with Faker.js

Taswar Bhatti August 2, 2016 No Comments

Last time I did a sample app to generate large amount of data using GenFu in C# so that I can be in .NET land but I wanted to also try out what it would be like to generate fake data in node.js using VSCode.

My investigation lead me to Faker.js, which would allow me to generate fake data in Node.js with Faker.js

Installation

To install faker.js you just need to use npm package manager

1	npm install faker --save-dev

Usage

I will use a similar kind of example I had used previously to generate logs, below is the sample code I used to generate, it is a bit different but the idea is the same.

var faker = require('faker');
var uuidV4 = require('uuid/v4');

var log = {
  "v": "1.0",
  "category": "ERRORLOG",
  "level": "INFO",
  "timestamp": "2016-11-04T22:02:02.365Z",
  "application": {
    "name": "My Fav application",
    "version": "1.0.0-SNAPSHOT",
    "thread": "default task-16",
    "component": "com.AuditLogInterceptor"
  },
  "context": {
    "tenantId": "saml-demo",
    "principalId": "griduser",
    "userGuid" : "c92c7642-777a-41a2-87d6-79a8acec1d7f",
    "sessionId": "123456789",  
    "globalAccessId": "593b740b-d53e-490a-b5d1-fb3fa2fc1c4c",
    "policyId": "c92c7642-777a-41a2-87d6-79a8acec1d7f",
    "policyName": "Default",
    "scenarioId": "0011c00d-b48b-b82c-4a66-1d394b64ff67",
    "scenarioName": "trusted network",
    "applicationId" : "c92c7642-777a-41a2-87d6-79a8acec1d7f",
    "applicationName" : "Office 365",
  },
  "details": {
    "type": "ACCESS REQUEST",
    "state": "",
    "action" : "",
    "credentials" : [
        {
            type: "OTP",
            state: "" 
        },
        {
            type: "Password",
            state: ""
        }
    ]
  }
}

var state = ["Initiated", "Accepted", "Denied" ,"Waiting"];
var action = ["auth", "change password", "push", "restart"];
var cred_state = ["Pending", "VerifiedInSession","Verified" ];

log.details.state = state[state_randon];
  log.details.action = action[action_random];
  log.details.credentials[0].state = cred_state[cred_random];
  log.details.credentials[1].state = cred_state[cred_random2];
  
  console.log("%j", log);
}

genlog();

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

var faker = require('faker');

var uuidV4 = require('uuid/v4');

var log = {

"v": "1.0",

"category": "ERRORLOG",

"level": "INFO",

"timestamp": "2016-11-04T22:02:02.365Z",

"application": {

"name": "My Fav application",

"version": "1.0.0-SNAPSHOT",

"thread": "default task-16",

"component": "com.AuditLogInterceptor"

},

"context": {

"tenantId": "saml-demo",

"principalId": "griduser",

"userGuid" : "c92c7642-777a-41a2-87d6-79a8acec1d7f",

"sessionId": "123456789",

"globalAccessId": "593b740b-d53e-490a-b5d1-fb3fa2fc1c4c",

"policyId": "c92c7642-777a-41a2-87d6-79a8acec1d7f",

"policyName": "Default",

"scenarioId": "0011c00d-b48b-b82c-4a66-1d394b64ff67",

"scenarioName": "trusted network",

"applicationId" : "c92c7642-777a-41a2-87d6-79a8acec1d7f",

"applicationName" : "Office 365",

},

"details": {

"type": "ACCESS REQUEST",

"state": "",

"action" : "",

"credentials" : [

{

type: "OTP",

state: ""

},

{

type: "Password",

state: ""

}

]

}

var state = ["Initiated", "Accepted", "Denied" ,"Waiting"];

var action = ["auth", "change password", "push", "restart"];

var cred_state = ["Pending", "VerifiedInSession","Verified" ];

function genlog() {

log.timestamp = faker.date.recent();

log.context.tenantId = faker.company.companyName().replace(/\s+/g, ''); //remove space in name

log.context.principalId = faker.internet.userName();

log.context.userGuid = uuidV4(); //generate guid

log.context.sessionId = uuidV4();

log.context.globalAccessId = uuidV4();

log.context.policyId = uuidV4();

log.context.policyName = faker.commerce.productName();

log.context.scenarioId = uuidV4();

log.context.scenarioName = faker.commerce.productName();

log.context.applicationId = uuidV4();

log.context.applicationName = faker.commerce.productName();

var state_randon = faker.random.number({min:0, max:state.length});

var action_random = faker.random.number({min:0, max:action.length});

var cred_random = faker.random.number({min:0, max:cred_state.length});

var cred_random2 = faker.random.number({min:0, max:cred_state.length});

log.details.state = state[state_randon];

log.details.action = action[action_random];

log.details.credentials[0].state = cred_state[cred_random];

log.details.credentials[1].state = cred_state[cred_random2];

console.log("%j", log);

}

genlog();

As you can see faker.js is also quite easy to use and generate fake data, although in GenFu we were able to generate a List of objects, I was not able to find such functionality in faker.js but one nice thing faker.js had was localization where it can generate data in different languages.

Faker.js also has quite a few api methods to generate data take a look at their github page for all the methods https://github.com/marak/Faker.js/.

Check out faker.js if you are using node.js and need to generate large amount of fake data.

Node, node.js

faker.js, node, node.js

Redis for .NET Developers – Redis List Datatype

Taswar Bhatti July 21, 2016 2 Comments

Redis List Datatype are similar in C# world as LinkeList e.g LinkedList. Just like in LinkedList in C#, Redis can store items at Head or Tail of a List. The nice thing about them is that the insertion speed is just O(1), but do note that if you have a large list, the reading speed goes down since it sequentially traverses through the items (i.e walking down the chain). One could also say its an advantage since Redis was made to write faster than read, and LinkedList are great data type for such things. List are ideal candidate for logs, since the write speed is more important than read when dealing with logs.

Redis List Datatype – Operations

LPUSH: Prepends the values to the left of the list O(1).
RPUSH: Prepends the values to the right of the list O(1).
LPUSHX: Prepends the values to the left of the list if key exist O(1).
RPUSHX: Prepends the values to the right of the list if key exist O(1).
LINSERT: Inserts a value in the list after position calculated from left O(N).
LSET: Sets the value of an element in a list based on index O(N).
LRANGE: Gets the sub list of elements based on start and end position O(S+E).
LTRIM: Deletes the elements outside the range specified O(N), where N is the length.
RPOP: Removes the last element O(1).
LREM: Removes the element at the index point O(N) , where N is the length.
LPOP: Removes the first element of the list O(1).
LINDEX: Gets the element from the list based on the index O(N) where N is the length of traverse.
LLEN: This command gets the length of the list O(1).
RPOPLPUSH: Operates on two lists source and destination list, takes the last element on the source list and push it to the first element of the destination list O(1).

C# code using Redis List Datatype

class Program
    {
        static void Main(string[] args)
        {
            var redis = RedisStore.RedisCache;

var listKey = "listKey";

redis.KeyDelete(listKey, CommandFlags.FireAndForget);

redis.ListRightPush(listKey, "a");

var len = redis.ListLength(listKey);
            Console.WriteLine(len); //output  is 1

redis.ListRightPush(listKey, "b");

Console.WriteLine(redis.ListLength(listKey)); //putput is 2

//lets clear it out
            redis.KeyDelete(listKey, CommandFlags.FireAndForget);
          
            redis.ListRightPush(listKey, "abcdefghijklmnopqrstuvwxyz".Select(x => (RedisValue) x.ToString()).ToArray());

Console.WriteLine(redis.ListLength(listKey)); //output is 26

Console.WriteLine(string.Concat(redis.ListRange(listKey))); //output is abcdefghijklmnopqrstuvwxyz

var lastFive = redis.ListRange(listKey, -5);

Console.WriteLine(string.Concat(lastFive)); //output vwxyz

var firstFive = redis.ListRange(listKey, 0, 4);

Console.WriteLine(string.Concat(firstFive)); //output abcde

redis.ListTrim(listKey, 0, 1);

Console.WriteLine(string.Concat(redis.ListRange(listKey))); //output ab

//lets clear it out
            redis.KeyDelete(listKey, CommandFlags.FireAndForget);

redis.ListRightPush(listKey, "abcdefghijklmnopqrstuvwxyz".Select(x => (RedisValue)x.ToString()).ToArray());

var firstElement = redis.ListLeftPop(listKey);

Console.WriteLine(firstElement); //output a, list is now bcdefghijklmnopqrstuvwxyz

var lastElement = redis.ListRightPop(listKey);

Console.WriteLine(lastElement); //output z, list is now bcdefghijklmnopqrstuvwxy

redis.ListRemove(listKey, "c");

Console.WriteLine(string.Concat(redis.ListRange(listKey))); //output is bdefghijklmnopqrstuvwxy   
            
            redis.ListSetByIndex(listKey, 1, "c");

Console.WriteLine(string.Concat(redis.ListRange(listKey))); //output is bcefghijklmnopqrstuvwxy

var thirdItem = redis.ListGetByIndex(listKey, 3);

Console.WriteLine(thirdItem); //output f

//lets clear it out
            var destinationKey = "destinationList";
            redis.KeyDelete(listKey, CommandFlags.FireAndForget);
            redis.KeyDelete(destinationKey, CommandFlags.FireAndForget);

redis.ListRightPush(listKey, "abcdefghijklmnopqrstuvwxyz".Select(x => (RedisValue)x.ToString()).ToArray());

var listLength = redis.ListLength(listKey);

for (var i = 0; i < listLength ; i++)
            {
                var val = redis.ListRightPopLeftPush(listKey, destinationKey);

Console.Write(val);    //output zyxwvutsrqponmlkjihgfedcba
            }
           
            Console.ReadKey();
        }
    }

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

class Program

{

static void Main(string[] args)

{

var redis = RedisStore.RedisCache;

var listKey = "listKey";

redis.KeyDelete(listKey, CommandFlags.FireAndForget);

redis.ListRightPush(listKey, "a");

var len = redis.ListLength(listKey);

Console.WriteLine(len); //output is 1

redis.ListRightPush(listKey, "b");

Console.WriteLine(redis.ListLength(listKey)); //putput is 2

//lets clear it out

redis.KeyDelete(listKey, CommandFlags.FireAndForget);

redis.ListRightPush(listKey, "abcdefghijklmnopqrstuvwxyz".Select(x => (RedisValue) x.ToString()).ToArray());

Console.WriteLine(redis.ListLength(listKey)); //output is 26

Console.WriteLine(string.Concat(redis.ListRange(listKey))); //output is abcdefghijklmnopqrstuvwxyz

var lastFive = redis.ListRange(listKey, -5);

Console.WriteLine(string.Concat(lastFive)); //output vwxyz

var firstFive = redis.ListRange(listKey, 0, 4);

Console.WriteLine(string.Concat(firstFive)); //output abcde

redis.ListTrim(listKey, 0, 1);

Console.WriteLine(string.Concat(redis.ListRange(listKey))); //output ab

//lets clear it out

redis.KeyDelete(listKey, CommandFlags.FireAndForget);

redis.ListRightPush(listKey, "abcdefghijklmnopqrstuvwxyz".Select(x => (RedisValue)x.ToString()).ToArray());

var firstElement = redis.ListLeftPop(listKey);

Console.WriteLine(firstElement); //output a, list is now bcdefghijklmnopqrstuvwxyz

var lastElement = redis.ListRightPop(listKey);

Console.WriteLine(lastElement); //output z, list is now bcdefghijklmnopqrstuvwxy

redis.ListRemove(listKey, "c");

Console.WriteLine(string.Concat(redis.ListRange(listKey))); //output is bdefghijklmnopqrstuvwxy

redis.ListSetByIndex(listKey, 1, "c");

Console.WriteLine(string.Concat(redis.ListRange(listKey))); //output is bcefghijklmnopqrstuvwxy

var thirdItem = redis.ListGetByIndex(listKey, 3);

Console.WriteLine(thirdItem); //output f

//lets clear it out

var destinationKey = "destinationList";

redis.KeyDelete(listKey, CommandFlags.FireAndForget);

redis.KeyDelete(destinationKey, CommandFlags.FireAndForget);

redis.ListRightPush(listKey, "abcdefghijklmnopqrstuvwxyz".Select(x => (RedisValue)x.ToString()).ToArray());

var listLength = redis.ListLength(listKey);

for (var i = 0; i < listLength ; i++)

{

var val = redis.ListRightPopLeftPush(listKey, destinationKey);

Console.Write(val); //output zyxwvutsrqponmlkjihgfedcba

}

Console.ReadKey();

}

So this covers the basic usage of Redish List Datatype, in the next blog post I will cover using Sets Datatype.

For the code please visit
https://github.com/taswar/RedisForNetDevelopers/tree/master/5.RedisList/RedisList

For previous Redis topics

Redis for .NET Developer – Hash Datatype

Redis

.net, C#, redis, StackExchange.Redis

Redis for .NET Developers – Redis Password

Taswar Bhatti June 10, 2016 No Comments

I wanted to cover some security aspect of Redis, using Redis Password feature. Redis uses config file to provide a password for it. The config file is usually located at /etc/redis/redis.conf if you are using linux. Look for the SECURITY section.

1	requirepass taswar-foobared

If you are using docker you can set the option when you start the docker container.

1	docker run --name my-redis -d redis redis-server --requirepass taswar-foobared

Afterwards if you are trying to connect to redis using redis-cli you will not be able to launch any commands in it,

1

2

3

# redis-cli

127.0.0.1:6379> set x hello

(error) NOAUTH Authentication required.

In order to set the value x, one will need to authenticate

1

2

3

4

5

6

127.0.0.1:6379> auth taswar-foobared

OK

127.0.0.1:6379> set x hello

OK

127.0.0.1:6379> get x

"hello"

If you are using StackExchange.Redis you can use the sample code below to connect, as you can see I am just adding the password to the connection string.

1

2

3

4

5

6

7

8

9

10

11

12

private static Lazy<ConnectionMultiplexer> lazyConnection = new Lazy<ConnectionMultiplexer>(() =>

{

return ConnectionMultiplexer.Connect("localhostEndpoint,password=taswar-foobared");

});

public static ConnectionMultiplexer Connection

{

get

{

return lazyConnection.Value;

}

For other Redis topics

.NET, Redis

.net, redis, StackExchange.Redis

Generating data in dotnet core with GenFu

Taswar Bhatti June 6, 2016 No Comments

dotnet core with GenFu

Couple of weeks ago I had to test out the performance of a third party product but in order to test we had to use a lot of logs that would feed into the system. The requirements was to pump out around 1000 logs per seconds and feed into the system.

As you know I am a .NET fan wanted to write this small program in dotnet core C# and was looking for a library that would generate me some fake data but something realistic at the same time. I found GenFu developed by James and Dave, 2 fellow Canadian Microsoft MVP.

The description of GenFu is provided below:

GenFu is a library you can use to generate realistic test data. It is composed of several property fillers that can populate commonly named properties through reflection using an internal database of values or randomly created data. You can override any of the fillers, give GenFu hints on how to fill them.

Installation of GenFu

GenFu comes in NuGet package thus the installation process is as simple as

1	Install-Package GenFu

Usage

My usage was quite simple to generate log files. Thus I started by creating an Object that would contain all my data.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

class LogActivity

{

public Guid UserId { get; set;}

public int TenantId { get; set; }

public string TenantUsed { get; set;}

public Guid Serial { get; set;}

public int Action { get; set;}

public int Result { get; set;}

public string SourceIp { get; set;}

public int AgentId { get; set;}

public string Message { get; set;}

public DateTime ActDate { get; set;}

public string SiteHost { get; set;}

public string UsedName { get; set;}

public string CredentialType { get; set;}

}

GenFu provides an easy way to fill the data like below

1

2

3

var singleLog = A.New<LogActivity>();

//or generate a list of logs

var thousandLogs = A.ListOf<LogActivity>(1000);

But my requirements had certain criteria and GenFu allows you to use Fillers (PropertyFiller) to fill in data to your needs i.e a way to override its default values. In my case I wanted to override multiple fields, an example below of override of TenantName

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

//First call FillerManager

A.Default().FillerManager.RegisterFiller(new TenantNameFiller());

//the class is TenantNameFiller

public class TenantNameFiller: PropertyFiller<string>

{

protected static Random _random = new Random(Environment.TickCount);

private string[] Company = new string[] {"AGT", "KIO", "GGR", "Smith", "Orion","GMC", "Ford", "Hundai", "Pacific Atlantic", "Mother's", "Frank's", "Berkshire Hathaway", "Pepsi", "Coca-cola"};

//second value is the property name to fill

public TenantNameFiller(): base(new[] {"object"}, new [] {"TenantName", "TenantNameUsed"})

{

}

public override object GetValue(object instance)

{

//random value from array

return Company[_random.Next(0, Company.Length)];

}

With the code above and the setup I was able to generate data that was very realistic to my needs. GenFu also provides Fill to add Ranges to your data or Email addresss like below:

1

2

3

4

5

6

7

A.Configure<LogActivity>().

Fill(x => x.Result).WithinRange(0, 8).

Fill(x => x.AgentId).WithinRange(1, 23).

Fill(x => x.UsedName).AsEmailAddress().

Fill(x => x.ActDate, () => { return DateTime.Now;});

var activities = A.ListOf<LogActivity>(1000);

Thanks to GenFu I was able to create some interesting and realistic data to feed into the system. Do check out the library 🙂

.NET

.net core, C#, genfu

Redis for .NET Developers – Redis running in AWS ElastiCache

Taswar Bhatti May 20, 2016 5 Comments

I wanted to go through the process of Redis running in AWS ElastiCache, since we have already covered Docker, Azure and plain vanilla Redis on Linux Server. Redis in AWS is under the umbrella of ElastiCache, where they offer Redis and Memcached.

There are different type of nodes offered by AWS ElastiCache:

Standard
Memory Optimized

I wont be going through the AWS offering, since AWS ElastiCache does a good job in explaining the differences between the Nodes.

First you will need to login to your AWS console, once logged in.

Click on the Database Section of ElastiCache

AWS-ElastiCache
Select the Redis option on the Dashboard Menu

AWS-ElastiCache-2
Click on Create

AWS-ElastiCache-3
Select the node that you wish to use. Here I am just selecting small, but feel free to choose which one serves you the best.

AWS-ElastiCache-4
Select Redis and fill in the Name of the cluster and click Create.

AWS-ElastiCache-5
The process takes couple of minutes and will complete

AWS-ElastiCache-6
Once complete you will have a AWS domain name to connect to your Redis

AWS-ElastiCache-7

AWS does not have the option to see ElastiCache Redis running within its Console like Azure, but one can connect to the endpoint the same way you connect to any Redis instance. As we saw there is a default hostname that is given by AWS to connect to, there is no secret/key pair provided like Azure. The access to the ElastiCache Redis instance is configured through AWS Security Groups, just like connecting to your RDS, EC2 instance.

There are two important items to note when running AWS ElastiCache with Redis:

One cannot set password on the Redis instance. Any EC2 instance in the same security group will have access to your Redis instance.
Connection to the ElastiCache instance is only within the region/VPC

When using StackExchange.Redis you can connect to the Redis instance, the information to connect to AWS ElastiCache is located in the AWS Console under your Redis instance running, one can use the sample code below to connect. (Note: AWS uses Security Group to connect)

1

2

3

4

5

6

7

8

9

10

11

12

private static Lazy<ConnectionMultiplexer> lazyConnection = new Lazy<ConnectionMultiplexer>(() =>

{

return ConnectionMultiplexer.Connect("whatever-redis.bd1zu2.0009.usw2.cache.amazonaws.com:6379");

});

public static ConnectionMultiplexer Connection

{

get

{

return lazyConnection.Value;

}

For other Redis topics

.NET, AWS, Redis

aws, elastiCache, redis

Redis for .NET Developers – Redis running in Azure

Taswar Bhatti May 5, 2016 No Comments

In this post I wanted to go through the process of Redis running in Azure. Redis in Azure is called Azure Redis Cache. Azure offers a secure access to a dedicated Redis cache, which allows you to connect from any Azure applications.

There are certain tiers that are offered in Azure Redis:

Basic
Standard
Premium

I wont be going through each of them, since Azure does a good job in explaining the differences between the tiers but for this post we will be using the Basic.

First you will need to login to your Azure portal.
Once logged in.

Click on New (+) sign, and it will open up the New Section
Select the Databases option
Select Redis Cache
redis-azure1

Once you have selected the right option, then it is just 5 clicks away to create Redis

Select a DNS name that is not used.
Select your Resource Group.
Select your Location.
Select your pricing tier (I have selected Basic here, but feel free to choose the one that satisifies your needs)
Click on Create.
redis-azure2
It takes a bit of time to create it, but you can see the progress.
redis-azure3

One can access Redis using access keys and one can test it by launching the console.

Azure-Redis-Cache

The Console allows one to use Redis commands.

Azure-Redis-Preview-Console

When using StackExchange.Redis you can connect to the Redis instance, the information to connect to Azure is located in the Azure portal under Settings > Properties, one can use the sample code below to connect.

1

2

3

4

5

6

7

8

9

10

11

12

private static Lazy<ConnectionMultiplexer> lazyConnection = new Lazy<ConnectionMultiplexer>(() =>

{

return ConnectionMultiplexer.Connect("whatever.redis.cache.windows.net,abortConnect=false,ssl=true,password=...");

});

public static ConnectionMultiplexer Connection

{

get

{

return lazyConnection.Value;

}

For other Redis topics

.NET, Azure, Redis

azure, redis

Redis for .NET Developers – Redis running in Docker

Taswar Bhatti April 4, 2016 No Comments

To get Redis running in Docker is quite easy. The windows installation follows but if you on the Linux platform, first you will need to install docker. Docker has very good documentation on how to get docker running on Linux. I personally use Ubuntu so would suggest you to give Ubuntu a try for Docker.

Once docker is installed, all it takes from command prompt is to run

1	$ docker pull redis

Then to run redis

1	$ docker run --name my-redis -d redis

If you are on the windows 10 installing Docker is also a breeze platform, and to install Redis you can use Kitematic.

The steps to install Redis with Kitematic.

Launch Kitematic
kitematic-windows
Search for Redis
Install the official release of redis by clicking on Create
docker-redis-2
Done
docker-redis-3

Now one can connect to localhost port 32768 with Redis. Docker makes it super easy to get Redis up and running.

For other Redis topics

docker, Redis

docker, redis

Redis for .NET Developer – Redis Hash Datatype

Taswar Bhatti March 3, 2016 10 Comments

Redis Hash Datatype are similar in C# world as Dictionary e.g Dictionary<string, string>. Just like in C# redis stores map of attributes using key value pair. One thing to note is in Redis a Hash both the field name and the value are strings. Therefore, a Hash Datatype is a mapping of a string to a string.

Memory Optimized

Redis Hashes are memory-optimized. Their optimization is based on the hash-max-ziplist-entries and hash-max-ziplist-value configurations. Internally in Redis, a Hash can be a ziplist or a hash table. So what does it mean for a ziplist? Basically it is designed to be memory efficient dually linked list and integers are stored as real integers rather than a sequence of characters.

Speed

Something to note is a hashtable in Redis has a constant time lookup, the down side is it is not memory optimized, while the ziplist is memory optimized but the lookup time are not constant. To read more on Redis Memory Optimization please view Redis documentation.

So which one is used when?

The ziplist is by default when the number of fields do not exceed the configuration ones in hash-max-ziplist-entries
The hashtable is used when a the size or any of its values exceeds

Redis Hash Datatype – Operations

HSET: Sets the value of a field for a key O(1).
HGET: Gets the value of a field for a key O(1).
HLEN: Gets the number of fields for the key O(1).
HMGET: Gets the values for the fields for a key O(N), where N is the number of fields and O (1) if N is small.
HMSET: Sets multiple values for respective fields for a key O(N), where N is the number of fields and O (1) if N is small.
HGETALL: Gets all the values and fields for a key O(N). Where N is the size of the hash
HKEYS: Gets all the fields in the Hash for the key O(1).
HEXISTS: Checks for the existence of a field for a key O(1).
HVALS: Gets all the values in the Hash for the key O(N), where N is the number of fields and O(1) if N is small.
HSETNX: Sets the value against the field for the key provided the field does not exist O(1).
HDEL: Deletes the fields for a key O (N), where N is the number of fields and O(1) if N is small.
HINCRBY: Increments the value (provided the value is an integer) of a field for a key O(1).
HINCRBYFLOAT: Increments the value if value is a float of a field for a key O(1).

C# code using Redis Hash Datatype

var hashKey = "hashKey";

HashEntry[] redisBookHash = {
    new HashEntry("title", "Redis for .NET Developers"),
    new HashEntry("year", 2016),
    new HashEntry("author", "Taswar Bhatti")
  };

redis.HashSet(hashKey, redisBookHash);

if (redis.HashExists(hashKey, "year"))
  {
    var year = redis.HashGet(hashKey, "year"); //year is 2016
  }

var allHash = redis.HashGetAll(hashKey);

//get all the items
  foreach (var item in allHash)
  {
    //output 
    //key: title, value: Redis for .NET Developers
    //key: year, value: 2016
    //key: author, value: Taswar Bhatti
    Console.WriteLine(string.Format("key : {0}, value : {1}", item.Name, item.Value));
  }

//get all the values
  var values = redis.HashValues(hashKey);

foreach (var val in values)
  {
    Console.WriteLine(val); //result = Redis for .NET Developers, 2016, Taswar Bhatti
  }

//get all the keys
  var keys = redis.HashKeys(hashKey);

foreach (var k in keys)
  {
    Console.WriteLine(k); //result = title, year, author
  }

var len = redis.HashLength(hashKey);  //result of len is 3
            
  if (redis.HashExists(hashKey, "year"))
  {
    var year = redis.HashIncrement(hashKey, "year", 1); //year now becomes 2017
    var year2 = redis.HashDecrement(hashKey, "year", 1.5); //year now becomes 2015.5
  }

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

var hashKey = "hashKey";

HashEntry[] redisBookHash = {

new HashEntry("title", "Redis for .NET Developers"),

new HashEntry("year", 2016),

new HashEntry("author", "Taswar Bhatti")

};

redis.HashSet(hashKey, redisBookHash);

if (redis.HashExists(hashKey, "year"))

{

var year = redis.HashGet(hashKey, "year"); //year is 2016

}

var allHash = redis.HashGetAll(hashKey);

//get all the items

foreach (var item in allHash)

{

//output

//key: title, value: Redis for .NET Developers

//key: year, value: 2016

//key: author, value: Taswar Bhatti

Console.WriteLine(string.Format("key : {0}, value : {1}", item.Name, item.Value));

}

//get all the values

var values = redis.HashValues(hashKey);

foreach (var val in values)

{

Console.WriteLine(val); //result = Redis for .NET Developers, 2016, Taswar Bhatti

}

//get all the keys

var keys = redis.HashKeys(hashKey);

foreach (var k in keys)

{

Console.WriteLine(k); //result = title, year, author

}

var len = redis.HashLength(hashKey); //result of len is 3

if (redis.HashExists(hashKey, "year"))

{

var year = redis.HashIncrement(hashKey, "year", 1); //year now becomes 2017

var year2 = redis.HashDecrement(hashKey, "year", 1.5); //year now becomes 2015.5

}

For the code please visit
https://github.com/taswar/RedisForNetDevelopers/tree/master/4.RedisHashes

So this covers the basic usage of Redish Hash Datatype, in the next blog post I will cover using List Datatype.

For previous Redis topics

.NET, Learn Series, Redis

C#, hash, redis, StackExchange.Redis

Get Started with Visual Studio Code with Python Flask

Visual Studio Code with Python Flask

Get Started with Visual Studio Code with Python

Visual Studio Code with Python

Requirements

Generating fake data in Node.js with Faker.js

Installation

Usage

Redis for .NET Developers – Redis List Datatype

Redis List Datatype – Operations

Redis for .NET Developers – Redis Password

Generating data in dotnet core with GenFu

dotnet core with GenFu

Installation of GenFu

Usage

Redis for .NET Developers – Redis running in AWS ElastiCache

Redis for .NET Developers – Redis running in Azure

Redis for .NET Developers – Redis running in Docker

Redis for .NET Developer – Redis Hash Datatype

Memory Optimized

Speed

Redis Hash Datatype – Operations

Opinions are my own and not the views of my employer

LinkedIn

My Book

Archives