Taswar Bhatti
The synonyms of software simplicity

How to create AWS Budget

Taswar Bhatti May 12, 2019 No Comments

For anyone starting new on Amazon they might want to start with a small aws budget on a their free trial to learn amazon. Even if you are a small to large company you will want to have a budget for your cloud services.
Amazon provides an easy way to alert and to create a budget for your cloud.
Lets get started on how to create a budget

Step by Step – How to create a AWS Budget in Amazon Web Services

First login to amazon console and then click on my account and you will see My Billing Dashboard in the drop down like below.

My Billing Dashboard Menu

Once inside the page, on the left hand you will see the menu for the section, click on budget.
aws-my-billing-budget-menu

aws-my-billing-budget-menu

Then click on the Create a Budget blue button.
aws-budgets

create aws budgets

I will create a cost budget so I have selected the cost budget and clicked on Set your budget
aws-cost-budget

aws-cost-budget

Then I have to set a name for the budget below I have set it as aws testing budget, it is set up as monthly and recurring.
aws-set-your-budget

aws-set-your-budget

I have set it as a fixed budget with a 10 dollar limit.
aws-fixed-budget-amount

aws-fixed-budget-amount

Afterwards you can click on Configure Alerts to configure when to alert you when the budget is close to a certain percentage or amount
aws-config-budget-alerts

aws-config-budget-alerts

I have set up the alert to 10% of the amount which is just 1 dollar, so aws will notify me on the email provided below when I am at 1 dollar
aws-budget-alert

aws-budget-alert

It will then confirm your budget on a page and you can see the details in a table
aws-budget-table

aws-budget-table

Summary

I hope this helps you in setting up your budget in aws when you are trying out aws or when you want to set up a budget for your company.

AWS
,
Taswar_Bhatti_VS2019_With_Python

Getting productive with Python on Visual Studio 2019

Taswar Bhatti May 2, 2019 No Comments

I had a talk on Getting productive with Python on Visual Studio 2019, at the Ottawa .NET User Group (https://www.meetup.com/ottawaitcommunity/events/259081628/). Thanks to all the people who came to the talk, I wish my demo worked properly unfortunately sometimes things just dont work 🙂

Nevertheless here are the slides for the talk.

Public Speaking, python, visual studio
,
8 Cloud Design Patterns Update Conference

8 Cloud Design Pattern you ought to know – Update Conference Prague Nov 2018

Taswar Bhatti April 19, 2019 No Comments

Here is the youtube recording for my 8 Cloud Design Pattern you ought to know talk at update conference in Prague Nov 2018. Hope you like it.

Cloud, conference
,
Javascript

FowardJS Ottawa – Cloud Design Patterns

Taswar Bhatti April 11, 2019 No Comments

Thanks to the organizers of ForwardJS Ottawa to let me speak on Cloud Design Patterns using Nodejs, I had a great time at the conference for anyone who is looking for my sides there are located right below.
If anyone wants to view the demo code I showed please look at this blog post on using Promise in Nodejs for Retry Pattern.

conference, node.js
, ,
retry-pattern-nodejs-with-promise

Retry Pattern in Node.js using Promise

Taswar Bhatti April 2, 2019 No Comments

One of the easiest cloud design pattern that one can try out is the Retry Pattern. I wanted to show how to use an Retry Pattern in Node.js using Promise as a example. So what does the Retry Pattern achieves?

Problem Statement – What is the issue the pattern solves?

When building applications you always have some sort of outside/external service including another MicroService that you have to consume or call. Sometimes there could be momentary loss of network connectivity, or a temporary unavailability, or timeouts that occur when that service is busy. You may be calling a database or a restful service that may be busy and fail but if you try back again it will pass. These types of faults are usually self-correcting, and most of the time require some type of delay in calling it again, which will have a success response.

Retry Pattern

  • Enable an application to handle transient failures
  • When the applications tries to connect to a service or network resource
  • By transparently retrying a failed operation
  • Improves the stability of your application

    • Typical Application

    Below is a typical application diagram, where you a service or web app.

    TypicalApplication

    TypicalApplication

    But when the connection to the service fails we usually get an error on our application.

    TypicalApplication-Network-Failure

    Typical-Application-Network-Failure

    When to use Retry Pattern

    • Use retry for only transient failure that is more than likely to resolve themselves quickly
    • Match the retry policies with the application
    • Otherwise use the circuit break pattern

    When not to use Retry Pattern

    • Don’t cause a chain reaction to all components
    • For internal exceptions caused by business logic
    • Log all retry attempts to the service

    Sample Code

    Below is a sample in node.js that shows the usage using Promise in Node.js. The code tries to call https://httpbin.org/status/200,408 with a POST which gives us a status of 200 or 408 randomly. First, lets create our code and add the package fetch into it.

    Without Promise

    We will write a sample application that will call the the web service without retry to get 408 errors.

    I am just using a console logger but you should be using a proper logger when you do retry pattern.

    After couple of runs you will see it response back with 408 RequestTimeout

    Using Retry with Promise

    Now we will introduce the retry pattern with using Promise into our code with an incremental delay of 1 second to 3 seconds and lastly 9 seconds.

    Output

    Below you will see three runs of the application with sample output.

    Summary

    As you can see Retry Pattern is quite useful for transient and self correcting failure, not to mention it is quite simple to implement in NodeJS with the help of Promise.

Cloud, node.js
, ,
update_conference_taswar_bhatti

Managing your secrets in a cloud environment – UpdateConf Prague 2018

Taswar Bhatti March 11, 2019 No Comments

Here is the youtube version of my presentation that I did in Prague at Update Conference. You can now watch my entire presentation. Thanks to Update Conference for doing such a fantastic job.

Enjoy.

conference
, ,
DevTeach Mntreal Speaker

Speaking at ForwardJS Ottawa April

Taswar Bhatti February 12, 2019 No Comments

I will be speaking at forwardjs Ottawa on April 10th and 11th 2019. I will be doing 8 Cloud Design Pattern you ought to know but more specific to using Node.js. There should be a video for it months later that I hope to share.

If you wish to watch my last year talk on Using Vault for your Nodejs Secrets

To purchase tickets for the session check out https://ti.to/forwardjs/forwardjs-ottawa-2019

Cloud, conference, node.js
,
Microsoft Ignite Tour - Taswar Bhatti

Microsoft Ignite Tour – 8 Cloud Design Patterns you ought to know

Taswar Bhatti January 14, 2019 No Comments

On January 11 2019, I had the opportunity to speak at Toronto Microsoft Ignite Tour – 8 Cloud Design Patterns you ought to know. It was a great seeing that over 700+ people registered to my talk, it was pretty much full house session over 500+ attended. Unfortunately Microsoft did not record any session in Toronto. I have had people ask me of a recording, although I do remember a gentleman who had his camera and was recording all my slides. (If someone knows him please feel free to ask him to get in touch with me, I would also love a copy of it also).

Attendees Review

One benefit of speaking at Ignite is they have a good evaluation system where users can give feedback.
So far the feedback has been very good. Here are the result of the evaluation score.

Ignite - Taswar Bhatti Review

Ignite – Taswar Bhatti Review

Attendee Comments

Some comments that I received are below. I actually wanted to speak slower but knowing the fact that I have 8 topics to go through and 90 slides in total with 2 demos, would rather like to cover all topics than missing any. In fact I covered 9 patterns with a bonus pattern,

Ignite - Taswar Bhatti Review Comments

Ignite – Taswar Bhatti Review Comments

Attendee Suggested Improvements

Its sometimes hard to satisfy all people in a talk, with over 500 people there will be someone who doesn’t like your talk. You cannot make everyone happy but I think I did make it clear sometimes patterns are like “Duh” moments where it is just common sense. Overall I still think people learned something new and enjoyed the talk. Below are some of the improvements.

Ignite - Taswar Bhatti Review Improvement

Ignite – Taswar Bhatti Review Improvement

Slides 8 Cloud Design Patterns you ought to know

Last but not least here are the slides from the talk. Enjoy…..

Cloud, conference, Public Speaking
,
update_conference_managing_cloud_secrets

Managing your secrets in a cloud environment – Update Conference

Taswar Bhatti November 25, 2018 No Comments

In November I did a presentation at the Update Conference 2018 on Managing your secrets in a cloud environment using Azure Key Vault and Hashicorp Vault. Comparing both products and demo on using Key Vault to store your keys and secrets. It was excellent at Prague and once again thanks Update Conference inviting me to speak there.

Transcript

1. Managing your Secrets in a Cloud Environment Taswar Bhatti System/Solutions Architect at Gemalto (Canada) Microsoft MVP
2. Is your personal data important?
3. Who am I • Taswar Bhatti – Microsoft MVP since 2014 • Global Solutions Architect/System Architect at Gemalto • In Software Industry since 2000 • I know Kung Fu (Languages)
4. Good old days robbery
5. Today’s Robbery
6. Data breach……
7. Consequences
8. System with no Trust
9. Salesman
10. Data breach??
11. Delivery
12. Agenda • Intro • What are we trying to solve with KeyVault? • What is Azure Key Vault • Using Azure Key Vault with your application • Managed Service Identity • Demo • HashiCorp Vault • Best practices • Questions
13. So what are secrets? • Secrets grants you AuthN or AuthZ to a system • Examples • Username & Passwords • Database credentials • API Token • TLS Certs
14. Typical Application
15. Storing Configuration in file
16. Multiple application
17. Secret Sprawl • Secrets ends up in • Source Code • Version Control Systems (Github, Gitlab, Bitbucket etc) • Configuration Management (Chef, Puppet, Ansible etc)
18. Problems • Configuration becomes part of deployment • Multiple applications share the same configuration • Hard to have access control over the configuration
19. Issues • How do we know who has access to those secrets • When was the last time they accessed it? • What if we want to change/rotate the secrets
20. Desire secrets • Encryption in rest and transit • Only decrypted in memory • Access control • Rotation & Revocation
21. What is Azure Key Vault? • Secrets Management – Azure Key Vault can be used to Securely store and tightly control access to tokens, passwords, certificates, API keys, and other secrets. • Key Management – Azure Key Vault can also be used as a Key Management solution. Azure Key Vault makes it easy to create and control the encryption keys used to encrypt your data. • Certificate Management – Azure Key Vault is also a service that lets you easily provision, manage, and deploy public and private Secure Sockets Layer/Transport Layer Security (SSL/TLS) certificates for use with Azure and your internal connected resources. • Store secrets backed by Hardware Security Modules – The secrets and keys can be protected either by software or FIPS 140-2 Level 2 validates HSMs.
22. Gemalto Luna HSM (New)
23. PKCS11 Interop • Managed .NET wrapper for unmanaged PKCS#11 libraries • https://pkcs11interop.net/
24. Typical Application • In web.config
25. With Key Vault
26. Azure Key Vault • Register your app with Active Directory • Associated credential, and using that credential to get a token • Retrieve your secrets from Key Vault • PROBLEM SOLVED
27. Adding it back to web.config • • • •
28. Code that looks like this ClientCredential clientCred = new ClientCredential( WebConfigurationManager.AppSettings[“ClientId”], WebConfigurationManager.AppSettings[“ClientSecret”]);
29. But???? • Confused?? • Isn’t that still in web.config?
30. Security doesn’t have to be like this
31. Managed Service Identity (MSI) • MSI gives your code an automatically managed identity for authenticating to Azure services, so that you can keep credentials out of your code • You create an identity for your application in Azure Active Directory using Managed Service Identity
32. Benefits • No need to authenticate to Azure Key Vault to get secrets • No client id and client secret is needed in the code • Easier to configure comparing to Azure Key Vault • You can authenticate to any service that supports Azure AD authentication
33. Demo
34. HSBC Hong Kong PayMe Hack
35. HashiCorp Vault • Centralized Secret Management • Encrypted at rest and transit • Lease and Renewal • ACL • Audit Trail • Multiple Client Auth Method (Ldap,Github, approle) • Dynamic Secrets • Encryption as a Service
36. Secure Secrets • AES 256 with GCM encryption • TLS 1.2 for clients • No HSM is required • One could also integrate with Azure Key Vault
37. Unsealing the Vault • Vault requires encryption keys to encrypt data • Shamir Secret Key Sharing • Master key is split into multiple keys
38. Shamir Secret Sharing
39. Unseal • Unseal Key 1: QZdnKsOyGXaWoB2viLBBWLlIpU+tQrQy49D+Mq24/V0B • Unseal Key 2: 1pxViFucRZDJ+kpXAeefepdmLwU6QpsFZwseOIPqaPAC • Unseal Key 3: bw+yIvxrXR5k8VoLqS5NGW4bjuZym2usm/PvCAaMh8UD • Unseal Key 4: o40xl6lcQo8+DgTQ0QJxkw0BgS5n6XHNtWOgBbt7LKYE • Unseal Key 5: Gh7WPQ6rWgGTBRSMecuj8PR8IM0vMIFkSZtRNT4dw5MF • Initial Root Token: 5b781ff4-eee8-d6a1-ea42-88428a7e8815 • Vault initialized with 5 keys and a key threshold of 3. Please • securely distribute the above keys. When the Vault is re-sealed, • restarted, or stopped, you must provide at least 3 of these keys • to unseal it again. • Vault does not store the master key. Without at least 3 keys, • your Vault will remain permanently sealed.
40. How to unseal • vault unseal -address=${VAULT_ADDR} QZdnKsOyGXaWoB2viLBBWLlIpU+tQrQy49D+Mq24/V0B • vault unseal -address=${VAULT_ADDR} bw+yIvxrXR5k8VoLqS5NGW4bjuZym2usm/PvCAaMh8UD • vault unseal -address=${VAULT_ADDR} Gh7WPQ6rWgGTBRSMecuj8PR8IM0vMIFkSZtRNT4dw5MF
41. Writing Secrets • vault write -address=${VAULT_ADDR} secret/hello value=world • vault read -address=${VAULT_ADDR} secret/hello • Key Value • — —– • refresh_interval 768h0m0s • Value world
42. Policy on secrets • We can assign application roles to the policy path “secret/web/*” { policy = “read” } • vault policy write -address=${VAULT_ADDR} web-policy ${DIR}/web-policy.hcl
43. Reading secrets based on policy • vault read -address=${VAULT_ADDR} secret/web/web-apps • vault read -address=${VAULT_ADDR} secret/hello • Error reading secret/hello: Error making API request. • URL: GET http://127.0.0.1:8200/v1/secret/hello • Code: 403. Errors: • * permission denied
44. Docker and Secrets • Docker does not have good integration with secrets • If you use env variables, it will show in docker inspect
45. Mount Temp File System into App • docker run –v /hostsecerts:/secerts …. • To mitigate reading from Env • Store your wrap token in the filesystem to use with vault • Have limit time on wrap token
46. Wrap Token for App Secrets • Limit time token • Used to unwrap some secrets • vault read -wrap-ttl=60s -address=http://127.0.0.1:8200 secret/weatherapp/config • Key Value • — —– • wrapping_token: 35093b2a-60d4-224d-5f16-b802c82de1e7 • wrapping_token_ttl: 1m0s • wrapping_token_creation_time: 2017-09-06 09:29:03.4892595 +0000 UTC • wrapping_token_creation_path: secret/weatherapp/config
47. Kubernetes with Vault • Read Service Account JWT • App Sends Jwt and Role Name to Vault • Vault checks the signature of Jwt • Sends to TokenReviewer API • Vault sends back valid token for app
48. Token Reviewer in K8s
49. Best Practices or Patterns • Cache Aside Encryption Key • Tag version of encryption
50. Cache Aside Encryption Key • Use Key Vault to Encrypt your Generated AES Key • For all encryption of your data you can use the AES Key rather than going back and Key Vault to encrypt • Allows you to penny pinch KeyVault
51. Tag Version of Encryption Level • Each Row of your database is tagged with the encryption version • This allows you when you rotate keys or change encryption level for example moving to a new Encryption Key to eventual encryption of data that gets updated or new.
52. New and Updated Data
53. Advantages • You do not have to go through all the records to re-encrypt them • Eventual Encryption of all data to new encryption • Mitigates the risk of all data or updating all records
54. Questions? • taswar@gmail.com • @taswarbhatti • http://taswar.zeytinsoft.com
55. Credits • For the background • www.Vecteezy.com

Azure, conference
, ,
  • 8 Cloud Design Patterns Update Conference

8 Cloud Design Patterns – Update Conference Prague

Taswar Bhatti November 23, 2018 No Comments

In November I went to Prague my first time and it was amazing being hosted by Update Conference there.
Below you will find my presentation on 8 Cloud Design Patterns that I did for Update Conference In Prague, I wanted to give a special thanks for @tomasherceg and his team for creating such an awesome conference.
Enjoy the sides!.

Transcript

1. 8 Cloud Design Patterns you ought to know Taswar Bhatti System/Solutions Architect at Gemalto (Canada) Microsoft MVP
2. Ponder • For every 25 percent increase in problem complexity, there is a 100 percent increase in solution complexity. • There is seldom one best design solution to a software problem. • If cars were like software, they would crash twice a day for no reason, and when you called for service, they’d tell you to reinstall the engine.
3. Who am I • Taswar Bhatti – Microsoft MVP since 2014 • Global Solutions Architect/System Architect at Gemalto • In Software Industry since 2000 • I know Kung Fu (Languages)
4. What I am not
5. Agenda • What are Patterns? • The External Configuration Pattern • The Cache Aside Pattern • The Federated Identity Pattern • The Valet Key Pattern • The Gatekeeper Pattern • The Circuit Breaker Pattern • The Retry Pattern • The Strangler Pattern • Demo • Questions
6. Bad Design
7. Bad Design
8. Bad Design
9. Bad Design
10. Bad Design
11. Feature doesn’t make sense????
12. Anger
13. Bad day at work
14. Happy family
15. Next day at work
16. Ship it
17. Customer Feature didn’t make sense
18. Bad Design?
19. Itunes when I use it
20. What are Patterns? • General reusable solution to a recurring problem • A template on how to solve a problem • Best practices • Patterns allow developers communicate with each other in well known and understand names for software interactions.
21. External Configuration Pattern
22. External Configuration Pattern • Helps move configuration information out of the application deployment • This pattern can provide for easier management and control of configuration data • For sharing configuration data across applications and other application instances
23. Typical Application
24. Storing Configuration in file
25. Multiple application
26. Problems • Configuration becomes part of deployment • Multiple applications share the same configuration • Hard to have access control over the configuration
27. External Configuration Pattern
28. When to use the pattern • When you have shared configuration, multiple application • You want to manage configuration centrally by DevOps • Provide audit for each configuration
29. When not to use • When you only have a single application there is no need to use this pattern it will make things more complex
30. Cloud Solution Offerings • Azure Key Vault • Vault by Hashicorp • AWS KMS • Keywhiz
31. Demo KeyVault
32. Cache Aside Pattern
33. Cache Aside Pattern • Load data on demand into a cache from datastore • Helps improve performance • Helps in maintain consistency between data held in the cache and data in the underlying data store.
34. Typical Application
35. Cache Aside Pattern
36. When to use the pattern • Resource demand is unpredictable. • This pattern enables applications to load data on demand • It makes no assumptions about which data an application will require in advance
37. When not to use • Don’t use it for data that changes very often
38. Things to consider • Sometimes data can be changed from outside process • Have an expiry for the data in cache • When update of data, invalidate the cache before updating the data in database • Pre populate the data if possible
39. Cloud Offerings • Redis (Azure and AWS) • Memcache • Hazelcast • Elastic Cache (AWS)
40. Federated Identity Pattern
41. Federated Identity Pattern • Delegate authentication to an external identity provider. • Simplify development, minimize the requirement for user administration • Improve the user experience of the application • Centralized providing MFA for user authentication
42. Typical Application
43. Problem
44. Problem • Complex development and maintenance (Duplicated code) • MFA is not an easy thing • User administration is a pain with access control • Hard to keep system secure • No single sign on (SSO) everyone needs to login again to different systems
45. Federated Identity Pattern
46. When to use • When you have multiple applications and want to provide SSO for applications • Federated identity with multiple partners • Federated identity in SAAS application
47. When not to use it • You already have a single application and have custom code that allows you to login
48. Things to consider • The identity Server needs to be highly available • Single point of failure, must have HA • RBAC, identity server usually does not have authorization information • Claims and scope within the security auth token
49. Cloud Offerings • Azure AD • Gemalto STA and SAS • Amazon IAM • GCP Cloud IAM
50. Valet Key Pattern
51. Valet Key Pattern • Use a token that provides clients with restricted direct access to a specific resource • Provide offload data transfer from the application • Minimize cost and maximize scalability and performance
52. Typical Application Client App Storage
53. Problem Client App Storage Client Client Client Client
54. Valet Key Pattern Client App Generate Token Limited Time And Scope Storage
55. When to use it • The application has limited resources • To minimize operational cost • Many interaction with external resources (upload, download) • When the data is stored in a remote data store or a different datacenter
56. When not to use it • When you need to transform the data before upload or download
57. Cloud Offerings • Azure Blob Storage • Amazon S3 • GCP Cloud Storage
58. Gatekeeper Pattern
59. Gatekeeper Pattern • Using a dedicated host instance that acts as a broker between clients and services • Protect applications and services • Validates and sanitizes requests, and passes requests and data between them • Provide an additional layer of security, and limit the attack surface of the system
60. Typical Application
61. Problem
62. Gatekeeper Pattern
63. When to use it • Sensitive information (Health care, Authentication) • Distributed System where perform request validation separately
64. When not to use • Performance vs security
65. Things to consider • WAF should not hold any keys or sensitive information • Use a secure communication channel • Auto scale • Endpoint IP address (when scaling application does the WAF know the new applications)
66. Circuit Breaker Pattern
67. Circuit Breaker Pattern • To handle faults that might take a variable amount of time to recover • When connecting to a remote service or resource
68. Typical Application
69. Problem
70. Client Circuit Breaker Api Closed State Timeout Closed State Open State Half Open State After X Retry Closed State
71. Circuit Breaker
72. When to use it • To prevent an application from trying to invoke a remote service or access a shared resource if this operation is highly likely to fail • Better user experience
73. When not to use • Handling access to local private resources in an application, such as in-memory data structure • Creates an overhead • Not a substitute for handling exceptions in the business logic of your applications
74. Libraries • Polly (http://www.thepollyproject.org/) • Netflix (Hystrix) https://github.com/Netflix/Hystrix/wiki
75. Retry pattern
76. Retry Pattern • Enable an application to handle transient failures • When the applications tries to connect to a service or network resource • By transparently retrying a failed operation
77. Typical Application Network Failure
78. Retry Pattern • Retry after 2, 5 or 10 seconds
79. When to use it • Use retry for only transient failure that is more than likely to resolve themselves quickly • Match the retry policies with the application • Otherwise use the circuit break pattern
80. When not to use it • Don’t cause a chain reaction to all components • For internal exceptions caused by business logic • Log all retry attempts to the service
81. Libraries • Roll your own code • Polly (http://www.thepollyproject.org/) • Netflix (Hystrix) https://github.com/Netflix/Hystrix/wiki
82. Demo Retry
83. Strangler Pattern
84. Strangler Pattern • Incrementally migrate a legacy system • Gradually replacing specific pieces of functionality with new applications and services • Features from the legacy system are replaced by new system features eventually • Strangling the old system and allowing you to decommission it
85. Monolith Application
86. Strangler Pattern
87. When to use • Gradually migrating a back-end application to a new architecture
88. When not to use • When requests to the back-end system cannot be intercepted • For smaller systems where the complexity of wholesale replacement is low
89. Considerations • Handle services and data stores that are potentially used by both new and legacy systems. • Make sure both can access these resources side-by-side • When migration is complete, the strangler façade will either go away or evolve into an adaptor for legacy clients • Make sure the façade doesn’t become a single point of failure or a performance bottleneck.
90. Questions? Taswar Bhatti System Solutions Architect (Gemalto) Microsoft MVP http://taswar.zeytinsoft.com @taswarbhatti
91. Credits • For the background • www.Vecteezy.com

Cloud, conference, Design, Public Speaking, Vault
, ,
Newer Posts
Older Posts
UA-4524639-2