Providing Solutions For Life

Git simplified and master it in 10 minutes!

OK, what is this Git thing! Just because the developers use it, doesn't mean its rocket science.
Some are using it to work on family photo albums so it should be useful for all.

Imagine if you were composing a document and wanted a far off friend's help on it, you would write the initial draft, then email the attachment to your friend, he would download, save a copy, edit it and then email it to you.
Now suppose instead of this emailing overhead, if we could just have this document stored on your computer and your final edits pushed and stored on a web repository which your friend can download from and work on it locally and push the changes to the web repo until its perfect.

Different versions of this document from the initial one to the final one is stored and available and could be called Versions, since the docs are available with everyone working on it and not central its distributed and you and your friend have a set of rules to control the flow, combing it, distributed version control system (DVCS).
The reason for its increased adaptation is 1. No central repo so no point of failure. 2. Open Source

Ok going right into the crux.

Any time you need help on something, and to also check if git it already installed, launch a terminal and type the below, if git is not installed, just google for how to install it on your machine.

  • git help

Just adding our details on Git now so that it know who we are and changes to a doc could be accredited to us


  • git config --global user.name "Venu Murthy"
  • git config --global user.email "xyz@xyz.com" 

Once that is done, lets now go into a directory that you would like to put on Git. mkdir testgit, change to that directory by typing cd testgit


  • git init
Output is "Initialized empty Git repository in /Users/venum/testgit/.git/"

Once you initialised the git, we can check the status of what files have been added or modified by 
  • git status
The workflow is

File created or modified ====staged==> Ready for Save ======commit====> Saved on Repo on disk

So now we need to stage the file i.e. similar to save the modifications to the file on our harddisk. 
  • git add <filename>
filename could be the file name or a wildcard like *.txt or --all to checkin all files in the directory.

once this has been done, we now need to launch the staged file to the repo, its called commit. 
  • git commit -m "some comment--keep it in present tense"
to see the history of what has been done so far
  • git log


To see the changes or differences that have happened after the last commit of a file
  • git diff <filename>
So far we've been able to add files to our repo on disk, now lets push our local "master" to Git source control so that it can be shared. Register on Github.com for eg. Once you create a repo below is how you tell your git on where to push these files
  • git remote add origin https://github.com/venumurthy/testgit.git
  • git push -u origin master
in the above command -u is so that it remembers the parameters so that we wouldn't have to give it the next time. Origin is destination to which master is going to be copied to. 

Go to github and check if your files have come in. After this when you make any changes to the file, remember to do the following:-

  • git add <filename>
  • git commit -m "some comment"
  • git push

Congratulate yourself now for having tamed the octocat! 

OAuth Simplified

In my earlier post we saw what is SAML. Basically it works on the HTTP i.e. for webpages and etc. 
But what if you wanted have SSO on an app, for example on a Mobile App? There are work arounds to implement SAML but the straight forward way of doing it would be to use OAuth, a recent boy in neighbourhood which unlike SAML which is older than 6 years, is designed considering the future, of native apps and mobile apps. 

As usual the definition is 



OAuth is an open standard for authorisation. OAuth provides a method for clients to access server resources on behalf of a resource owner (such as a different client or an end-user). It also provides a process for end-users to authorise third-party access to their server resources without sharing their credentials (typically, a username and password pair), using user-agent redirections.

Please observe the definition, it says Authorisation and not Authentication!


What it means is that if you have an app on your phone and would like it be able to 


OAuth 2.0 is a relatively simple protocol. To begin, you register your application with Google. Then your client application requests an access token from the Google Authorisation Server, extracts a token from the response, and sends the token to the Google API that you want to access.


From Googles doc we see this

OAuth 2.0 is a relatively simple protocol. To begin, you register your application with Google. Then your client application requests an access token from the Google Authorization Server, extracts a token from the response, and sends the token to the Google API that you want to access.



So its basically a user trying to use a web or a standalone mobile or desktop app, which first requests permission from Google servers, and once the user approves this app, the app can fetch resources from the service provider .

One very nice def to lookup is in here.

This is what OAuth does, it allows you the User to grant access to your private resources on one site (which is called the Service Provider), to another site (called Consumer, not to be confused with you, the User). 

At this point in time I would like to give some relief to those who haven't understood it still, relax, seems like Oauth will be obsolete soon.   

SAML (Security Assertion Markup Language) -- Simplified


As discussed earlier SAML provides the web SSO as in the earlier blog on SSO.

By Definition SAML is-

Security Assertion Markup Language is an XML-based open standard data format for exchanging authentication and authorization data between parties, in particular, between an identity provider and a service provider.

Breaking it down.

IdP is the Identity Provider -- someone who has a database containing usernames and passwords. For eg. Google apps.

SP is service provider -- a service we would like to use. For eg. apps like soundcloud and etc which allow us to log into their service using the credentials stored in IdP.

So basically, if you want to add a comment to this blog post, you would have to authenticate with Google apps over the Browser and one Google identifies you as the right user, will allow you to access this resource and leave a comment. This is quite simply SAML at work.

The below video is amazing and a must watch to just understand it further.


For something more detailed see below


SSO--Single Sign On simplified

Before we go there, lets understand what is meant by Authentication and Authorisation.
Lets take an example, if you are watching a video on youtube, its available to public and hence needs not Authentication, but to read your mail on gmail you would need to authenticate, i.e. say who you are and the gmail server checks by comparing your username and password with what has stored, this process is called Authentication. Now suppose you want be able to make changes to a resource, the system checks your Authority to see if are allowed to do that this is authorisation, like the uploader of a video can delete or edit the video where was as we are authorised to only view the video. 

by definition

Single sign-on (SSO) is mechanism whereby a single action of user authentication and authorisation can permit a user to access all computers and systems where he has access permission, without the need to enter multiple passwords. Single sign-on reduces human error, a major component of systems failure and is therefore highly desirable but difficult to implement. 

making it more simple, when you log into Gmail, you can access Google Calendar, Docs and the other 144+ Google services without having to register/signup nor having to log in with username and password, you just go in.... this is SSO or Single Sign On.

SSO can be achieved by SAML or OAuth in the following articles we will see what that means.

Error: SMTPIN_ADDED_BROKEN@mx.google.com

Some times we see the below in the email header information. SMTPIN_ADDED_BROKEN@mx.google.com

The cause of the "...SMTPIN_ADDED_BROKEN@mx.google.com" is because of the 'https://' in the original Message-ID, which is not allowed. There could be some dodo systems which might be doing this and they would need to be tweaked to now allow message-id (see my earlier posts on what it means) and to have https!

Leverage the power of Cloud with Python

My well honoured Presentation at the Cloud Connect International Event in Renaissance Hotel, Mumabi in 2013




Leverage the power of the Cloud using Python from Venu Murthy



It was am amazing moment when my proposal was shortlisted and I was invited to speak in the much famed Cloud Connect event in Mumbai, June 2013.

Below are some pics, the audience were amazing and as one of my friends said, "wow, you are listed with the C level people already, here!"





Google Mail or Gmail threading or conversations demystified



Many of us like the mail threading or in Gamil they are also called as conversations and for those who do not like it, Google does gives us the option to disable threading. To do so, go to the General tab of your Gmail Settings, and select the radio button next to 'Conversation view off'.


 However for those of you who are interested in Gmail threading. Below is a threaded mail




Gmail threading is based on these factors. 

  1. Subject must be same or similar (it can have a prefix of Re:, Fw:)
  2. Sender must be a part of the thread OR in-reply-to must be used. (see this link for the basics) 
  3. Time from when the original was sent and its reply.

A typical header message looks as below:-

Message-ID: <13dsaf693.25715.YahooMailNeo@web126206.mail.ne1.yahoo.com>Date: Mon, 23 Sep 2013 04:44:53 -0700 (PDT)From: Divakar Nadig <xyz@yahoo.com>Reply-To: Divakar Nadig <xyz@yahoo.com>Subject: please see this dear son.....
To: * V e n u * <xyz@gmail.com>

See the below screenshot for a crash course on mails in a thread and their various components.



Advanced version explaining the flow


  • Subject line is the primary factor, which is used to gather candidate threads. The don't have to be exactly be the same, they could have Re:, Fw: and etc but no changes should have been made to the subject of the mail. 
  • These threads are then evaluated to see if the "References" fields (as in the above screenshot) of the header contain message-IDs of threads already in the mailbox. If a reference is found, it is threaded to that reference. 


  • If no references are found, it is then moved to a date-based search, and narrowed down to a list of candidates as follows:
  • The preferred option (typically conversations and automated messages):
  • The incoming message's from address is listed as a to/cc/ from address of a message in the candidate thread, and one of its to/cc addresses is also listed as a to/cc/ from address of a message in the candidate thread. If there are any candidates in the preferred option, it is not even consider the fallback candidates.
The fallback option (typically mailing lists): The from address doesn't match, but one of the incoming message's to/cc addresses is listed as a to/cc/from address of a message in the candidate thread.

If any candidate messages match one of these two options, they are evaluated based on the date of the message. We then choose to thread the message with either the nearest older message or the nearest newer message, whichever is closer, with some important provisos:


There is a different time limit for threading messages that are considered to be a reply or forward (as judged by whether or not the subject has an "re" or "fw" prefix) - these messages can be threaded together with the nearest candidate thread message if it is up to 7 days away.


If the incoming message is not an re/fwd, then we will only thread it with the nearest candidate message if it is within 12 hours away, and also if it is from the same sender.


Whats an email anyway? What are it's components?

A some what weird question but when explored its very interesting and lets see how deep the rabbit hole goes!

Electronic mail is commonly called as e-mail or eMail, its been around from 1973. it’s a method of exchanging digital messages across the internet or other computer networks. After the Snowden era may be we will see a trend of peer to peer communication without a centralised server, the way it was in the beginning of time! 

An Internet email message contains 3 components:-


  • Message envelope
  • Message header – contains control information, flow and etc.
  • Message body
In a client-server model, client is from where the message originate and Email servers accept, forward, deliver and store messages. 

Originally emails could carry text-only (7-bit ASCII) communications medium, which was then extended to carry multi-media content attachments, a process standardised in RFC 2045 through 2049. Collectively, these RFCs have come to be called Multipurpose Internet Mail Extensions (MIME). The Internet email message format is now defined by RFC 5322, got from RFC 2822, 822.

Network-based email exchanged by the SMTP. In the process of transporting email messages between systems, SMTP communicates delivery parameters using a message envelope separate from the message (header and body) itself.

This standard specifies a syntax for text messages that are sent among computer users, within the framework of "electronic mail".

A message consists of header fields and, optionally, a body. The body is simply a sequence of lines containing ASCII characters. It is separated from the headers by a null line.

When we go to an email and analyse its header information, we see the following fields:-

 
TO:This field contains the identity of the primary recipients of the message.  
CC:This field contains the identity of the secondary (informational) recipients of the message.  
BCCThis field contains the identity of additional recipients of the message. The contents of this field are not included in copies of the message sent to the primary and secondary recipients. Gmail choose to include the text of the "Bcc" field only in receipent's copy due to which he got the mail.  
REFERENCE FIELDS 
MESSAGE-ID / RESENT-MESSAGE-IDThis field contains a unique identifier (the local-part address unit) that refers to THIS version of THIS message. The host that generates it guarantees the uniqueness of the message identifier. This identifier is intended to be machine readable and not necessarily meaningful to humans. A message identifier pertains to exactly one instantiation of a particular message; subsequent revisions like replies or forwards to the essage should each receive new message identifiers 
IN-REPLY-TOThe contents of this field identify previous correspondence, which this message answers. Note that if message identifiers are used in this field, they must use the msg-id specification format.A message just sent will not have a IN-REPLY-TO field. 

REFERENCESThe contents of this field identify other correspondence which this message references. Note that if message identifiers are used, they must use the msg-id specification format. i.e. this will have all the Message IDs so far. 
SUBJECTThis is intended to provide a summary, or indicate the nature, of the message. 

See below the Screenshot of the mail threads.



 Some more fields in the mail but not used.
ENCRYPTEDSometimes, data encryption is used to increase the privacy of message contents. If the body of a message has been encrypted, to keep its contents private, the "Encrypted" field can be used to note the fact and to indicate the nature of the encryption. Not applicable when it comes to gmail. 

EXTENSION-FIELDA limited number of common fields have been defined in this document. As network mail requirements dictate, additional fields may be standardized. To provide user-defined fields with a measure of safety, in name selection, such extension-fields will never have names that begin with the string "X-". 

USER-DEFINED-FIELDIndividual users of network mail are free to define and use additional heade fields. Such fields must have names which are not already used in the current specification or in any definitions of extension-fields, and the overall syntax ofthese user-defined-fields must conform to this specification's rules for delimiting and folding fields. Due to the extension-field publishing process, the name of a user- defined-field may be pre-empted Note: The prefatory string "X-" will never be used in the names of Extension-fields. This provides user-defined fields with a protected set of names.

The best IDE for Python -- PyCharm goes opensourced - i.e. Free!!!

It gives me great joy that PyCharm has been opensourced, I've been using it for the past 2 years and am kind of addicted to it, though I do agree that I haven't exploited it enough.

You can download it from this link
Whats so great about it!






Awesome Navigation

Command + Click on a module to go to where the class is defined.
Option + F7 gives a view of where all its used.
Command + Option + B ==> to see where all its implemented.

What I love the most is the way it generates functions for you.

Type name of a method which has not yet been defined. and then
Option + Enter and select Create Function

Option + Command + V will instantiate an object

Enjoy Python!