Providing Solutions For Life

Python on the Cloud for Gaming!


This is to show how Python was used on the cloud IDE called http://www.codeskulptor.org/

This game was not only a test of my programming concepts in Python but also a great learning in Physics, how things works in zero gravity, damping, angular velocity, and its damping.....

You can play this game here




It was honoured to have been played at the office wide annual event of India and Brazil Away Day of 2013. And was a massive crowd puller. (I was told that people begged to play this game and there was so much of crowd around the stalls that the GMs had to intervene and disperse the crowds!)


Below are some pics trying to capture the audience response to this game


Getting the Chef to cook for us!

We have three things here, in the earlier post, we saw how chef was installed on the workstation.

1. Enterprise Chef Server - Which is the main server which will be orchestrating the entire process.

2. Node -  this is the Virtual machine which will be hosting our app the one which will be carrying our payload. It will have the chef-client installed on it, which will be communicating with the Enterprise Chef server and controlling the configuration of the node or the VM.

3. Workstation - your laptop, that is the machine used to configure the Enterprise server and control the nodes. This is where we had installed chef in our earlier exercise.


Before we get started we would need to get a login name and the starter kit from the Chef Enterprise Server.


  • Go to this link and register. Remember that you will be using the organisation name, as we will be using it.
  • Once registered, signed in!
  • Download the Starter Kit.
  • Unzip the kit, this is the knife tool
  • rename the folder to "chef-repo" and move it into your home folder


TIP: have sublimetext installed on your machine, so that its easier to edit the configuration files. You could use this link if you are on Ubuntu.


Using Sublimetext lets open the folder of the chef-repo and also the knife.rb the ruby file which is the config file for knife and examine it's contents



When ever we are going to work on a terminal next, it would be within the chef-repo directory

so we go into the terminal and put in the command

knife --version

to ensure that that knife is ready for us to use.


and confirm the version number.

And to confirm if the configuration is perfect and the client is able to connect to the server


knife client list 

gives us the organisation name confirming that your knife is connected.

More information about Knife file configuration can be found on this link.

Checking it all into Git

In the chef-repo, we initiate Git and then check in the code, you can refer this post to get a hang of Git.






Looking at our Organisations!

Log into this link.
And we get the organisations by clicking on the Administration tab. 


and we can do the following from this page by clicking on the wrench on the right hand corner.




Enough of setting things up now, time for some action!

Getting a node or VM to try stuff out!

We will use Vagrant, to create our nodes. To know more about Vagrant you can see some of the earlier post "Vagrant - Simplified and mastered!" and some more in this series.

We see that within the chef-repo there is a Vagrantfile which is the config file to create a node or a VM.

So we will just bring up a VM by using the "vagrant up" command.

vagrant up

to know the details of the node
vagrant ssh-config 





To connect to this server

ssh vagrant@127.0.0.1 -p 2222

and we see on how to connect to it. Or even vagrant ssh comments also helps us in connecting to our instance. 

Bootstrapping a node!

Is nothing but installing the chef-client on the node, there are two ways of doing it one of them is the attended and the other unattended. 

The default for this approach assumes that node can access the Chef website so that it may download the chef-client package from that location. 
The omnibus installer will detect the version of the operating system, and then install the appropriate version of the chef-client using a single command to install the chef-client and all of its dependencies, including an embedded version of Ruby, RubyGems, OpenSSL, key-value stores, parsers, libraries, and command line utilities. 
The omnibus installer puts everything into a unique directory (opt/opscode/) so that the chef-client will not interfere with other applications that may be running on the target machine. Once installed, the chef-client requires a few more configuration steps before it can perform its first chef-client run on a node.




take from this place

knife bootstrap FQDN/IP address
--sudo 
-x ssh_username
-P ssh_password
-p ssh_port
-N "nickname to refer to latter"

all of this put together

knife bootstrap 127.0.0.1 --sudo -x vagrant -P vagrant -p 22 -N "vm1"

its sheer joy to see it come up!


Troubleshooting: In case you get the error such as this on vagrant up
'require': cannot load such file -- log4r (LoadError)
this comes up when Vagrant is installed using apt-get. Hence download and install vagrant from the download page.

Going back to the Chef Enterprise server console we see that this target has been added.


We see the "vm1" added and below is the complete and detailed profile, which has come due to the system profiler called Ohai running on the node!

we can run ohai on our machine and see the amount of details its throws up on the screen.



Creating our First Cookbook

Definition from here.
A cookbook is the fundamental unit of configuration and policy distribution. Each cookbook defines a scenario, such as everything needed to install and configure MySQL, and then it contains all of the components that are required to support that scenario, including:
  • Attribute values that are set on nodes
  • Definitions that allow the creation of reusable collections of resources
  • File distributions
  • Libraries that extend the chef-client and/or provide helpers to Ruby code
  • Recipes that specify which resources to manage and the order in which those resources will be applied
  • Custom resources and providers
  • Templates
  • Versions
  • Metadata about recipes (including dependencies), version constraints, supported platforms, and so on
Let's try to write a cookbook to bring up a webserver for this, we would do the following:-

  1. Install Apache
  2. Start the service
  3. Make sure it boots when the machine starts
  4. Customise the home page.

Creating the cookbook by the name "apache" by running the below command on the workstation and in the chef-repo directory

knife cookbook create apache


This creates a new folder called apache under cookbooks and we will edit the default recipe 


before we start editing this file, lets take a look at what chef "Resources" are

  • A resource is a key part of a recipe. 
  • A resource defines the actions that can be taken, such as when a package should be installed, whether a service should be enabled or restarted, which groups, users, or groups of users should be created, where to put a collection of files, what the name of a new directory should be, and so on. 
  • Each resource is processed in the same order as they appear in a recipe. 
  • The chef-client ensures that the same actions are taken the same way everywhere and that actions produce the same result every time. 
  • A resource is implemented within a recipe using Ruby.
  • Where a resource represents a piece of the system (and its desired state), a provider defines the steps that are needed to bring that piece of the system from its current state into the desired state. These steps are de-coupled from the request itself. The request is made in a recipe and is defined by a lightweight resource. The steps are then defined by a lightweight provider.
A resource is a Ruby block with four components: a type, a name, one (or more) attributes (with values), and one (or more) actions. The syntax for a resource is like this:
type "name" do
   attribute "value"
   action :type_of_action
end
For example, a resource that is used to install a tar.gz package for version 1.16.1 may look something like this:
package "tar" do
  version "1.16.1-1"
  action :install
end
Every resource has its own set of actions and attribute. Most attributes have default values. Some attributes are available to all resources, for example some attributes are used to send notifications to other resources, whereas others are used to set up conditional execution rules.
All actions have a default value. Only non-default behaviours of actions and attributes need to be specified. For example, the package resource’s default action is to install and the name of the package defaults to the "name" of the resource, therefore it is possible to write a resource block like this:
package "tar"
and the chef-client will use the default action (:install) to install the tar package. refer this for more


so we will have the following now

package "apache2" do
    action :install
end

now that we want this service to be enabled even in reboot.

service "apache2" do
    action [:start, :enable]
end

next we will need to write the home page, so we will use the resource "cookbook_file" which is
The cookbook_file resource is used to transfer files from a sub-directory of the files/ directory in a cookbook to a specified path that is located on the host running the chef-client or chef-solo. 
cookbook_file "/var/www/index.html" do
    source "index.html"
    mode "0644"
end


now in the above we are asking chef to take the file called index.html and push it to the node


now to push this cookbook to the node, we use the command

knife cookbook upload apache


and to take a look at what has been uploaded


Now about the recipes 

A recipe: Must be added to a run-list before it can be used by the chef-client
Is always executed in the same order as listed in a run-list

What's a RunList now!

Again by definition:-

A run-list is an ordered list of recipes and/or roles that are run in an exact order. 
A run-list is always specific to the node on which it runs, though it is possible for many nodes to have run-lists that are similar or even identical. 
The items within a run-list are maintained using Knife and are uploaded to the server and stored as part of the node object for each node. 
The chef-client always configures a node in the exact order specified by its run-list and will never run the same recipe twice.
A recipe is the most fundamental configuration element within the organization. 


So we see that a recipe in itself is of now use unless we've added it to the runlist.


and now log into the VM or the node and run the command

sudo chef-client


yes, it will give an error, as the apt-get cache is outdated and we would need to update it, but let's automate that, instead of having a manual process to do this and at the same time see how order matters in runlists. We will create one more recipe in a new cookbook for apt.

We create the cookbook called apt_cache_update, then add a recipe and upload it.




and go into the Chef management console and add the newly created recipe and add it to being the first one in the order. 


save and now lets run the chef-client on the node to see what happens!


and finally when we try to access use a browser to get this webpage on port 8080 we get the home page that we created!

Installing Guest Additions for VirtualBox on a Ubuntu node



When working with VirutalBox it is quite helpful to have the VirtualBox guest additions installed.

The main reason for us to have this would be to blow up the VM we are working on to full screen.


Assuming that you have already launched the VirtualBox and firedup the VM we going to work on


1. Open a terminal on the VM.
2. Just so that everything is uptodate

sudo apt-get update

3. Followed by

sudo apt-get install build-essential module-assistant

4. To Configure the system for building kernel modules 

sudo m-a prepare

5. On the VirutalBox menu Click on Devices  >> Install Guest Additions which is at the bottom of the menu.

6. We switch over to the CD.
cd /media

7. Using "ls" find the contents of the CD and swtich to it.
cd VOBXADDITIONS_4.3.6...

8. finally running 

sudo ./VBoxLinuxAdditions.run

You would need to restart the VM to get these changes effective sometimes. 

shutdown -r now


And now we can see that on Full Screen, the VM blows up fully!


Chef Advanced - Setup and Installation

There was an earlier post in which we were able to install chef in no time, but now we will dig deeper and understand what exactly happens so that we have more control over this process. 

Firstly we will start off by installing VirtualBox, Vagrant, Git and Ruby on a linux machine, I am using Ubuntu today to demonstrate this. 

Installing VirtualBox and getting Ubuntu ready to receive it.

I have tried to do this in two ways, one of easiest way of doing this by going to the terminal and entering

sudo apt-get install virtualbox

The more detailed Version is as below, which will give us more control so that we could be as precise as possible with versions and dependencies. 

Lets confirm our Ubuntu version before we install the VirtualBox using the below

lsb_relase -a


We go to this download page for VirtualBox to get the link for the virtualBox that we would like to have installed. Before we run the single line command to install VirtualBox, lets just ensure that the system is upto date.
sudo apt-get update

And then also Confirm that VirtualBox is not already installed and running. 


And then the magically formulation to have it all installed in a single command!

wget http://download.virtualbox.org/virtualbox/4.3.6/virtualbox-4.3_4.3.6-91406~Ubuntu~precise_amd64.deb | sudo dpkg --install virtualbox-4.3_4.3.6-91406~Ubuntu~precise_amd64.deb



And here is Virtualbox 

to launch it from terminal, just type 

sudo VirtualBox




Troubleshooting

If you have some trouble installing VirtualBox in this way, running the below command should help in fixing anything that is broken. 

sudo apt-get --install -f

Now Installing Vagrant on Ubuntu!

The simple way of doing it is:

sudo apt-get install vagrant

To get more control on this i.e. for example if I want to install the vagrant version 1.2.7_x86_64.deb while the latest one is 1.3.5 and that is what gets installed through the above command. 

Go to the Terminal and put in the following command, I got the link by manually going to the download page. 

wget http://files.vagrantup.com/packages/7ec0ee1d00a916f80b109a298bab08e391945243/vagrant_1.2.7_x86_64.deb | sudo dpkg --install vagrant_1.2.7_x86_64.deb



And confirm that vagrant is installed by using the command

vagrant -v


Almost there, check if GIT is installed too !

Git is the version control we will be using. Its an awesome tool! You can check about it on this link here


Just a few commands used

sudo apt-get install git

git --version

the first one is to install Git and the latter is used to just confirm that Git is installed and is giving us the version number. 


Installing Chef! Finally!


In one of my earlier posts here, we see how we can install chef in just a few seconds!
But to get finer control and to know what exactly is happening, lets do it the long way here. 

Just before we go ahead and jump in, lets take a peek at Chef's definition

Chef is a systems and cloud infrastructure automation framework that makes it easy to deploy servers and applications to any physical, virtual, or cloud location, no matter the size of the infrastructure. Each organization is comprised of one (or more) workstations, a single server, and every node that will be configured and maintained by the chef-client. Cookbooks (and recipes) are used to tell the chef-client how each node in your organization should be configured. The chef-client (which is installed on every node) does the actual configuration.
And it is written in a programming language called Ruby so we will now install rbenv which is the Ruby Version Manager that will provide us some virtual environments to have multiple versions of Ruby on our system and the ability to easily switch between them.

We start off by first installing the support package

sudo apt-get install build-essential zlib1g-dev


Followed by installing the rbenv from the official git repository and add it to your path


git clone git://github.com/sstephenson/rbenv.git .rbenv 

echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> .bash_profile

echo 'eval "$(rbenv init -)"' >> .bash_profile

and reloading our bash profile

source ~/.bash_profile



Installing Ruby-build now

git clone git://github.com/sstephenson/ruby-build.git
 
cd ruby-build

./install.sh



Now installing the target Ruby version

 
rbenv rehash

rbenv install 1.9.3-p385

rbenv shell 1.9.3-p385

rbenv global 1.9.3-p385


And the final command

gem install chef

rbenv rehash



Congratulate yourself for getting the chef to cook meals for you in the future!  


TroubleShooting!

In case of error like this just do the following, (that is the reason I have included zlib the support packages that we downloaded a while ago. 

~$ gem install chef
ERROR: Loading command: install (LoadError)
    cannot load such file -- zlib
ERROR: While executing gem ... (NameError)
    uninitialized constant Gem::Commands::InstallCommand



rbenv uninstall 1.9.3-p286

rbenv install 1.9.3-p286

gem install chef




Another error that could come up are

Error: While Generating Documentation for pry-0.9.12.6
Message: Undefined method name for #<RDoc::RubyToken::TKDOT:......


I just put in the "gem install chef" command again and this resolved the issue,

But one interesting observation is the number of Ruby gems before installing chef and after!



After gem install chef!



Vagrant - Creating our own Box File from ISO.


Install VirtualBox

I assume that you've already got the VirutalBox installed. Else its very simple, just have it installed

Install Vagrant 

Then refer the previous post on Vagrant to have it installed. 

Creating a CentOS VM

Download the CentOS 6.5 x86_64 DVD.

Launch VirtualBox and do the following by click on the "New" on the left hand upper corner






This is like the hardware has been built, 


Now installing the OS on this machine, before we do that we tell it, where to pick up the ISO file from and disable unnecessary stuff. 



Now while our VM is highlighted click on the Start 

And this will load the lovely centos onto our VM!



In case it reboots, just log in and shutdown the VM using "shutdown -h now" and making the VirtualBox image a little clean we do the following. 



Configuring the VM to be in action when need!

Configuring the network on this device. 


Now you should be able to get the interface listed in ifconfig and able to ping Google!

Configuring Time on this machine!

yum install ntp ntpdate ntp-doc
chkconfig ntpd on
service ntpd stop
ntpdate time.nist.gov
service ntpd start



Check if the ssh service is running on this server and ensure that it will be running after we restart

service --status-all | grep ssh
chkconfig sshd on

Relaxing on Security!

Now something which you shouldn't be doing if this box will be used to create prods. We will disable the iptables and make SELinux premissive due to being lazy in dealing with network services. 

chkconfig iptables off

Open the file in a vi editor or the editor of your choice 

vi /etc/selinux/config 

and add 

SELINUX=permissive


Let's add a user who will have access to this VM

useradd vagrant

now creating the .ssh folder for the vagrant user, the usual mkdir with -m to set the mode of the folder that is being created and -p the path.

mkdir -m 0700 -p /home/vagrant/.ssh

We will use the ssh keys provided by vagrant 

curl https://raw.github.com/mitchellh/vagrant/master/keys/vagrant.pub >> /home/vagrant/.ssh/authorized_keys

chmod 600 /home/vagrant/.ssh/authorized_keys

we make sure that the vagrant user owns the ssh folder 

chown -R vagrant:vagrant /home/vagrant/.ssh

We will need to allow ssh to be able to send remote commands without using sudo and hence will comment out requiretty in /etc/sudoers

you can do it simply by typing visudo and this will open the file and search for the word by typing /requiretty and comment out the line once found 


And added the line

vagrant ALL=NOPASSWD: ALL

so that user vagrant will be able to use sudo without having to enter a password.

Now when we reboot the system the network interface flys away and to avoid that from happening, we change the ONBOOT to yes


We will remove the udev persistence  as we will be using this box else where

rm -f /etc/udev/rules.d/70-persistent-net.rules

Install some additional repository package that might be useful for us:


yum install -y git vim man wget  openssh-clients


now that are VM is all to be on Air! we will clean up the yum and history

yum clean all

history -c

and shutdown with

shutdown -h now









Now what we have is these files, these files can themselves be distributed or kept for future use with VirtualBox, but the reason we need a box file is so that Vagrant can do it automatically and create several VMs in a matter of few minutes!


Creating the Box out this now!

Create a directory I've created one called vagrant_test in the home directory. 

And we use the command 

vagrant package --base centos-6.5x86_64-minimal --output centos-6.5x86_64-minimal.box

syntax is

vagrant package --base <name of the Vitual box image> --output < filename of your choise>

<name of the Vitual box image> is the name we gave while creating the Virtual box image. 

This is the output we get and there is sometime delay for the new VM to get created.



Out brand new baby is now available to spin any number of VM now.

Taking the box file for a test ride now!

To do that let us tell Vagrant that now it has box which it can use. 



We've finally done this, and I am going to be sharing this box with you all on a link.