Providing Solutions For Life

Software Defined Environment - Environments on Demand

Get the Development, QA, Staging or Production Environment you need at the click of a button.


Software Defined Environment

The current situation


It wouldn’t be a bold statement to say that all software’s ultimate goal is to enhance the customer experience. How many times have we not read such comments on app stores or heard business say?

 “Great app, but I can only give it three stars until the developers add ...”

But the Development team’s side of the story is

    “I am waiting for the environment to test the code with new features”

Continuous Delivery and Continuous Integration can help release software updates more frequently and with almost no manual intervention, but there are some bottlenecks to being able to do this. Following are a few: -

Delay in getting the Environments


  Lack of self-provisioning creates dependency on IT department.

Lack of easily customizable Environments


For Development, Testing and Staging with new features or updates to dependencies.

Manual Provisioning of Environments


Being repetitive and involving several steps, we would not be able to leverage the power of Automated Deployments and CI.

And the hilarious but unfortunately true risk of

“Oh! But it works on my laptop!”

Not being able to recreate the environments easily and consistently can lead to not being able to recreate performance issues or release code or updates to production confidently!

Inconsistent environments could result in such scenarios as a new update has been released to the production system and the system Admin might have put in the configuration or dependencies that only he or she is aware of to get the app working. Similarly the developer might have put in the unique settings on his or her laptop to get the code working on his or her workstation or laptop. Due to which every server becomes “works of art” and as unique as snowflakes. Needless to say inconsistent environments make it very difficult to determine why an application breaks when it's promoted to the next environment. Wasting the Developer and Operation teams time in determining if an issue is due to the source code or environment configuration.

What is an Environment?


It is not just an image or template of a virtual machine but all the compute, storage, network and several other resources (XaaS) that are required to host your application. Quite simply put, everything you can find inside the server room!

Environments on Demand at the click of a button


A solution that could give the Development, QA, Staging or Production Environment at the click of a button could remove all the bottlenecks and risks that we had discussed earlier and at the same time orchestrate Software-Defined Compute, Networking, Storage, Security and such to provide a smart infrastructure that is aware of resources needed by the application and is adaptive and responsive to the workloads dues to fluctuating business demand. All this while being easy to customize and simple to use!

Please watch this video demonstrating how it works.

How can Software Defined Environments help?


Self-Provisioning, empowers the Developers, QAs and etc. to bring up the environments they need, cutting delay due to dependency and bureaucracy!

Push-button deployments to get environments easily and run automated tests, on any version of the app to any environment, helps in getting faster feedback. Using policies, teams can be allocated quota of resources and authorization to use them can also be fine grained.

Everything can be parameterized due to which getting the n.x update of a dependency into an environment is just a click away.

The other advantages of having a Software Defined Environment is Automating orchestration will vastly reduce the possibility of human error, and make it possible to scale far beyond what people could do manually.

Reducing the cost of cloud ownership by sharing resources, time to market drastically and by being able to reuse the existing hardware resources



Increase the quality of service by improving the application performance by auto scaling which is achieved by having a hardware that is intelligent and adaptive to the needs of the app.

Phoenix Environments

           
Providing resilient, fault tolerant Environments, which can bring up your infrastructure in one click.

Mitigating the risks of inconsistent environments by providing Consistent Environments throughout the software development life cycle, i.e. from a Developers laptop unto the production systems.

Embracing the following advantages of Infrastructure as code, making your infrastructure IMMUTABLE!
           

Extend the advantages of version control from your app to Infrastructure.

Auto deployment will cut the repetitive and manual process of configuring all infrastructure resources.

Get a unified view simplifying the monitoring and management of all resources.

As you test the app at scale and once deployed, there could be hundreds of servers that might need to be brought up at need to scale the app. Using this approach we could cut the repetitive process and copy the configurations to more servers, virtual machines, switches, routers and storage servers instantly.

What do we do with our existing cloud investments?


As we are going to use Open Source Software, like OpenStack, Go CD and such it will be interoperable with any existing private cloud technologies like the VMWare, Xen and such allowing us to reuse the exiting hardware capital and get started with as little investments as possible and scale out by adding in more capacity seamlessly when ever the utilization is expected to reach higher.

Please watch this video demonstrating how it works or let us know and we would be delighted to take you on this journey!

Welcome to the era of Software Defined Economy!



You can see the accompanying slides here. 





The code has been open sourced here. Enjoy!

I will be presenting this in The India Cloud Expo conference

neutron IOError: [Errno 2] No such file or directory: '/proc/sys/net/ipv6/conf/default/disable_ipv6'


Error:
neutron IOError: [Errno 2] No such file or directory: '/proc/sys/net/ipv6/conf/default/disable_ipv6'


Due to this error, the neutron-l3-agent does not start up. End errors out. with the above error. There is a bug filed for this and patched in releases after Icehouse.

Solution

This bug is fixed by doing this

curl -o /usr/lib/python2.6/site-packages/neutron/common/ipv6_utils.phttps://raw.githubusercontent.com/openstack/neutron/stable/icehouse/neutron/common/ipv6_utils.py

Error: Chef::Exceptions::JSON::ParseError: parse error: premature EOF

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
52.0.234.85
52.0.234.85 Starting Chef Client, version 11.18.6
52.0.234.85
52.0.234.85 ================================================================================
52.0.234.85 Chef encountered an error attempting to load the node data for "first1"
52.0.234.85 ================================================================================
52.0.234.85
52.0.234.85 Unexpected Error:
52.0.234.85 -----------------
52.0.234.85 Chef::Exceptions::JSON::ParseError: parse error: premature EOF
52.0.234.85
52.0.234.85                      (right here) ------^
52.0.234.85
52.0.234.85 [2015-02-09T02:36:38-05:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
52.0.234.85 Chef Client failed. 0 resources updated in 0.854925495 seconds
52.0.234.85 [2015-02-09T02:36:38-05:00] ERROR: parse error: premature EOF
52.0.234.85
52.0.234.85                      (right here) ------^
52.0.234.85
52.0.234.85 [2015-02-09T02:36:38-05:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
 ✘ ⮀ ~/devops/learnChef ⮀


Solution:


Go to the knife.rb file (generally in ~/.chef/)
And change the


chef_server_url          'https://manage.chef.io/organizations/my_org'

to

chef_server_url          'https://api.opscode.com/organizations/my_org'

error: can't create transaction lock on /var/lib/rpm/.rpm.lock (Permission denied)

While bootstrapping a redhat node which is on AWS EC2, which supports only the SSH identity file for authentication, I was getting the below error after this command 

knife bootstrap 52.0.13.12 --ssh-user ec2-user --identity-file ~/.ssh/xyz.pem --node-name first1 --run-list 'recipe[some_thing]'


Error

52.0.13.132 Installing Chef Client...
52.0.13.132   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
52.0.13.132                                  Dload  Upload   Total   Spent    Left  Speed
100 18285  100 18285    0     0  48457      0 --:--:-- --:--:-- --:--:-- 48630
52.0.13.132 Downloading Chef 11 for el...
52.0.13.132 downloading https://www.opscode.com/chef/metadata?v=11&prerelease=false&nightlies=false&p=el&pv=7&m=x86_64
52.0.13.132   to file /tmp/install.sh.940/metadata.txt
52.0.13.132 trying curl...
52.0.13.132 url https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-11.18.6-1.el6.x86_64.rpm
52.0.13.132 md5 b4ccffea24007b83ffdd99b16aea9661
52.0.13.132 sha256 f531541c6786f274bd62fb46bc1ea8f2d70c083e10777b2544c6503c0f90c598
52.0.13.132 yolo true
52.0.13.132 downloaded metadata file looks valid...
52.0.13.132 downloading https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-11.18.6-1.el6.x86_64.rpm
52.0.13.132   to file /tmp/install.sh.940/chef-11.18.6-1.el6.x86_64.rpm
52.0.13.132 trying curl...
52.0.13.132 Comparing checksum with sha256sum...
52.0.13.132
52.0.13.132 WARNING: Chef-Client has not been regression tested on this O/S Distribution
52.0.13.132 WARNING: Do not use this configuration for Production Applications.  Use at your own risk.
52.0.13.132
52.0.13.132 Installing Chef 11
52.0.13.132 installing with rpm...
52.0.13.132 warning: /tmp/install.sh.940/chef-11.18.6-1.el6.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 83ef826a: NOKEY
error: can't create transaction lock on /var/lib/rpm/.rpm.lock (Permission denied)
52.0.13.132 Installation failed
52.0.13.132 Version: 11
52.0.13.132
52.0.13.132 Please file a Bug Report at https://github.com/opscode/opscode-omnitruck/issues/new
52.0.13.132 Alternatively, feel free to open a Support Ticket at https://www.getchef.com/support/tickets
52.0.13.132 More Chef support resources can be found at https://www.getchef.com/support
52.0.13.132
52.0.13.132 Please include as many details about the problem as possible i.e., how to reproduce
52.0.13.132 the problem (if possible), type of the Operating System and its version, etc.,
52.0.13.132 and any other relevant details that might help us with troubleshooting.
52.0.13.132
52.0.13.132 mkdir: cannot create directory ‘/etc/chef’: Permission denied
52.0.13.132 bash: line 36: /etc/chef/validation.pem: No such file or directory
52.0.13.132 chmod: cannot access ‘/etc/chef/validation.pem’: No such file or directory
52.0.13.132 bash: line 69: /etc/chef/client.rb: No such file or directory
52.0.13.132 bash: line 77: /etc/chef/first-boot.json: No such file or directory
52.0.13.132 Starting first Chef Client run...
52.0.13.132 bash: line 83: chef-client: command not found

Solution

Using the option "--sudo" does the trick



knife bootstrap 52.0.13.132 --ssh-user ec2-user --identity-file ~/.ssh/xyz.pem --sudo --node-name first1 --run-list 'recipe[some_thing]'

52.0.13.132 Installing Chef Client...
52.0.13.132   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
52.0.13.132                                  Dload  Upload   Total   Spent    Left  Speed
100 18285  100 18285    0     0  49365      0 --:--:-- --:--:-- --:--:-- 49552
52.0.13.132 Downloading Chef 11 for el...
52.0.13.132 downloading https://www.opscode.com/chef/metadata?v=11&prerelease=false&nightlies=false&p=el&pv=7&m=x86_64
52.0.13.132   to file /tmp/install.sh.1005/metadata.txt
52.0.13.132 trying curl...
52.0.13.132 url https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-11.18.6-1.el6.x86_64.rpm
52.0.13.132 md5 b4ccffea24007b83ffdd99b16aea9661
52.0.13.132 sha256 f531541c6786f274bd62fb46bc1ea8f2d70c083e10777b2544c6503c0f90c598
52.0.13.132 yolo true
52.0.13.132 downloaded metadata file looks valid...
52.0.13.132 downloading https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-11.18.6-1.el6.x86_64.rpm
52.0.13.132   to file /tmp/install.sh.1005/chef-11.18.6-1.el6.x86_64.rpm
52.0.13.132 trying curl...
52.0.13.132 Comparing checksum with sha256sum...
52.0.13.132
52.0.13.132 WARNING: Chef-Client has not been regression tested on this O/S Distribution
52.0.13.132 WARNING: Do not use this configuration for Production Applications.  Use at your own risk.
52.0.13.132
52.0.13.132 Installing Chef 11
52.0.13.132 installing with rpm...
52.0.13.132 warning: /tmp/install.sh.1005/chef-11.18.6-1.el6.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 83ef826a: NOKEY
52.0.13.132 Preparing...                          ################################# [100%]
52.0.13.132 Updating / installing...
52.0.13.132    1:chef-11.18.6-1.el6               ################################# [100%]
52.0.13.132 Thank you for installing Chef!
52.0.13.132 Starting first Chef Client run...
52.0.13.132 [2015-02-06T07:25:37-05:00] WARN:
52.0.13.132 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
52.0.13.132 SSL validation of HTTPS requests is disabled. HTTPS connections are still
52.0.13.132 encrypted, but chef is not able to detect forged replies or man in the middle
52.0.13.132 attacks.
52.0.13.132
52.0.13.132 To fix this issue add an entry like this to your configuration file:
52.0.13.132
52.0.13.132 ```
52.0.13.132   # Verify all HTTPS connections (recommended)
52.0.13.132   ssl_verify_mode :verify_peer
52.0.13.132
52.0.13.132   # OR, Verify only connections to chef-server
52.0.13.132   verify_api_cert true
52.0.13.132 ```
52.0.13.132
52.0.13.132 To check your SSL configuration, or troubleshoot errors, you can use the
52.0.13.132 `knife ssl check` command like so:
52.0.13.132
52.0.13.132 ```
52.0.13.132   knife ssl check -c /etc/chef/client.rb
52.0.13.132 ```
52.0.13.132
52.0.13.132 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
52.0.13.132
52.0.13.132 Starting Chef Client, version 11.18.6
52.0.13.132 Creating a new client identity for first1 using the validator key.
52.0.13.132 resolving cookbooks for run list: ["learn_chef_httpd"]
52.0.13.132 Synchronizing Cookbooks:
52.0.13.132   - learn_chef_httpd
52.0.13.132 Compiling Cookbooks...
52.0.13.132 Converging 4 resources
52.0.13.132 Recipe: learn_chef_httpd::default
52.0.13.132   * package[httpd] action install
52.0.13.132     - install version 2.4.6-19.el7_0 of package httpd
52.0.13.132   * service[httpd] action start
52.0.13.132     - start service service[httpd]
52.0.13.132   * service[httpd] action enable
52.0.13.132     - enable service service[httpd]
52.0.13.132   * template[/var/www/html/index.html] action create
52.0.13.132     - create new file /var/www/html/index.html
52.0.13.132     - update content in file /var/www/html/index.html from none to ef4ffd
52.0.13.132     --- /var/www/html/index.html 2015-02-06 07:25:47.501523711 -0500
52.0.13.132     +++ /tmp/chef-rendered-template20150206-1075-lczwdi 2015-02-06 07:25:47.502523687 -0500
52.0.13.132     @@ -1 +1,6 @@
52.0.13.132     +<html>
52.0.13.132     +  <body>
52.0.13.132     +    <h1>hello world</h1>
52.0.13.132     +  </body>
52.0.13.132     +</html>
52.0.13.132     - restore selinux security context
52.0.13.132   * service[iptables] action stop (up to date)
52.0.13.132
52.0.13.132 Running handlers:
52.0.13.132 Running handlers complete
52.0.13.132 Chef Client finished, 4/5 resources updated in 10.298957604 seconds