Friday, February 12, 2016

One thing to rule them all – Ansible on Vagrant



Vagrant allows to play with multiple VMs easily, but with multiple VMs, it is dizzy to switch VMs back and forth, it is easy to do something on VM B when the intended VM is A.

Ansible comes to the rescue. With Ansible, you sit comfortably at one VM and control other VMs. In this blog, I will show you how to setup Ansible on Vagrant. 

The eventual setup will be like this:
 

My physical machine is win7, on it, I start 4 VMs. VM mgmt is my management VM, which I will install ansible and rule over other VMs. VM infra is where I install infrastructure tools, such consul and other monitoring tools, the rest VMs app1 and app2 is where I install applications. 

I do development work on win7, after testing out, I deploy them into VMs. A shared folder is setup between win7 and mgmt, so it is easy to move things around between win7 and VMs.

 

Vagrantfile

Vagrant.configure("2") do |config|
   if Vagrant.has_plugin?("vagrant-proxyconf")
#replace with your proxies
        config.proxy.http= " https://<user>:<password>@<proxy-host>:<proxy-port>"
        config.proxy.https= " https://<user>:<password>@<proxy-host>:<proxy-port>"
        config.proxy.ftp= " https://<user>:<password>@<proxy-host>:<proxy-port>"
         config.proxy.no_proxy  = "localhost,127.0.0.1"  
end
  
    if Vagrant.has_plugin?("vagrant-timezone")
                config.timezone.value="Asia/Shanghai"
    end
  
   config.vm.box = "ubuntu14.04-amd64"
   config.vm.box_url = "https://github.com/kraksoft/vagrant-box-ubuntu/releases/download/14.04/ubuntu-14.04-amd64.box"
   config.ssh.forward_agent = true
   config.vm.provider "virtualbox" do |vb|      
             vb.gui = true 
            vb.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
              vb.customize ["modifyvm", :id, "--natdnsproxy1"       , "on"]            
             vb.customize ["modifyvm", :id, "--memory", 8192]
             vb.customize ['modifyvm', :id, '--nicpromisc1', 'allow-all']

             vb.customize ['modifyvm', :id, '--nicpromisc2', 'allow-all']
             vb.customize ["modifyvm", :id, "--ioapic"  , "on"]
             vb.customize ["modifyvm", :id, "--cpus"    , 2]
             vb.customize ["modifyvm", :id, "--pae"    , "on"]
             vb.customize ["modifyvm", :id, "--nictype1", "virtio"]
             vb.customize ["modifyvm", :id, "--nictype2", "virtio"]   
             vb.customize ["modifyvm", :id, "--chipset" , "ich9"]            
  end
  

  # create mgmt node
  config.vm.define :mgmt do |mgmt_config|
      mgmt_config.vm.hostname = "mgmt"
      mgmt_config.vm.network :private_network, ip: "192.168.33.10" 
      mgmt_config.vm.provision :shell, path: "bootstrap-mgmt.sh"        
       mgmt_config.vm.synced_folder "../../devops", "/devops" 
  end
 
   # create infra node
  config.vm.define :infra do |infra_config|
      infra_config.vm.hostname = "infra"
      infra_config.vm.network :private_network, ip: "192.168.33.11"         
  end
 
  # create app nodes
  (1..2).each do |i|
    config.vm.define "app#{i}" do |node|       
        node.vm.hostname = "app#{i}"
        node.vm.network :private_network, ip: "192.168.33.2#{i}"     
            
             if Vagrant.has_plugin?("vagrant-proxyconf")               
                    node.proxy.no_proxy  = "localhost,127.0.0.1,192.168.33.2#{i},app#{i}"
        end
    end 
  end

end

I’ve shared the Vagrantfile techniques in my previous blogs, there are only a couple of differences:
  • This single Vagrant file creates 4 VMs: mgmt, infra, app1, app2 
  • This Vagrant installs ansible on VM mgmt using bootstrap-mgmt.sh.

And here is the content of bootstrap-mgmt.sh:
#!/usr/bin/env bash

# install ansible (http://docs.ansible.com/intro_installation.html)
apt-get update
apt-get -y install software-properties-common
apt-add-repository -y ppa:ansible/ansible
apt-get update
apt-get -y install ansible

cat >> /etc/hosts <<EOL
# vagrant environment nodes
192.168.33.10  mgmt
192.168.33.11  infra
192.168.33.21  app1
192.168.33.22  app2
EOL

Note, when you run this Vagrantfile using vagrant up, Vagrant will fail for each VM complaining shared folder can’t be setup, refer to my previous blogs for the solution.

vagrant up will create and start all 4 VMs, you can also use vagrant up hostname to start one VM, and use vagrant ssh hostname to log on one VM.

 

Ansible

Now vagrant ssh mgmt, and verify ansible is installed correctly:

vagrant@mgmt:~$ ansible --version
ansible 2.0.0.2
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides

Now configure Ansible to work on infra and app1, app2 nodes. To do so, we need to setup inventory.ini and ansible.cfg, here is my file structure:

inventory.ini configures the nodes that are under the control of Ansible:
[infra]
infra
[app]
app1
app2

ansible.cfg configures Ansible properties:
[defaults]
inventory = /devops/SaaS-Example/ansible/inventory.ini
callback_plugins = /devops/SaaS-Example/ansible/callback_plugins
(this callback will be explained later)

Let us say hello from mgmt to other VMs. The command to use is ansible all -m ping. Let us break this command down:
  • all: the target nodes, all means all nodes in inventory.ini, we could also use a single node or node groups, such as app1 or app.
  •  -m: module
  •  ping: the module name. Ansible has a lot of modules, using Ansible is to use these modules to accomplish certain things. If you fail to find a certain module, you can always use the old good shell module.
 
Our first hello attempt fails. This is because the other nodes do not know who mgmt is, we need to setup the ssh trusty relationship between mgmt and other nodes. 



Although this attempt fails, it puts infra, app1, app2 into the known_hosts of mgmt, another way to accomplish this is to ssh to these boxes directly, and enter yes upon prompting.  


To establish ssh trust, first create public and private keys using ssh-keygen -t rsa -b 2048. At prompting, simply hit enter. This will create keys in /home/vagrant/.ssh/id_rsa.pub and /home/vagrant/.ssh/id_rsa.

For the next step we can use Ansible to setup the trusty relationship. Create file ssh-addkey.yml:
---
- hosts: all
  become: yes
  become_method: sudo 
  gather_facts: no

  tasks:
  - name: install ssh key
    authorized_key: user=vagrant
                    key="{{ lookup('file', '/home/vagrant/.ssh/id_rsa.pub') }}"
                    state=present

This yml file is an ansible playbook, despite its format, it follows similar structure as the above ansible command line:
  •  hosts: all: target at all nodes
  •  authorized_key: a module that sets up keys for user (here vagrant) for remote nodes, user, key, state are parameters for this module
Now run this playbook with command ansible-playbook ssh-addkey.yml --ask-pass:

With --ask-pass, ansible will prompt you for password, enter vagrant.

Now the trust relationship is setup. We can say hello again:
This time, it succeeds.

By the way, ansible will cache ssh connections for 10 seconds (configurable in ansible.cfg), you can check out the connections:
So if you say hello again very soon, ansible will reuse the existing ssh connections.

Yml format is very strict, in the beginning this always gets on me (still does):

 
Indention must be lined up, and must be whitespaces. If you use notepad++, you can convert tab to whitespaces in settings/preferences:

 

8 comments:

  1. Thanks for sharing such a wonderful blog on Amazon Web Services .
    This blog contains so much data about Amazon Web Services ,like if anyone who is searching for the Amazon Web Services data,They will easily grab the knowledge from this .Requested you to please keep sharing these type of useful content so that other can get benefit from your shared content.
    Thanks and Regards,
    Amazon Web Services training in Chennai
    Best Amazon Web Services training in chennai
    Top Amazon Web Services Training in chennai
    Amazon Web Services training fees in Velachery,Chennai

    ReplyDelete
  2. Appericated the efforts you put in the content of Azure .The Content provided by you for Azure is up to date and its explained in very detailed for Azure like even beginers can able to catch.Requesting you to please keep updating the content on regular basis so the peoples who follwing this content for Azure . can easily gets the updated data.
    Thanks and regards,
    Azure training in chennai .
    Azure course in chennai with placement .
    Azure course in OMR .
    Azure certification in Chennai.

    ReplyDelete
  3. Appericated the efforts you put in the content of Data Science .The Content provided by you for Data Science is up to date and its explained in very detailed for Data Science like even beginers can able to catch.Requesting you to please keep updating the content on regular basis so the peoples who follwing this content for Data Science can easily gets the updated data.
    Thanks and regards,
    Data Science training in Chennai
    Data Science course in chennai with placement
    Data Science certification in chennai
    Data Science course in Omr

    ReplyDelete

  4. Such a wonderful blog on Amazon Web Services .Your blog having almost full information about Amazon Web Services .Your content covered full topics of Amazon Web Services that it cover from basic to higher level content of Amazon Web Services .Requesting you to please keep updating the data about Amazon Web Services in upcoming time if there is some addition.
    Thanks and Regards,
    Best institute for AWS(Amazon Web Services)
    AWS training fees in Chennai
    AWS training institute in chennai
    Amazon Web Services training fees in OMR, Chennai

    ReplyDelete
  5. Thanks for the information. The information you provided is very helpful for DevOps Training users.

    ReplyDelete