Chef and Puppet each have a bit of a learning curve, and if you only need to provision a few servers, you might think the effort to learn either isn't worth it. You can get far with just writing down your provisioning steps, manually executing them, imaging the resulting server and using that image as the basis for your testing, staging, production and failover. Besides, now you know that these environments are exactly the same, because they are from the same image.
Using images for your deployments is not the problem. In fact, you should be making images and skipping the provisioning step when deploying multiple instances of the same application on a service like EC2. The problem is with relying solely on documentation for building an image from scratch.
With an automation tool, you can destroy and rebuild your local development and test environments quickly and make sure they match your production and staging environments command-for-command without any chance of "fat-fingering" or accidentally skipping a step. This can also make bootstrapping a new developer easier.
There are also a few scenarios where rebuilding your production server from scratch might be needed or is the better option:
- Distribution upgrades --
apt-get dist-upgradeworks, but it's probably best to start from scratch when upgrading the distro
- Moving to a new hosting provider -- not impossible to move an image, but sometimes can cause issues
- Tweaks to your installed libraries or programs -- a lot of time it's best to isolate your tweaks with fresh rebuilds in case of conflicts
- Re-architecting your cluster -- started off with the DB on the same box as the app and now need the DB on its own server
Ansible could be an answer
Ansible is an automation tool that can be as involved or as simple as you want it to be. It stays installed on your local machine and communicates to the servers via SSH, so there is nothing to install on the servers. It's like you logged in and performed the tasks yourself, and I feel that makes it easy to understand.
Another thing that makes Ansible easy to use is the fact that tasks are written in YAML instead of a block-based DSL like Chef. Take the following Ansible task for example:
copy: src=/myfiles/foo.conf dest=/etc/foo.conf owner=foo group=foo mode=0644
An Ansible module has options passed to it much like a command-line tool has flags. The
copy module will take a local file
src and copy it to a remote server location
dest and set the ownership and permissions mode on it. This one line replaces what could be three steps (
This post is just a preview
Without going too much in to how tasks, roles, playbooks, and modules all work together, I want to show you how you can turn a multi-step provisioning guide in to a simple playbook that can be run on a server with single command. Hopefully this will encourage you to learn more about Ansible and ditch the provisioning documentation for automation.
The steps I will be turning in to an Ansible playbook are from the DigitalOcean tutorial on how to add swap to Ubuntu. I will be using the
lineinfile Ansible modules as well as registering a conditional.
Step 1: check for swap space
To check for swap, run the command
sudo swapon -s. It should output the results as either a empty table (with only headers) or a list of swaps.
Filename Type Size Used Priority
With a swap:
Filename Type Size Used Priority /swapfile file 262140 0 -1
To make this command more boolean friendly, we can run it through
grep. If there is no output, there is no swap.
sudo swapon -s | grep -E "^/"
In an Ansible task, we can use the
shell module to run this command. Ansible tasks can also "register" the result of the module to a variable we can use elsewhere. However, this particular case is a little different. Since there is a possibility of the command failing to return any output (remember, no output means no swap currently on the system), we need to ignore the failing error.
shell: 'sudo swapon -s | grep -E "^/"' register: swapfile ignore_errors: yes
Now we only have to check for a swap once. We are doing this because we don't want to run the following steps if there is already a swap on the system.
Step 2: create and enable a swap file
The next command to run is
sudo dd if=/dev/zero of=/swapfile bs=1024 count=256k which will create an empty swapfile at the location
With Ansible, we will use the
shell module again to run this command, but with one catch: we only want to run the command "when" ansible failed to find a swap file (using the variable
swapfile we created in the previous task).
shell: "sudo dd if=/dev/zero of=/swapfile bs=1024 count=256k" when: swapfile|failed
Next is to prepare the swap file with the command
sudo mkswap /swapfile and we will use the
shell module yet again and with the same "when" conditional.
when: swapfile|failed shell: "sudo mkswap /swapfile"
Notice that the when is now on top. Order doesn't matter.
And finally enable the swapfile with
sudo swapon /swapfile, which we will translate to an Ansible task with the
shell module again.
when: swapfile|failed shell: "sudo swapon /swapfile"
Step 3: mount the swap
Next, we want to add the following line to the file
/swapfile none swap sw 0 0
For this, we will use a different module, the
lineinfile module. This module will let us look through the file using a regular expression and make sure it's present (or absent). If you provide a
line= option to the module (required when checking if a line is present), it will insert/replace that line in the file. Since we only want one entry for
/etc/fstab, this is the module we will use.
when: swapfile|failed lineinfile: dest=/etc/fstab regexp="^/swapfile" state=present line="/swapfile none swap sw 0 0"
Modules that take options can be written on new lines (as above) or all in one line (as below). The indention is not required in the above example, but I personally like the way the indention reads.
lineinfile: dest=/etc/fstab regexp="^/swapfile" state=present line="/swapfile none swap sw 0 0"
Step 4: set the system "swappiness"
This will determine how aggressive the system will be about hitting the swapfile. With a setting of "1", the system will swap only to avoid an out of memory condition (kernel 3.5+). If you run a kernel older than 3.5, use "0" for the same behavior. The kernel default value is "60". Setting the value higher results in more aggressive swapping.
To set the swappiness temporarily we will use the
when: swapfile|failed shell: "echo 1 | sudo tee /proc/sys/vm/swappiness"
And to make it permanent so the preference will survive a reboot, we will use the
lineinfile module again.
when: swapfile|failed lineinfile: dest=/etc/sysctl.conf regexp="^vm.swappiness" state=present line="vm.swappiness = 1"
Step 5: set permissions on swap
Lastly, we need to make sure the permissions are set correctly on the
swapfile. While we could just use the shell module again to execute the command in the DigitalOcean tutorial, I'd like to introduce the
file module. This module sets attributes of files, symlinks, and directories, or removes files, symlinks, or directories. With the
file module we can set the owner, group and permissions mode the file at the path
/swapfile all in one task.
when: swapfile|failed file: path=/swapfile owner=root group=root mode=0600
Putting it in a playbook
If you want to try out the following playbook, install Ansible with
brew install ansible on a Mac, or with
pip (more details about installation can be found in the Ansible documentation).
An Ansible playbook is just a YAML file with a little bit of ceremony. At the very least you need need a
host and list of
tasks. Also since the tasks in this playbook require
sudo access, we need to note that.
- hosts: all sudo: yes tasks: - name: "test for swap partition" shell: 'sudo swapon -s | grep -E "^/"' register: swapfile ignore_errors: yes - name: "create swapfile" when: swapfile|failed shell: "sudo dd if=/dev/zero of=/swapfile bs=1024 count=256k" - name: "set swapfile permissions" when: swapfile|failed file: path=/swapfile owner=root group=root mode=0600 - name: "prepare swapfile" when: swapfile|failed shell: "sudo mkswap /swapfile" - name: "enable swap" when: swapfile|failed shell: "sudo swapon /swapfile" - name: "add swapfile" when: swapfile|failed lineinfile: dest=/etc/fstab regexp="^/swapfile" state=present line="/swapfile none swap sw 0 0" - name: "set swappiness (temporarily)" when: swapfile|failed shell: "echo 1 | sudo tee /proc/sys/vm/swappiness" - name: "set swappiness (permanent)" when: swapfile|failed lineinfile: dest=/etc/sysctl.conf regexp="^vm.swappiness" state=present line="vm.swappiness = 1"
The three dashes at the top of the file are not really required. They help some editors and processors note that it is a YAML file. I put them in out of habit. While YAML is simple format, it does have arrays (items in a YAML array start with a
-). An Ansible playbook is an array (note
- hosts: and everything else is indented.), and
tasks:, within a playbook, is an array of tasks (each new tasks is denoted by
- name: ... in the above playbook). The
name for each task is not required, but I do like to add a name to the top of each task for documentation. This is what will scroll by when Ansible is running through the tasks.
There are a few thing we could do to DRY up this playbook such as extract variables like the swappiness value, moving all tasks to a role and redo how the conditional works, but that's for another post.
Running the play
ansible-playbook swapfile.yml -i 192.168.1.123,
-i for passing in a server IP address or domain name instead of using an inventory file, and make sure there is a comma after the IP address or domain name (it's a quirk).
Also, in the playbook above, the
hosts value is set to "all." That means when it runs, it will run the tasks on all the servers in your inventory list. For adding more servers inline, write them out in a comma separated list.
ansible-playbook swapfile.yml -i 192.168.1.123,192.168.1.124
The above command works if you SSH in to the server with the same username that log in to your local machine with, use ssh keys for authentication and have sudo access with no password required. You can add flags to the command if this ideal scenario isn't the case.
ansible-playbook swapfile.yml -i 192.168.1.123, -u jonathan --ask-pass --ask-sudo-pass
The take away
Ansible has many built-in modules to make tasks simpler, more readable, and more uniform across different distribution families, but you can always fallback on directly translating your actions to one of a small handful of modules like
file. We could've used the
mount module or the
sysctl module in this guide. Be sure to skim through the Ansible module index to see if there is anything that will make your task creation simpler.
With a nice collection of modules and a very readable and rememberable syntax, Ansible can replace provisioning notes and command snippets with something almost as readable but with the power to be executed across many servers all at once with a single command, thus reducing the amount of time provisioning would normally take and facilitating in keeping all those servers in sync.