Tratif

In this post, I am going to introduce one of the tools which we use here at Tratif for automating software installations.

Most of our projects written in Java require a lot of third party services to function (e.g. MySQL, RabbitMQ, Redis). This imposes a mandatory installation step to be performed before we deploy our Java services to a new machine. We call this step, as many of you probably do, provisioning.

Having provisioned a machine, we can deploy the layer with our software, so the solution is fully operational and the machine can be sent to the client.

Sounds like a simple two-step process – provision and deploy…

But, how do you actually go about provisioning and deployment? How do you handle dozens of installations that vary slightly or significantly? Finally, how do you track the differences between installations so that you know what is deployed where?

The last question is especially important when you are operating in an environment, where frequent deployments are not allowed. Although I wish all of you to live happily in Continuous Delivery world, it is not always possible.

Let us see how we can tackle some of these problems with Ansible.

What is Ansible?

Ansible is a tool that provides an automation language to engineers for managing IT infrastructure. It is very flexible and lightweight. It supports great variety of operating systems, has a rich library of modules and is fully customizable. Its agent-less nature means that you do not have to install anything on the target server. All you need to provide is SSH access to it. With that in place Ansible is able to connect to the server and perform tasks specified in provisioning and deployment scripts.

There are several basic concepts which will be covered in this article:

• inventories and hosts
• playbooks
• variables
• vaults

Inventories and hosts

An inventory file defines a specific installation and hosts on which that installation should be performed. A host is a concrete machine. Inventory file allows you to define the actual host machines info by using details such as: IP address, SSH connectivity details, etc. It also gives you a way of assigning an inventory to a group. Groups will be described later.

Here is an example inventory file with two host machines – multi-node-installation:

[multi-node-installation]
app-server ansible_ssh_host=192.168.0.2 ansible_ssh_port=22 ansible_ssh_user=user ansible_ssh_private_key_file=~/.ssh/key.pem
db-server ansible_ssh_host=192.168.0.3 ansible_ssh_port=22 ansible_ssh_user=root

[app-host]
app-server

[db-host]
db-server

[app-hosts:children]
app-host

[db-hosts:children]
db-host

...

The above file defines an inventory named multi-node-installation which consists of two host machines named: app-server and db-server. Those hosts are assigned to two groups, respectively: app-hosts and db-hosts.

The :children suffix states that all variables defined for a group should be used by Ansible at runtime.

Playbooks

Automation can be thought of as a type of orchestration of tasks.

Playbook is a term used mostly in theatres, but it suits automation tasks quite well. A playbook is a specification of tasks which need to be played on a specific stage, i.e. a host machine.

Let us see an example playbook – play.yml:

- hosts: app-hosts
  become: yes
  tasks:
    - name: creating user my_user
      user:
        name: my_user
        state: present
    - name: creating group my_group
      user:
        name: my_group
        state: present
    - name: creating directory for my_service and its configuration
      file:
        path: /opt/my_service/configuration
        state: directory
        owner: my_user
        group: my_group
    - name: creating a configuration file for my_service
      copy:
        content: "{{ config_content }}"
        dest: /opt/my_service/configuration/my_service.conf
        owner: my_user
        group: my_group

- hosts: db-hosts
  become: yes
  tasks:
    - name: installing database
      yum:
        name: mysql-server
        state: latest

Here is the explanation of what each of the lines mean:

Line 1 – specifies all inventories and hosts on which this part of playbook is played – in our example, this will be executed on app-server machine only as per the setup in inventory file

Line 2 – specifies that all the tasks shall be ran as root user

Line 3 – specifies a start of a section with list of tasks to be acted out

Lines 4 to 23 – show a list of four tasks

Lines 5, 9, 13 and 19 – show name of module executed for a task. Below these lines are lines with parameters to the modules

Line 20 – shows a way of accessing a variable config_content (you will find more about variables later on) using a templating engine named jinja supported by Ansible

So, how do you actually play this playbook using Ansible? Here is how:

$ ansible-playbook play.yml -i multi-node-installation

The -i parameter specifies the inventory on which play.yml should be executed.

And here is an excerpt from running the playbook:

___________________
 PLAY [app-hosts] 
-------------------
 \  ^__^
  \ (oo)\_______
    (__)\       )\/\
        ||----w |
        ||     ||

________________________
 TASK [Gathering Facts] 
------------------------
 \  ^__^
  \ (oo)\_______
    (__)\       )\/\
        ||----w |
        ||     ||

ok: [app-server]
 ______________________________
  TASK [creating user my_user] 
 ------------------------------
 \  ^__^
  \ (oo)\_______
    (__)\       )\/\
        ||----w |
        ||     ||

changed: [app-server]
 ________________________________
  TASK [creating group my_group] 
 --------------------------------
 \  ^__^
  \ (oo)\_______
    (__)\       )\/\
        ||----w |
        ||     ||

changed: [app-server]
 __________________________________________
  TASK [creating directory for my_service] 
 ------------------------------------------
 \  ^__^
  \ (oo)\_______
    (__)\       )\/\
        ||----w |
        ||     ||

changed: [app-server]
 _____________________________________________________
  TASK [creating a configuration file for my_service] 
 -----------------------------------------------------
 \  ^__^
  \ (oo)\_______
    (__)\       )\/\
        ||----w |
        ||     ||

changed: [app-server]
 _________________
  PLAY [db-hosts] 
 -----------------
 \  ^__^
  \ (oo)\_______
    (__)\       )\/\
        ||----w |
        ||     ||

____________________________
 TASK [installing database] 
----------------------------
\  ^__^
 \ (oo)\_______
   (__)\       )\/\
       ||----w |
       ||     || 

changed: [app-server]
 ____________
  PLAY RECAP 
 ------------
 \  ^__^
  \ (oo)\_______
    (__)\       )\/\
        ||----w |
        ||     ||

app-server : ok=7 changed=0 unreachable=0 failed=0

Ansible outputs the tasks that it is running. It displays a line with status of a task: ok, changed (failed and reachable). At the end, there is a play recap with summed up statuses of all tasks.

An important feature of Ansible is tasks idempotency. In case the installation fails for example during fifth task, you can re-run the playbook (after fixes) and Ansible will notice that the results of the first four tasks are already on the host, so it will not re-apply them.

Variables

Ansible allows you to define multitude of variables which can be used to determine properties such us software versions, configuration parameters, toggle switches, etc. You can define anything and the types of variables supported are extensive – string, boolean, list, dictionary. You can even define a multi-line string directly in a variable and later on use that content to fill a file on a machine.

Variable files allow you to configure a deployment for given environment requirements.

Let us see an example vars file:

my_service_version: '1.0.0'

is_ssl_enabled: yes
is_database_encrypted: False

feature_toggles: ['auto-save', 'newsletter', ...]

my_service_config:
  jar_service_name: 'my_service'
  http_proxy:
    port: 9050

ssl_certificate: |
  -----BEGIN CERTIFICATE-----
  AXfHRTSFjGdRTfgghj...

The syntax used for variable files is YAML.

Managing sensitive data – Ansible Vault

Deployment-specific data should be kept in variables, so that your scripts can be re-used for multiple installations (e.g. for test and prod environments). This often includes sensitive information, such as passwords. Typically, provisioning and deployment scripts are kept in a version control system such as Git, though. You probably do not want to give access to all passwords to everybody who has access to the repository. How can we store these variables in a safe way then?

Ansible allows you to encrypt files using its vault feature. Put all passwords, private SSH keys and other values that need to be securely stored in a text file. Then, use Ansible’s vault tool to encrypt and decrypt it.

You can define a vault password file that Ansible can use to encrypt and decrypt your vaults or provide a password via prompt each time. They can be passed directly between your team members.

Example vault_password file:

this is a vault password

Keep the above file secure!

Using Ansible’s vault feature with vault password file:

// encrypting a vault
$ ansible-vault encrypt vault_file --vault-password-file=./vault_password

// decrypting a vault
$ ansible-vault decrypt vault_file --vault-password-file=./vault_password

// editing a vault - this one allows you to make changes in a cmdln text editor like VI and upon closing the file Ansible will encrypt it back
$ ansible-vault edit vault_file --vault-password-file=./vault_password

Encrypted files will look similarly to the following example:

$ cat vault_file

 $ANSIBLE_VAULT;1.1;AES256
 63316233393232383033613231336535343662636538656234626165636461363135313638643231
 343665333064336164383...

One downside of vaults is that every time you encrypt it, even without any changes to the content, the ciphered content will change. This causes a VCS system to assume the file needs to be committed. It is sometimes burdensome when you want to quickly check a variable that happens to be in a vault file. Our practice is: decrypt the vault file, grab the variable value, and ask VCS to re-load the file from repository.

Example project structure

All the above files and concepts form a “provisioning and deployment automation” project, which can be stored in any VCS.

Let us see an example with nicely cut responsibilities:

provision/
  play.yml           - playbook for provisioning installation step
deploy/
  play.yml           - playbook for deploy installation step
inventories/         - folder with inventories data
  group_vars/        - a folder holding all group variables
    app-hosts/
      vars           - variables file for app-hosts
      vault          - vault file for app-hosts
    db-hosts/        - another group folder; you do not need to have a vault file everywhere
      vars
  multi-node-installation  - the inventory file

With the above structure, you can run the two steps:

$ ansible-playbook provision/play.yml -i inventories/multi-node-installation --vault-password-file=./vault_password
...
$ ansible-playbook deploy/play.yml -i inventories/multi-node-installation --vault-password-file=./vault_password

The division of provisioning and deployment steps is introduced not only to clearly cut responsibilities. It also gives an additional benefit which is being able to update only our software components on a machine which has been provisioned in the past. Typically, changes are done less often to the third party services than to internal components (just think how many times you change the database in your module as opposed to how often you introduce a new feature). This approach is a good practice as it saves a lot of time when executing the scripts.

Summary

As you saw, Ansible is a pretty handy tool. It allows you to define inventories, group them, specify variables for each group, encrypt files with sensitive data, define playbooks, and most importantly – run everything with a single command line.

There is a lot more to cover, so stay tuned for new posts on automating installations!