ansible

Contents

ansible

About

Installation

   1 aptitude install ansible ansible-doc \
   2         ansible-lint yamllint python3-jmespath

Configure

Initialize playbook structure

Initialize directory structure as stated in
docs.ansible.com - sample setup

   1 install -m 755 -d \
   2         group_vars host_vars \
   3         inventory/prod/{group,host}_vars \
   4         library module_utils \
   5         filter_plugins roles
   6 
   7 ansible-config init --disabled > ansible.cfg
   8 
   9 FILES=( "inventory/prd.yml" "site.yml" "README.md" ".gitignore" )
  10 for FILE in "${FILES[@]}"; do
  11         install -m 644 /dev/null "$FILE";
  12 done
  13 
  14 git init
  15 #git remote add …
  16

Initialize role structure

   1 ### INITIALIZE A NEW ROLE NAMED "base"
   2 #ansible-galaxy init --init-path roles role-example
   3 
   4 ### MOUNT A REPO "ansible_role_base" AS SUBMODULE
   5 #git submodule add git@git.rockstable.it:rockstableit/ansible_role_base.git roles/base
   6 #git commit
   7 #git push
   8

Use

Check playbook

   1 ansible-lint site.yml
   2 yamllint inventory_prd.yml
   3 ### WITHOUT AN INVENTORY (IMPLICTLY) ONLY LOCALHOST IS AVAILABLE
   4 ansible-playbook --syntax-check site.yml

Dry-run with differences

   1 ### LONG
   2 ansible-playbook --verbose \
   3         --inventory inventory_prd.yml \
   4         --ask-become-pass --diff --check \
   5         --limit debian-jessie \
   6         site.yml
   7 ### SHORT
   8 ansible-playbook -vv \
   9         -i inventory_prd.yml \
  10         -KDC \
  11         -l debian-jessie \
  12         site.yml

When you run ansible-playbook without --ask-become-pass|-K with a template-task you may get a wrong error message: "Failed to get information on remote file".

Facts about host

Ad-hoc commands

   1 mkdir facts
   2 echo 'facts/*' >> .gitignore
   3 export HOSTNAME=hostname
   4 ansible -i inventory_prd.yml \
   5         -m setup "$HOSTNAME" \
   6         > facts/"$HOSTNAME".json
   7 ### DEFAULT MODULE IS  "-m command"
   8 ansible -i inventory/prd.yml all \
   9         --ask-vault-pass \
  10         -a 'lsb_release -a'

Debugging types

ansible.builtin.type_debug filter – show input data type

Sometimes a Dict is not what it should be.

   1 MSG:
   2 
   3 'str object' has no attribute 'os_releases'. 'str object' has no attribute 'os_releases'

Debug the Dict

   1 - name: "Debug: Dict"
   2   ansible.builtin.debug:  
   3     msg:
   4       - "repo: {{ repo }}"
   5       - "repo type: {{ repo|type_debug }}"
   6   tags:
   7     - debug_dict

   1 ['repo: "{\'os_releases\': [{\'provider\': \'core\', \'release\': \'trixie\', \'priority\': 400}, {\'provider\': \'core\', \'release\': \'bookworm\', \'priority\': 500}, {\'provider\': \'core\', \'release\': \'bullseye\', \'priority\': 300}, {\'provider\': \'core\', \'release\': \'sid\', \'priority\': 120}, {\'provider\': \'core\', \'release\': \'experimental\', \'priority\': 110}], \'3rd_party_repos\': [\'icinga2\', \'grafana\']}"\n', 'repo type: str']
   2

It is of type str.

Please be careful when quoting

   1 - name: "Combine 'repo_defaults' with user defined 'repo' recursively"
   2   ansible.builtin.set_fact:
   3     repo: >
   4       "{{ repo_defaults
   5       | combine(repo | default({}), recursive=True) }}"
   6   tags:
   7     - repo_combine             
   8     - base_prepare

The double quotes "" cast the Dict to Str.

A more correct version

   1 - name: "Combine 'repo_defaults' with user defined 'repo' recursively"
   2   ansible.builtin.set_fact:
   3     repo: >
   4       {{ repo_defaults
   5       | combine(repo | default({}), recursive=True) }}
   6   tags:
   7     - repo_combine             
   8     - base_prepare

So please keep a look on the type.

Client

https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html#connecting-to-hosts-behavioral-inventory-parameters

Jinja2

https://jinja.palletsprojects.com/en/stable/
- Jinja2 - Template Designer Documentation

Jinja is a fast, expressive, extensible templating engine. Special placeholders in the template allow writing code similar to Python syntax. Then the template is passed data to render the final document.

Jinja2 highlighting

Enable vim#Modelines and append to your jinja2-template

   1 {# vim: syntax=htmldjango
   2 #}
   3

Jinja2 linter - j2lint

github.com aristanetworks/j2lint

Thanks for the linter!

Debugging Jinja2 is not easy.
You'll write better templates.

Install

   1 pipx --global install j2lint

Hints

Jinja2 templates should have the filename extensions ['.j2', '.jinja', '.jinja2'] (not .jn2)
- You may add an extension with the option -e|--extension jn2

Usage

   1 j2lint -e jn2 -v -- path/to/directory/file.jn2
   2 j2lint -e jn2 -v -- path/to/directory
   3 j2lint -e jn2 -v -i jinja-statements-no-tabs -- path/to/directory
   4 j2lint -e jn2 \
   5         -i jinja-statements-no-tabs jinja-statements-indentation \
   6         -w jinja-statements-delimiter \
   7         -- path/to/directory/file.jn2
   8 j2lint -i jinja-statements-no-tabs jinja-statements-indentation \
   9         -w jinja-statements-delimiter \
  10         -- .

Macros

Boolean to "yes" or "no" else string

Many configs do not use booleans, instead they use "yes" or "no"

   1 {% macro render_bool_or_str(value = None) -%}
   2 {% if value is boolean and
   3 value is true %}
   4 yes
   5 {%- elif value is boolean and
   6 value is false %}
   7 no
   8 {%- else %}
   9 "{{ value | string }}"
  10 {%- endif %}
  11 {%- endmacro %}
  12 
  13 var = {{ render_bool_or_str(__variable | default("true")) }}

Comment undefined

   1 {% macro comment_undef(key = None) %}
   2 {% if key is not defined %}
   3 ##
   4 {%- endif -%}
   5 {%- endmacro -%}
   6 
   7 {{ comment_undef(__variable) -}}
   8 var = {{ __variable }}

Hints

General

truthy value should be one of [false, true] (lowercase) Ansible Docs - Using Variables Boolean variables
Jinja2 variables and filters should have spaces before and after e.g.
{{ variable | filter }}
fqcn-builtins: Use FQCN for builtin actions
(fully qualified collection names) e.g.
ansible.builtin.template
Put a newline on every file
- Correct count of lines, correct concatenation, simplifies diffs, required by some language standards

Pretty output

ANSIBLE_STDOUT_CALLBACK=debug ansible-playbook …

./ansible.cfg

   1 ## (string) Set the main callback used to display Ansible output, you can only have one at a time.
   2 ## You can have many other callbacks, but just one can be in charge of stdout.
   3 #stdout_callback=default
   4 stdout_callback=debug

ansible-lint

https://ansible-lint.readthedocs.io/

Use ansible-lint to check your playbooks, roles and tasks!

Some handy commands to satisfy ansible-lint

   1 ### WRITE THE LINTER OUTPUT TO A LOG FILE
   2 LOG="/tmp/ansible-lint.log"
   3 ansible-lint site.yml --nocolor > "$LOG"
   4 
   5 ### YOU MAY ALSO SKIP SOME RULES
   6 ansible-lint site.yml --nocolor \
   7         --skip-list 'yaml[comments]' > "$LOG"
   8 
   9 ### IDENTIFY FILES
  10 grep -A1 -e 'trailing-spaces' "$LOG" \
  11         |grep -v -e 'yaml' -e '--' \
  12         |sed -r 's/:[0-9]+$//'0, ß
  13 
  14 ### DELETE TRAILING SPACES
  15 grep -A1 -e 'trailing-spaces' "$LOG" \
  16         |grep -v -e 'yaml' -e '--' \
  17         |sed -r 's/:[0-9]+$//'
  18         |sort -u \
  19         |xargs sed -ri 's/\s+$//'
  20 
  21 ### REPLACE BOOLEAN VALUES INPLACE WITH true AND false
  22 grep -A1 -e 'truthy value' "$LOG" \
  23         |grep -v -e 'yaml' -e '--' \
  24         |sed -r 's/:[0-9]+$//' \
  25         |sort -u \
  26         |xargs sed -ri \
  27                 -e 's/: (yes|on|True)$/: true/' \
  28                 -e 's/: (no|off|False)$/: false/'
  29 
  30 ### TOO MANY SPACES AFTER COLON
  31 grep -A1 -e 'too many spaces after colon' "$LOG" \
  32         |grep -v -e 'yaml' -e '--' \
  33         |sed -r 's/:[0-9]+$//' \
  34         |sort -u \
  35         |xargs sed -ri -e 's/: +/: /'
  36 
  37 ### APPEND NEW LINE TO FILES
  38 grep -A1 -e 'no new line character at the end of file' "$LOG" \
  39         |grep -v -e 'yaml' -e '--' \
  40         |sed -r 's/:[0-9]+$//' \
  41         |sort -u \
  42         |xargs -I{} bash -c 'echo -e '\n' >> {}'
  43 
  44 ### EXAMINE IN VIM
  45 grep -A1 -e 'wrong indentation' "$LOG" \
  46         |grep -v -e 'yaml' -e '--' \
  47         |sed -r 's/:[0-9]+$//' \
  48         |sort -u \
  49         |xargs vim -p

Accept SSH TOFU

If you have many hosts and don't want to abandon strict host checking.

   1 for HNAME in $(ansible -i hosts.yml --list-hosts all \
   2         | tail -n +2 | grep -v -e exemption); do
   3         ssh $HNAME -- \
   4                 echo\; echo '"$(hostname -f) done"'\; echo;
   5 done 2>&1 | tee ssh.txt

Logging

It's handy to have logs, when

you run on errors
working in tmux (with a limited buffer)
you need to have documentation

Create an alias llog (like last log) in the shell configuration file which looks into the relative directory log, finds the newest file and pages it with nice colors ~/.bashrc

   1 alias llog='ls -rt1 log/* | tail -n1 | xargs less -R'

Source the config

   1 source ~/.bashrc

Create a log directory

   1 [ -d log ] || mkdir log

Run the playbook with colored output enabled and log it to a file

   1 ANSIBLE_STDOUT_CALLBACK=debug \
   2 ANSIBLE_FORCE_COLOR=true \
   3 ansible-playbook -i inventory/test.yml \
   4         -vv --diff --check \
   5         src/main.yml \
   6         | tee "log/run_$(date +%F_%H%M%S).log"

Open the latest log

   1 llog

ANSI escape code may be removed using
oneliners#Strip_ANSI_control_codes

sudo NOPASSWD: (devel)

During the development of a ansible playbook on your test-VM you'll often have to type the sudo password, which disturbs the "flow" massively. It's probably simpler to temporarily disable the password prompt in sudoers and leave out --ask-become-pass|-K on the ansible-playbook command-line.

You can edit /etc/sudoers relatively safe using visudo, but consider leaving another root-shell open - just in case.

%sudo   ALL=(ALL:ALL) NOPASSWD: ALL

Speeds up development, but you shouldn't do this in production, imho.

Vaults

Use vaults!!!
Protect your secrets when using version control
Save the vault password in pass

Generate a password

   1 pass generate -nc RockstableIT/ansible/ansible-monitoring/vault-pass

Encrypt file

   1 ansible-vault encrypt host_vars/file_with_credentials.yml
   2 New Vault password: 
   3 Confirm New Vault password: 
   4 Encryption successful

Use it with --ask-vault-pass

   1 ansible-playbook -i inventory/prd.yml -K \
   2         --ask-vault-pass \
   3         --diff --check site.yml

Or use it with --vault-password-file, that must not be stored in git (or it will probably get added someday). Please keep in mind, that any file may be read by a superuser …

   1 VPF="$(mktemp)"
   2 [ -f "$VPF" ] \
   3         && pass RockstableIT/ansible/ansible-monitoring/vault-pass \
   4                 > "$VPF"
   5 ansible-playbook -i inventory/prd.yml -K \
   6         --vault-password-file "$VPF" \
   7         --diff --check site.yml

Break long lines

Satisfy ansible- and yaml-linter.

Example first

   1 - name: "apache2 - enable configurations"
   2   become: true
   3   command: a2enconf "{{ item }}"
   4   register: a2enconf_result
   5   loop:
   6     - "{{ php_fpm.stdout }}"
   7     - security
   8   when: php_fpm.stdout is defined
   9   changed_when: >-
  10     'Enabling conf ' + php_fpm.stdout + '.'
  11     in a2enconf_result.stdout_lines
  12   notify:
  13     - restart apache2
  14   tags:
  15     - apache2_a2enconf

Jinja2

Jinja2 templating in YAML breaks long lines and adds a tremendous amount of flexibility!

Conditional statements

Conditional statements like when: or changed_when: should not include jinja2 templating delimiters such as "{{ }}".

Boolean Tests

You don't need to compare with true and false, the evaluated variable is sufficient and shorter.

   1 - vars:
   2     bool_test: True
   3   debug:
   4     msg: "'bool_test' is true"
   5   when:
   6     - bool_test == true

is the same as

   1 - vars:
   2     bool_test: True
   3   debug:
   4     msg: "'bool_test' is true"
   5   when:
   6     - bool_test

Ansible tests

YAML multiline

Use YAML Block Chomping and jinja2!

Directly from the YAML specification

8.1.1.2. Block Chomping Indicator

Chomping controls how final line breaks and trailing empty lines are interpreted. YAML provides three chomping methods:

Strip

Stripping is specified by the “-” chomping indicator.
- In this case, the final line break and any trailing empty lines are excluded from the scalar’s content.

Clip

Clipping is the default behavior used if no explicit chomping indicator is specified.
- In this case, the final line break character is preserved in the scalar’s content. However, any trailing empty lines are excluded from the scalar’s content.

Keep

Keeping is specified by the “+” chomping indicator.
- In this case, the final line break and any trailing empty lines are considered to be part of the scalar’s content. These additional lines are not subject to folding.

The chomping method used is a presentation detail and must not be used to convey content information.

   1 [164]   c-chomping-indicator(t)         ::=     “-”         ⇒ t = strip
   2                                                 “+”         ⇒ t = keep
   3                                                 /* Empty */ ⇒ t = clip

The interpretation of the final line break of a block scalar is controlled by the chomping indicator specified in the block scalar header.

   1 [165]   b-chomped-last(t)       ::=     t = strip ⇒ b-non-content  | /* End of file */
   2                                         t = clip  ⇒ b-as-line-feed | /* End of file */
   3                                         t = keep  ⇒ b-as-line-feed | /* End of file */

ansible-galaxy

Install a role from ansible-galaxy

Ansible Docs - Galaxy User Guide

Manual installation

By default, Ansible downloads roles to the first writable directory in the default list of paths ~/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles. This installs roles in the home directory of the user running ansible-galaxy.

Manual installation

   1 ### WITHIN THE FIRST WRITEABLE PATH
   2 ansible-galaxy install -vvv \
   3          grog.management-user
   4 ### TO A SPECIFIC roles_path
   5 ansible-galaxy install -vvv \
   6          --roles-path roles \
   7          grog.management-user

Install using requirements.yml

Installation with a
requirements.yml

   1 ---
   2 # Please see:
   3 # https://docs.ansible.com/ansible/latest/galaxy/user_guide.html#install-multiple-collections-with-a-requirements-file
   4 # https://docs.ansible.com/ansible/latest/galaxy/user_guide.html#installing-multiple-roles-from-a-file
   5 roles:
   6   - name: grog.management-user
   7     #src:
   8     #scm:
   9     #version:
  10 
  11 #collections:
  12 # With just the collection name
  13   #- name:
  14   #  version:
  15   #  signatures:
  16   #  source:
  17   #  type:
  18

Install

   1 ### WITHIN THE FIRST WRITEABLE PATH
   2 ansible-galaxy -vvv install -r requirements.yml
   3 ### TO A SPECIFIC roles_path
   4 ansible-galaxy -vvv install -r requirements.yml --roles-path roles

Remove roles

Remove installed ansible-galaxy role

   1 ### REMOVE FROM SPECIFIC roles_path
   2 cd roles
   3 ansible-galaxy -vvv remove --roles-path . GROG.*
   4 
   5 ### REMOVE FROM SPECIFIC roles_path
   6 find roles \
   7         -maxdepth 1 -type d \
   8         -iname 'grog*' \
   9         |xargs -L1 basename \
  10         |xargs ansible-galaxy -vvv remove --roles_path roles
  11 
  12 ### REMOVE FROM STANDARD INSTALLATION PATHS
  13 find ~/.ansible/roles /usr/share/ansible/roles /etc/ansible/roles \
  14         -maxdepth 1 -type d \
  15         -iname 'grog*' \
  16         |xargs -L1 basename \
  17         |xargs ansible-galaxy -vvv remove

ansible

About

Installation

Configure

Initialize playbook structure

Initialize role structure

Use

Check playbook

Dry-run with differences

Facts about host

Debugging types

Client

Jinja2

Jinja2 highlighting

Jinja2 linter - j2lint

Macros

Boolean to "yes" or "no" else string

Comment undefined

Hints

General

Pretty output

ansible-lint

Accept SSH TOFU

Logging

sudo NOPASSWD: (devel)

Vaults

Tags

Break long lines

Example first

Jinja2

Conditional statements

Boolean Tests

YAML multiline

8.1.1.2. Block Chomping Indicator

ansible-galaxy

Install a role from ansible-galaxy

Manual installation

Install using requirements.yml

Remove roles