casual/Casual_blog
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8948aa7d02
Casual_blog/content/tech/HowTo_Ansible.md
casual 044ff1e465 new post
2024-11-26 22:50:29 +03:00

573 lines
19 KiB
Markdown
Raw Blame History

+++
title = 'HowTo Ansible'
date = 2024-06-14
hidden = false
+++
<!-- &nbsp; -->
## WhatIs Ansible?
You can think of `ansible` as a more easier and human-readable bash scripts that can run on almost any remote machine (E.g. GNU/Linux, network router/switch, Windows...).
__It can do anything__ that can be done via terminal, but more reliably and on many different hosts (for instance - with different OS).
## Terminology
- __Control node__ - machine which will run ansible
- __Inventory__ - list of remote host, can be devided to groups.
- __Fact__ - info about remote host.
- __"Gather Facts"__ - automatic procces that gathers all info about system to use in tasks.
- __Task__ - script.
- __Playbook__ - main script __file__.
- __Handler__ - operation that run when task successfully changed configuration (e.g. reload nginx service if config changed)
- __Role__ - profile which contains scripts and files for host group
### Roles structure
```sh
./roles
└── [RoleName]
├── defaults
│ └── main.yml
├── files
├── handlers
│ └── main.yml
├── meta
│ └── main.yml
├── README.md
├── tasks
│ └── main.yml
├── templates
├── tests
│ ├── inventory
│ └── test.yml
└── vars
└── main.yml
```
- `defaults` - Folder to store files with variables with default values
- `files` - Folder to store static files (e.g. `index.html`)
- `handlers` - Folder for handlers
- `meta` - Folder for role metadata (information about authors, licenses, compatibilities, and dependencies). If the current role relies on another role, that gets declared in a meta folder.
- `README.md` - Documentation (Info about role)
- `tasks` - Folder for tasks, you can create other tasks in this folder e.g. `configure_NGINX.yml`
- `templates` - Folder to store dynamic files, that Ansible will edit based on facts and variables. Uses template enigne `Jinja2` and thus files in this folder should end with `.j2` extension (e.g. `nginx.conf.j2`)
- `tests` - Folder contains test environment with `inventory.ini` and `playbook.yml` to test role. Primarily used in CI, you probably won't need it in the begining
- `vars` - Folder for files with variables
## Guide
In this example we have x2 GNU/Linux (Debian) PCs and 1 vyOS router
Task: dump config of a router and write role for PCs which will:
- Install browser if it isn't installed
- Write its version in terminal during run of Ansible
- Install Caddy webserver (and upload static config Caddyfile file)
- Generate webpage for Caddy based on a template and info from PC
- Restart Caddy webserver if we made any changes
### Preparation
1. Install Ansible on Controller node - `pip install ansible`
2. Install Python on Unix/Windows machines if it isn't installed (not strictly necessary, but otherwise need a workaround)
3. Create project folder - `mkdir projectName && cd projectName`
4. Make Git repo - [HowTo Git](/tech/howto_git)
5. Generate public SSH key (if it doesn't exist) - `ssh-keygen`, press x3 ENTER
6. Copy public SSH key to remote hosts (way depends per OS) - (GNU/Linux) `ssh-copy-id [TargetIP]`
### Creating Ansible project
1. Create `inventory.ini`:
```yaml
[PCs] # host group name
192.168.0.11 #debian11
192.168.0.5 #debian12
[routers]
192.168.0.13 #vyOS 1.5
```
2. Verify inventory file - `ansible-inventory -i inventory.ini --list`
Example output:
```yaml
{
"PCs": {
"hosts": [
"192.168.0.11",
"192.168.0.5"
]
},
"_meta": {
"hostvars": {}
},
"all": {
"children": [
"ungrouped",
"PCs",
"routers"
]
},
"routers": {
"hosts": [
"192.168.0.13"
]
}
}
```
3. Check that all hosts accessable by ansible - `ansible all -m ping -i inventory.ini`
```sh
192.168.0.13 | UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh: casual@192.168.0.13: Permission denied (publickey,password).",
"unreachable": true
}
192.168.0.11 | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"ping": "pong"
}
192.168.0.5 | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"ping": "pong"
}
```
As you can see, we have `UNREACHABLE!` host - it's router and we can't access it because it doesn't have SSH user `casual`. We have 3 ways for fixing it:
+ per cli run - `ansible all -m ping -i inventory.ini --user TARGETUSER --ask-pass `
+ per host - edit `inventory.ini`:
```yaml
[PCs] # host group name
192.168.0.11 #debian11
192.168.0.5 #debian12
[routers]
192.168.0.13 ansible_user=vyos #we set SSH access user to "vyos" vyOS 1.5
```
+ per hosts group/all hosts - edit `inventory.ini`:
```yaml
[PCs] # host group name
192.168.0.11 #debian11
192.168.0.5 #debian12
[routers]
192.168.0.13 #vyOS 1.5
[routers:vars]
ansible_user=vyos #SSH user to connect
[all:vars]
ansible_user=ssh_connect_user #SSH user to connect
```
If we would have many hosts in `routers` we would use solution 3, but we can use solution 2.
Run `ansible all -m ping -i inventory.ini` again
If you still get `Permission denied` - then you forgot to `ssh-copy-id vyos@192.168.0.13`
3. Create `playbook.yaml`
We will start with easiest task - dump config from router.
`playbook.yaml`:
```yaml
- name: dump config from router
hosts: routers
tasks:
- name: Dump config
vyos.vyos.vyos_config:
backup: yes
backup_options:
dir_path: ./router_backup
filename: ./backup.cfg
```
All `.yaml` files uses `YAML` so exact number of spaces IS IMPORTANT
Then run playbook - `ansible-playbook -i inventory.ini playbook.yaml`
```sh
PLAY [dump config from router] **************************************************************
TASK [Gathering Facts] **********************************************************************
ok: [192.168.0.13]
TASK [Dump config] **************************************************************************
fatal: [192.168.0.13]: FAILED! => {"changed": false, "msg": "Connection type ssh is not valid for this module"}
PLAY RECAP **********************************************************************************
192.168.0.13 : ok=1 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
```
As you can see, we have error "Connection type ssh is not valid for this module". The thing about this, we interact via SSH with router (not normal bash shell) and should inform about this ansible - `inventory.ini`:
```yaml
...
[routers:vars]
ansible_user=vyos
ansible_network_os=vyos # router OS
ansible_connection=network_cli # we inform that it's not regular bash shell
```
Run again:
```sh
ansibleExample ➤ ansible-playbook -i inventory.ini playbook.yaml git:master*
PLAY [dump config from router] **************************************************************
TASK [Gathering Facts] **********************************************************************
ok: [192.168.0.13]
TASK [Dump config] **************************************************************************
changed: [192.168.0.13]
PLAY RECAP **********************************************************************************
192.168.0.13 : ok=2 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
ansibleExample ➤ ls git:master*
inventory.ini playbook.yaml router_backup
ansibleExample ➤ ls router_backup git:master*
backup.cfg
ansibleExample ➤ cat backup.cfg git:master*
set interfaces ethernet eth1 address 'dhcp'
...
```
We successfully backuped config!
But where did I found `vyos.vyos.vyos_config` and its parameters?
__Search in [Ansible Docs](https://docs.ansible.com/ansible/latest/index.html) or Google.__
4. Create role for PCs - `mkdir roles; cd roles; ansible-galaxy init [Name]`
5. Edit `playbook.yaml`
`playbook.yaml`:
```yaml
...
- name: Configure PCs
hosts: PCs
roles:
- web-browser
```
Let's test playbook:
```sh
ansibleExample ➤ ansible-playbook -i inventory.ini playbook.yaml git:master*
PLAY [Dump config from router] **************************************************************
TASK [Gathering Facts] **********************************************************************
ok: [192.168.0.13]
TASK [Dump config] **************************************************************************
ok: [192.168.0.13]
PLAY [Configure PCs] ************************************************************************
TASK [Gathering Facts] **********************************************************************
ok: [192.168.0.11]
ok: [192.168.0.5]
PLAY RECAP **********************************************************************************
192.168.0.11 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
192.168.0.13 : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
192.168.0.5 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
```
Everything fine.
6. Edit role's tasks - `roles/web-browser/tasks/main.yml`:
```yaml
---
- name: Install browser
ansible.builtin.package:
name: firefox-esr
state: present # also we can use 'latest' to also update package
```
Let's test it:
```sh
ansibleExample ➤ ansible-playbook -i inventory.ini playbook.yaml git:master*
PLAY [Dump config from router] **************************************************************
TASK [Gathering Facts] **********************************************************************
ok: [192.168.0.13]
TASK [Dump config] **************************************************************************
ok: [192.168.0.13]
PLAY [Configure PCs] ************************************************************************
TASK [Gathering Facts] **********************************************************************
ok: [192.168.0.11]
ok: [192.168.0.5]
TASK [web-browser : Check if browser is installed] ******************************************
ok: [192.168.0.11]
fatal: [192.168.0.5]: FAILED! => {"cache_update_time": 1717928756, "cache_updated": false, "changed": false, "msg": "'/usr/bin/apt-get -y -o \"Dpkg::Options::=--force-confdef\" -o \"Dpkg::Options::=--force-confold\" install 'firefox-esr=115.11.0esr-1~deb12u1'' failed: E: Could not open lock file /var/lib/dpkg/lock-frontend - open (13: Permission denied)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), are you root?\n", "rc": 100, "stderr": "E: Could not open lock file /var/lib/dpkg/lock-frontend - open (13: Permission denied)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), are you root?\n", "stderr_lines": ["E: Could not open lock file /var/lib/dpkg/lock-frontend - open (13: Permission denied)", "E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), are you root?"], "stdout": "", "stdout_lines": []}
PLAY RECAP **********************************************************************************
192.168.0.11 : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
192.168.0.13 : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
192.168.0.5 : ok=1 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
```
We failed, because our user doesn't have root permissions. But why `192.168.0.11` didn't failed? - Because it already have installed firefox.
So how to fix permissions? We have 3 choices:
- Make manager user with root privileges and give his creds to ansible
- Give sudo priveleges to connecting user
- Give to ansible root password to use in `su` (Note: you should use [encrypted variables](https://docs.ansible.com/ansible/latest/vault_guide/vault_using_encrypted_content.html#playbooks-vault))
Recommended to make manager user with strong password and use encrypted password (1+3). But for sake of time I will just give ansible my password. And since on debian there is no `sudo` (what is default method of privelege escalation in Ansible), I will tell ansible to use `su`.
`inventory.ini`:
```yaml
...
[PCs:vars]
ansible_become=true
ansible_become_password=rootPassword
ansible_become_method=su
```
Try again:
```sh
TASK [web-browser : Gather the package facts] **********************************
chaanged: [192.168.0.5]
```
7. Next we will write browser version in terminal during run of Ansible
`roles/web-browser/tasks/main.yml`:
```yaml
...
- name: Gather the package facts
ansible.builtin.package_facts:
manager: auto
- name: Write browser version in terminal during run of Ansible
ansible.builtin.debug:
msg: "Firefox {{ ansible_facts.packages['firefox-esr'][0].version }} is installed!"
when: "'firefox-esr' in ansible_facts.packages"
```
Let's test it:
```sh
TASK [web-browser : Write browser version in terminal during run of Ansible] ***************
ok: [192.168.0.11] => {
"msg": "Firefox 115.11.0esr-1~deb11u1 is installed!"
}
ok: [192.168.0.5] => {
"msg": "Firefox 115.11.0esr-1~deb12u1 is installed!"
}
```
8. Next install Caddy and upload its config
`roles/web-browser/tasks/main.yml`:
```yaml
- name: Install Caddy
ansible.builtin.package:
name: caddy
state: latest
- name: Create Caddyfile
ansible.builtin.copy:
src: Caddyfile
dest: /etc/caddy/Caddyfile
owner: caddy
group: caddy
mode: '0644'
```
`roles/web-browser/files/Caddyfile`:
```caddy
:80 {
root * /var/www/html
file_server
}
```
Now we can access default Caddy page of our PCs via browser...
OR NOT
```sh
TASK [web-browser : Install Caddy] *********************************************
fatal: [192.168.0.11]: FAILED! => {"changed": false, "msg": "No package matching 'caddy' is available"}
ok: [192.168.0.5]
```
Ansible use package manager of remote host. So if system doesn't have package, it will not be installed (+1 to nixOS and its package manager)
9. Now we upload index file with facts from system (hostname and environment variables (insecure, lol))
`roles/web-browser/tasks/main.yml`:
```yaml
...
- name: Create web directory
ansible.builtin.file:
path: /var/www/html
state: directory
- name: write index webpage using jinja2 template
ansible.builtin.template:
src: index.html.j2
dest: /var/www/html/index.html
owner: caddy
group: caddy
mode: '0644'
```
`roles/web-browser/template/index.html.j2`:
```html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
<title>{{ ansible_facts['hostname'] }}</title>
<link rel="stylesheet" href="./style.css">
<link rel="icon" href="./favicon.ico" type="image/x-icon">
</head>
<body>
<main>
<h1>Welcome to {{ ansible_facts['hostname'] }}</h1> <!-- We can use other variables from facts that ansible gathered, you can check what it have with command - ansible 192.168.0.5 -m ansible.builtin.gather_facts -i inventory.ini --tree ./tmp/facts -->
</main>
<!-- We make jinja2 loop to iterate over environment variables -->
{% for key, value in ansible_env.items() %}
<p>variable {{ key }} is {{ value }}</p>
{% endfor %}
</body>
</html>
```
Result in `index.html`:
```html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
<title>debian12</title>
<link rel="stylesheet" href="./style.css">
<link rel="icon" href="./favicon.ico" type="image/x-icon">
</head>
<body>
<main>
<h1>Welcome to debian12</h1>
<!-- We can use other variables from facts that ansible gathered, you can check what it have with command - ansible 192.168.0.5 -m ansible.builtin.gather_facts -i inventory.ini --tree ./tmp/facts -->
</main>
<!-- We make jinja2 loop to iterate over environment variables -->
<p>variable MAIL is /var/mail/root</p>
<p>variable LANGUAGE is en_US:en</p>
<p>variable USER is casual</p>
<p>variable SSH_CLIENT is 192.168.0.56 40480 22</p>
<p>variable XDG_SESSION_TYPE is tty</p>
<p>variable SHLVL is 0</p>
<p>variable MOTD_SHOWN is pam</p>
<p>variable HOME is /root</p>
<p>variable SSH_TTY is /dev/pts/0</p>
<p>variable DBUS_SESSION_BUS_ADDRESS is unix:path=/run/user/1000/bus</p>
<p>variable LOGNAME is casual</p>
<p>variable _ is /bin/sh</p>
<p>variable XDG_SESSION_CLASS is user</p>
<p>variable TERM is xterm-256color</p>
<p>variable XDG_SESSION_ID is 49</p>
<p>variable PATH is /usr/local/bin:/usr/bin:/bin:/usr/games</p>
<p>variable XDG_RUNTIME_DIR is /run/user/1000</p>
<p>variable LANG is en_US.UTF-8</p>
<p>variable SHELL is /bin/bash</p>
<p>variable PWD is /home/casual</p>
<p>variable SSH_CONNECTION is 192.168.0.56 40480 192.168.0.5 22</p>
</body>
</html>
```
10. And last - make caddy restart if ansible makes any modification
`roles/web-browser/handlers/main.yml`:
```yaml
- name: restart caddy service
ansible.builtin.service:
name: caddy
state: restarted
```
and then we can add `notify` to tasks so they will restart caddy:
Let's edit index.html a bit and run entire playbook:
`roles/web-browser/tasks/main.yml`:
```yaml
...
- name: Create Caddyfile
ansible.builtin.copy:
src: Caddyfile
dest: /etc/caddy/Caddyfile
owner: caddy
group: caddy
mode: '0644'
notify: restart caddy service
...
- name: write index webpage using jinja2
ansible.builtin.template:
src: index.html.j2
dest: /var/www/html/index.html
owner: caddy
group: caddy
mode: '0644'
notify: restart caddy service
```
What will happen:
```sh
...
TASK [web-browser : write index webpage using jinja2] **************************
changed: [192.168.0.5]
RUNNING HANDLER [web-browser : restart Caddy service] **************************
changed: [192.168.0.5]
```
That's it.
Full example - https://git.sual.in/casual/AnsibleBlogExample
{{< source >}}
https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_reuse_roles.html
https://www.golinuxcloud.com/ansible-roles-directory-structure-tutorial/
https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_handlers.html
https://docs.ansible.com/ansible/latest/network/getting_started/basic_concepts.html
https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_privilege_escalation.html#passwords-for-enable-mode
more ansible docs
{{< /source >}}
Reference in New Issue View Git Blame Copy Permalink