server/ansible/README.md

Ansible

Bootstraps, syncs, and starts/stops the Docker Compose stacks in ../services/ on the three servers OpenTofu provisions (see ../infra/). OpenTofu still owns the actual cloud resources (servers, network, firewalls, DNS); everything downstream of "the server exists" - installing Docker, creating the deploy user, hardening SSH, copying services/<host>/, rendering .env / the Gitea runner compose file, and running spinup.sh/spindown.sh - lives here instead. Persistent service data lives in named Docker volumes backed up to S3, not on a Hetzner volume - see ../readme.md.

Contents

• inventory/hcloud.yml - dynamic inventory, queries the Hetzner Cloud API directly (not Terraform state) and groups servers by their role label (role_dev, role_prod, role_vpn - set in infra/modules/<host>/main.tf).
• group_vars/ - all.yml (shared: deploy_user, SSH key path, backup_s3_* S3 backup credentials), role_dev.yml / role_prod.yml / role_vpn.yml (per-role vars, e.g. dev_runner_count).
• roles/bootstrap/ - Docker install, deploy user creation, SSH hardening, unattended-upgrades. Replaces infra/scripts/bootstrap.sh.tftpl.
• roles/deploy/ - copies services/<host>/ to the server and renders .env (S3 backup credentials, see ../readme.md) and, for dev, Runners/docker-compose.yml.
• playbooks/ - bootstrap.yml, deploy.yml, spinup.yml, spindown.yml, and site.yml (all three in order).

Prerequisites

cd ansible
ansible-galaxy collection install -r requirements.yml
export HCLOUD_TOKEN=your-hetzner-api-token   # never commit this
export BACKUP_S3_BUCKET=your-bucket-name
export BACKUP_S3_ACCESS_KEY_ID=your-access-key-id
export BACKUP_S3_SECRET_ACCESS_KEY=your-secret-access-key
export BACKUP_S3_ENDPOINT=your-s3-endpoint   # omit for real AWS S3

ansible_ssh_private_key_file (see group_vars/all.yml) defaults to ~/.ssh/id_ed25519 - override with ANSIBLE_SSH_PRIVATE_KEY_FILE if you use a different key. It must match one of the SSH keys OpenTofu installed on the servers (var.ssh_key_names).

Usage

Same chicken-and-egg as OpenTofu's own first-apply ordering (see ../readme.md): dev/prod's firewalls only accept SSH from vpn's own public IP, so you must bring vpn up and connect to it before dev/prod are reachable at all.

# 1. vpn first - its firewall accepts SSH from var.allowed_ssh_source_ips directly
ansible-playbook playbooks/bootstrap.yml -l role_vpn
ansible-playbook playbooks/deploy.yml    -l role_vpn
ansible-playbook playbooks/spinup.yml    -l role_vpn

# 2. connect to the VPN you just started with an OpenVPN client - now dev/prod
#    are reachable, since your egress IP matches their firewall rule
ansible-playbook playbooks/bootstrap.yml -l role_dev,role_prod
ansible-playbook playbooks/deploy.yml    -l role_dev,role_prod
ansible-playbook playbooks/spinup.yml    -l role_dev,role_prod

# ...or once already bootstrapped and connected, do everything in one go:
ansible-playbook playbooks/site.yml -l role_dev,role_prod

Re-run deploy.yml + spinup.yml any time a compose file changes or dev_runner_count in group_vars/role_dev.yml is bumped - spinup.sh only starts/updates what's changed, same as before.

spindown.yml runs spindown.sh, which does a full docker system/volume prune -a on the host - it prompts for a yes confirmation before running, since the dynamic inventory makes it trivial to target multiple hosts at once (-l role_dev,role_prod).

Inspect what the dynamic inventory currently sees:

ansible-inventory --graph
ansible-inventory --list