Magnetizer

This repository is a collection of useful Ansible playbooks and roles.

• Magnetizer
  • Introduction
  • Installation
  • Inventory
    • host_list plugin
  • Playbooks
    • developer
    • vps init
      • Notes on servers providers
        • AWS EC2
        • DigitalOcean
    • enable SSL
    • install vim config
    • install zsh
    • install docker
    • secure ssh
    • authorize ssh key
    • deauthorize ssh key
    • DigitalOcean playbooks
      • create droplet
      • create A record
      • list domain records
      • delete DNS record
    • AWS playbooks
      • create A record
      • delete DNS record

Introduction

In Magnet we need to configure computers to work with, they may be servers or the environment a developer uses. To accomplish this we created a repository with bash scripts and Ansible Playbooks and roles. Since we work with Ubuntu, the installation scripts and playbooks all start with the assumption that this is running in Ubuntu 18.04.

Installation

To start using Ansible commands, the quickstart.sh script installs pip (python 3) and the following python packages:

• ansible
• ansible-lint
• paramiko

To check if Ansible was installed run ansible --version. If this fails, you have to restart your computer

Once ansible is available, the Ansible galaxy roles specified on requirements.yml are installed (git is required to use role versions).

Many of the features Magnetizer has (if not all of them), require an ssh key, so you need to generate one. If you don't have one, the init_ssh_key.sh script generates an SSH key (if the default pub key is not present), then adds it to the authorized keys of the current user. This last part is to run Ansible playbooks from your computer, on your computer, without asking for authorization credentials.

Inventory

An inventory must be defined to select the hosts you want ansible to run against.

At Magnet we have a list of all our servers that can be downloaded as a ssh config file that creates aliases to all of them.

To use an ssh alias to select a host for Ansible, we use an inventory plugin from the Ansible contrib files.

The update_ssh_config.sh script obtains the ssh_config file from Magnet's keygen repository and it merges with ~/.ssh/config_local into ~/.ssh/config. This allows the following syntax when running a playbook:

ansible-playbook -i inventory -l <host> playbooks/<playbook_name.yml>

For example, consider the following host defined at the ssh config file:

Host magnetizer.staging
    hostname magnetizer-stg.magnet.cl
    user magnet

To secure SSH you can run the following playbook:

ansible-playbook -i inventory -l magnetizer.staging playbooks/secure_ssh.yml

If you need to target the root user, instead of doing it manually with:

ansible-playbook -i [email protected], -l [email protected] playbooks/init_vps.yml

you can use inventory_root:

ansible-playbook -i inventory_root -l magnetizer.staging playbooks/init_vps.yml

Warning if you already have a ~/.ssh/config file, update_ssh_config.sh will replace it. To avoid this, first move your ~/.ssh/config to ~/.ssh/config_local

host_list plugin

With the host_list plugin a host can be set as command argument.

Following the host defined on the previous example, the playbook can be run with:

ansible-playbook -i '[email protected],' playbooks/secure_ssh.yml

Playbooks

developer

The playbook is at playbooks/developer.yml.

It includes the following roles considering localhost as target:

• common/main
• zsh
• vim

Since the target is localhost, the playbook can be run without specifying an inventory:

ansible-playbook playbooks/developer.yml

If the user needs a password to run sudo, the following option enables privilege escalation:

ansible-playbook playbooks/developer.yml --ask-become-pass

If the target is running WSL/WSL2 the ntp tag must be skipped, since the system clock is already managed.

ansible-playbook playbooks/developer.yml --skip-tags ntp

vps init

The playbook is at playbooks/vps_init.yml.

It includes the following tasks:

• Add user to set as admin on the target host.
• Add list of ssh public keys to the authorized keys (admins).
• Set default locale (en_US.UTF-8).
• Install recommended packages.
• Setup NTP.
• Set swap partition
• Install zsh, oh-my-zsh and custom plugins.
• Install vim and vim_config.

To run it against a server with passworded sudo, add --ask-become-pass (otherwise it fails with Missing sudo password).

The last post-task defined in the playbook upgrades system packages through apt safe upgrade, it might require a shell restart or a session restart.

Notes on servers providers

AWS EC2

Within the EC2 instance creation an SSH key must be selected and usually is not loaded on the agent running the playbook, there are two alternatives:

1. Add the key selected on aws to the ssh agent: ssh-add <aws-key.pem>
2. Use --private-key with ansible-playbook:

ansible-playbook -i inventory -l magnetizer.ec2 playbooks/vps_init.yml --private-key ~/.ssh/aws-key.pem

DigitalOcean

The latest Ubuntu Server 18.04 droplet available on Digital Ocean requires a system reboot after upgrading all system packages.

enable SSL

The playbook is at playbooks/enable_ssl.yml, it automatically enable HTTPS on the target host through certbot.

The following variables can be set as extra parameters when the playbook is played:

• certbot_domain: Comma-separated list of domains to obtain a certificate. If unset it will use the host specified in the ansible inventory.
• certbot_admin_email: Email for account notifications.

A related playbook is at playbooks/secure_nginx.yml, it hardens nginx through a galaxy role from https://dev-sec.io/.

Example:

ansible-playbook -i inventory -l "project*" playbooks/enable_ssl.yml -e certbot_domain="domain*"

install vim config

The playbook is at playbooks/install_vim_config.yml.

install zsh

The playbook is at playbooks/install_zsh.yml.

It performs the following tasks:

• Install zsh and oh-my-zsh through the ansible galaxy role gantsign.oh-my-zsh.
• Upload and set custom set of plugins.

install docker

To run it in your own computer, you will need to specify the inventory for this one:

ansible-playbook -i localhost, -l localhost -c local --ask-become-pass playbooks/install_docker.yml

The playbook is at playbooks/install_docker.yml. It uses the ansible galaxy role geerlingguy.docker.

It configures docker so you can run it without root, and also installs docker-compose. You will need to reboot for the group change to take effect (or just reconnect if using ssh).

secure ssh

The playbook is at playbooks/secure_ssh.yml.

It hardens SSH through a galaxy role from https://dev-sec.io/. Custom configurations are set as role variables.

authorize ssh key

The playbook is at playbooks/authorize_ssh_key.yml.

It prompts for a Magnet user to obtain its github username from the intranet platform. Then uses https://github.com/<username>.keys as key parameter on the authorized key module of ansible.

deauthorize ssh key

The playbook is at playbooks/deauthorize_ssh_key.yml.

It prompts for a Magnet user to obtain its github username from the intranet platform. Then uses https://github.com/<username>.keys as key parameter on the authorized key module of ansible with absent as state.

The playbook can iterate on all hosts from the inventory or limit it with an expression, for example:

ansible-playbook -i inventory -l "project*" playbooks/deauthorize_ssh_key.yml

update bitbucket ssh host key

Bitbucket is going to rotate their ssh key. This playbook automates the process described in https://bitbucket.org/blog/ssh-host-key-changes.

The playbook is at playbooks/bitbucket_ssh_host_key.yml.

To fix your own computer, run:

ansible-playbook -i localhost, -l localhost -c local playbooks/bitbucket_ssh_host_key.yml

DigitalOcean playbooks

In order to use these playbooks, the environment variable DIGITALOCEAN_ACCESS_TOKEN must be set.

create droplet

The playbook is at playbooks/do_create_droplet.yml, it requires the official command line interface for the DigitalOcean API doctl.

It creates a DigitalOcean droplet and register a DNS record to the obtained IP.

The following variables can be set as extra parameters when the playbook is played:

• hostname: This variable is mandatory (example: "demo.do.magnet.cl").
• do_size: Slug for the droplet size, the default value is s-1vcpu-1gb. The slugs can be listed with doctl compute size list.
• do_region: Slug for the region, the default value is nyc3. The slugs can be listed with doctl compute region list
• do_image: Slug for the droplet image, the default value is ubuntu-20-04-x64. The slugs can be listed with doctl compute image list.
• do_base_domain: Base domain for the DNS record, the default value is do.magnet.cl.

create A record

The playbook is at playbooks/do_create_a_record.yml, it requires the official command line interface for the DigitalOcean API doctl.

It will prompt for:

• domain: The default value is do.magnet.cl.
• hostname: The record, for example demo.
• ip: The IP for the record.

list domain records

The playbook is at playbooks/do_list_domain_records.yml, it requires the official command line interface for the DigitalOcean API doctl.

It will prompt for:

• domain: The default value is do.magnet.cl.

delete DNS record

The playbook is at playbooks/do_delete_dns_record.yml, it requires the official command line interface for the DigitalOcean API doctl.

It will prompt for:

• domain: The default value is do.magnet.cl.
• record id: The record id to be deleted, can be obtained with list domain records.

AWS playbooks

In order to use these playbooks, the AWS access and secret keys must be set through a boto configuration or with the following environment variables:

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY

create A record

The playbook is at playbooks/r53_create_a_record.yml, it requires boto (ideally the route53 module will use boto3 as other modules are already doing it).

It will prompt for:

• zone: The default value is aws.magnet.cl.
• record_name: The record without zone, for example demo.
• ip: The IP for the record.

delete DNS record

The playbook is at playbooks/r53_delete_dns_record.yml, it requires boto (ideally the route53 module will use boto3 as other modules are already doing it).

It will prompt for:

• zone: The default value is aws.magnet.cl.
• record_name: The record without zone, for example demo.
• type: The record type, the default value is A.