In web applications we usually have a few things that are sensitive, e.g. the login to the production database or API keys used to access a third party API. We need to be particularly careful about how we manage these secrets, as they may allow attackers to access data without going through the application itself.

There are trade-offs in managing secrets, depending on the size of the organization.

For a small team of developers who are also the admins, then we implicitly trust our devs. We may store the secrets on our dev machine and push them from there to the app servers. It's better not to have secrets in the build environment, though, particularly if it's a third party CI service.

We need to keep the secrets separate from the build, loading them separately on the target system. That might mean putting them in a separate file or reading them at runtime from an external source like an S3 bucket or AWS Parameter Store. Access is controlled by IAM instance roles.

For secure applications like health care or finance, we need to tightly control access to production systems. We can restrict access to your ops team. Ideally nobody would log into production systems, and if they do, there is an audit log.

Ansible vault

The Ansible automation tool a vault function which we can use to store keys. It automates the process of encrypting variable data so we can check it into source control, and only people with the password can read it. It's great for simple deployments with small teams. It has very few moving parts and dependencies, while being reasonably secure.

The following shows describes how you can use the vault.

Configuring the vault key

First, generate a vault key and put it in the file vault.key:

openssl rand -hex 16

You can specify the password when you are running a playbook with the --vault-password-file vault.key option, or you can make the vault password always available by setting it in ansible.cfg:

vault_password_file = vault.key

Defining secrets

There are two ways to store secrets in Ansible variable files. Either we can encrypt the file as a whole, or we can embed encrypted data inline.

To create an encrypted variable file:

ansible-vault create --vault-id=vault.key inventory/group_vars/web_servers/secrets.yml

Add variables normally. When you save the file, it will be encrypted. Later, edit the file like this:

ansible-vault edit --vault-id=vault.key inventory/group_vars/web_servers/secrets.yml

To encrypt a single variable inline:

openssl rand -hex 32 | ansible-vault encrypt_string --vault-id=vault.key --stdin-name 'db_pass'

That generates encrypted data like:

db_pass: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          64346139623638623838396261373265666363643264333664633965306465313864653033643530
          3830366538366139353931323662373734353064303034660a326232343036646339623638346236
          39623832656466356338373264623331363736636262393838323135663962633339303634353763
          3935623562343131370a383439346166323832353232373933613363383435333037343231393830
          35326662353662316339633732323335653332346465383030633333333638323735383666303264
          35663335623061366536363134303061323861356331373334653363383961396330386136636661
          63373230643163633465303933396336393531633035616335653234376666663935353838356135
          36323866346139666462

Copy that into a standard variable file.

Generating templates

Now, you can use Ansible's template function to create a config file template. The template file includes the variables, and Ansible will automatically decrypt vault variables and insert them into the template.

For example, here is a template which configures an Elixir Phoenix app:

[{{ elixir_app_name }}."{{ elixir_app_module }}Web.Endpoint"]
secret_key_base = "{{ secret_key_base }}"

[{{ elixir_app_name }}."{{ elixir_app_module }}.Repo"]
username = "{{ db_user }}"
password = "{{ db_pass }}"
database = "{{ db_name }}"
hostname = "{{ db_host }}"
ssl = {{ db_ssl }}
pool_size = {{ db_pool_size }}

The Ansible task could generate it on the production server:

- name: Create config.toml
  template: src=etc/app/config.toml.j2 dest=/etc/app/config.toml owner={{ app_user }} group={{ app_group }} mode=0644

Generating config files to S3

When deploying in the cloud, you can generate the app config file to an S3 bucket. When the app starts up, it can then sync the config file from the S3 bucket.

---
# Generate app config and upload to S3 bucket
#
# ansible-playbook -v -u $USER --extra-vars "env=$ENV" playbooks/$APP/config-app.yml -D

- name: Generate config file from template and upload to S3
  hosts: localhost
  gather_facts: no
  connection: local
  vars:
    app_name: foo
    comp: app
    file_format: toml
    input_template: ../../templates/{{ app_name }}/{{ comp }}/config.{{ file_format }}.j2
    output_file: config.{{ file_format }}
  vars_files:
    - ../../vars/{{ app_name }}/{{ env }}/common.yml
    - ../../vars/{{ app_name }}/{{ env }}/db-app.yml
    - ../../vars/{{ app_name }}/{{ env }}/app.yml
    - ../../vars/{{ app_name }}/{{ env }}/app-secrets.yml
  tasks:
    - name: Create tempfile
      tempfile:
        state: file
      register: temp_file

    # - debug: var=temp_file.path

    - name: Fill template to tempfile
      template:
        src: "{{ input_template }}"
        dest: "{{ temp_file.path }}"
      no_log: true

    - name: Put config to S3
      aws_s3:
        bucket: "{{ config_bucket }}"
        object: "{{ config_bucket_prefix }}/{{ output_file }}"
        src: "{{ temp_file.path }}"
        mode: put

    - name: Delete tempfile
      file:
        state: absent
        path: "{{ temp_file.path }}"