I’ve tried to cobble something together that’s quick to set up and works good enough (aka: not 100% identical to opsworks, but close enough)

To mimic an opsworks environment in vagrant, we need several tools:

• vagrant
• test kitchen
• berkshelf
• chef

Below we describe one way to create and test a new cookbook. There is another way (by using the opsworks provisioner) which we’ll use later. For now, the standard chef_zero provisioner works well enough.
The next setup allows us to test one single cookbook. If you need to test several cookbooks at the same time, things have to change slightly. More about this later.

Creating a project

mkdir [project name]
cd !$
mkdir cookbooks 
berks cookbook [cookbook name]  ## create the skeleton for a new cookbook

Modify the .kitchen.yml file:

---
driver:
  name: vagrant
  require_chef_omnibus: 11.10.4  ## opsworks still uses this version of chef
provisioner:
  name: chef_zero  ## chef_solo is deprecated
  environments_path: ./environments  ## this folder contains the json file that mimics the opsworks environment data
platforms:
  - name: centos-6.7  ## most similar to the current amazon linux
suites:
  - name: testsuite
    run_list:
      - recipe[cookbook name::recipe]
    attributes:  ## standard way to provide attributes to cookbooks, but it's better to use the environments functionality, that way we can just copy and paste the json data from opsworks to vagrant
    tester:
      test: "hello"
   provisioner:  ## Opsworks style attributes
     client_rb:
       environment: test ## the name of the actual file containing our json data

To obtain the opsworks attributes, create an opsworks stack + host, log in to that host, and run:

sudo opsworks-agent-cli get_json

This command will provide you the full list of attributes. It’s better to keep only the attributes we need and transfer those over to our environments file. However, the cool thing about opsworks is all hosts can access all data about the stack from these attributes, so it’s worth checking them out in detail to see what’s available.
The example below contains the bare minumum to get things working.

{
  "default_attributes": {
    "opsworks" : {
      "stack" : {
        "name" : "MyStack",
        "id" : "42dfd151-6766-4f1c-9940-ba79e5220b58"
      }
    }
  },
  ## custom attributes have to be added here and can be accessed with node['custom attribute']
  ## in opsworks you can just specify, in the custom json field, { 'custom attribute': 'value' }
  "chef_type" : "environment",
  "json_class" : "Chef::Environment"
}

Next, if our module has dependencies to other modules, we need to set up our berksfile.

source "https://supermarket.chef.io"
 
metadata

cookbook 'supermarket_cookbook'

You also need to add the dependencies in the metadata.rb file

name             'cookbook_name'
maintainer       'YOUR_COMPANY_NAME'
maintainer_email 'YOUR_EMAIL'
license          'All rights reserved'
description      'Installs/Configures my cookbook'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version          '0.1.0'
 
depends 'supermarket_cookbook'

Test

We didn’t actually specify any real tests in this example (check out the test kitchen manual on how to set this up), but we can provision our test machine with the following command:

kitchen converge

Once the machine has converged (or not), we can log into the instance to do some manual debugging with:

kitchen login

Just as a sidenote: if you want to debug on opsworks, all interesting files are under /opt/aws/opsworks. Even better: you can modifie your cookbooks directly under /opt/aws/opsworks/current/site-cookbooks and run the opsworks-agent-cli with the correct options to rerun chef with your modified cookbooks.

Testing multiple cookbooks at once

There is an important difference between how berksfiles are interpreted by opsworks. You need a different Berksfile and metadata per environment
As a reminder, this is our directory layout:

.
└── project
    ├── Berksfile   # testkitchen
    ├── .kitchen.yml   #  testkitchen
    └── cookbooks
        ├── Berksfile   # opsworks
        ├── cookbook-a
        │   ├── Berksfile.in   # testkitchen
        │   └── metadata.rb   # opsworks + testkitchen
        └── cookbook-b
            ├── Berksfile.in   # testkitchen
            └── metadata.rb   # opsworks + testkitchen

One special remark: Opsworks needs a git repo with only the Berksfile and the two cookbook directories, so take care about how you commit to git!
The main Berksfile for testkitchen is the one under /project/. This calls the Berksfile.in in every cookbook.
I found this code somewhere on the internet, but can’t find the link anymore, so my apologies about not giving credit where credit is due!

source "https://supermarket.chef.io"
 
# Note the absence of the metadata line.
 
def dependencies(path)
  berks = "#{path}/Berksfile.in"
  instance_eval(File.read(berks)) if File.exists?(berks)
end
Dir.glob('./cookbooks/*').each do |path|
  dependencies path
  cookbook File.basename(path), :path => path
end

The sub Berksfiles only contain the needed cookbooks for that specific cookbook. The metadata.rb file is automatically sourced by the cookbook

cookbook "dependency-a"

cookbook "dependency-b"

Opsworks needs the following file. Again no metadata is referenced as opsworks will find the necessary files automatically.

source "https://supermarket.chef.io"
 
cookbook "dependency-a"
cookbook "dependency-b"

This is the ‘normal’ Berksfile for cookbook-a. You still need this if you want to test it independently!

source "https://supermarket.chef.io"
 
metadata
 
cookbook "dependency-a"

References

http://pixelcog.com/blog/2014/virtualizing-aws-opsworks-with-vagrant/ (this uses the opsworks provisioner, didn’t get it working yet)
http://enriquecordero.com/programming/opsworks-chef-workflow/
http://docs.aws.amazon.com/opsworks/latest/userguide/opsworks-opsworks-mock.html (outdated)
https://www.youtube.com/watch?v=0sPuAb6nB2o (very interesting test kitchen / berkshelf tutorial)

The post Test kitchen opsworks workflow appeared first on Cloudar.

• stop the instance
• unmap the volume
• create a snapshot of the volume
• create a bigger volume of the snapshot
• mount the new volume on another stopped instance (in the same AZ! Otherwise we can’t map the volume)
• start that instance
• delete and recreate the partition table to fit to the size of the disk + resize the filesystem

(the two above steps are needed when using parted. Fdisk can handle modifying the root volume in situ, but i couldn’t get it to play nice through scripting

• stop the instance
• move the volume to the original instance
• start that instance

pfew! To avoid doing this manually and getting bored to death I wrote a small script to handle this for me. It’s not the cleanest and most flexible script in the world (end could be improved no end), but it gets the job done.

#!/bin/bash
#
# ATTENTION: this only works when root login is allowed on your intermediate host. Preferably use centos 6
#
# This script uses an intermediate host to resize the disk + you need an ssh tunnel to that machine through a nat host. 
# This way we can also reach machines in a private subnet
# ex: ssh -i  -L 2222::22 
#
# Fill in the following variables
# and DON'T forget to make sure grub boots the correct disk by changing root=/dev/xvda1 in /boot/grub/menu.lst on the intermediate host. Booting the wrong disk is caused
# by the fact that by default it mentions the UUID of the volume, and aws uses the same image for every instance of a specific ami.
#
# The script is called with two parameters:
# - the instance_id you want to resize
# - the new disk size in GB

# temporary host to use
temp_instance=
# temporary device name as called in the web gui
temp_device_aws=/dev/sdf # doesn't really matter, but this one works. I've had issues with higher letters
# temporary device name as known by the OS
temp_device_host=/dev/xvdf # has to be in accordance with the $temp_device
# identity file to log into temphost
pemfile=
#username to log into temp instance
temp_username=root # sorry, won't work otherwise. You could adapt all commands to use sudo, but some commands don't work with it
# aws profile to use
profile=
# local port for ssh tunnel
tunnel_port=2222 # use the port you opened up for the ssh tunnel

source_instance=$1
new_size=$2
[[ $1 == '' ]] || [[ $2 == '' ]] &# for ease of use, let's put all our actions into functions

function get_instance_name {
  instance_id=$1
  aws --profile $profile ec2 describe-instances --query 'Reservations[*].Instances[*].Tags[?Key==`Name`].Value' --output text --instance-ids $instance_id
}

function get_ip {
  instance_id=$1
  aws --profile $profile ec2 describe-instances --query 'Reservations[0].Instances[0].PrivateIpAddress' --output text --instance-ids $instance_id
}

function get_bootvol_id {
  instance_id=$1
  aws --profile $profile ec2 describe-instances --query 'Reservations[0].Instances[0].BlockDeviceMappings[0].Ebs.VolumeId' --output text --instance-ids $instance_id
}

function get_az {
  instance_id=$1
  aws --profile $profile ec2 describe-instances --query 'Reservations[0].Instances[0].Placement.AvailabilityZone' --output text --instance-ids $instance_id
}

function check_stopped {
  instance_id=$1
  aws --profile $profile ec2 wait instance-stopped --instance-ids $instance_id
}
# the difference between this and the previous function is subtle, but needed.
function is_stopped {
  instance_id=$1
  aws --profile $profile ec2 describe-instances --query 'Reservations[*].Instances[*].State.Name' --instance-ids $instance_id --output text | grep -q 'stopped'
}

function wait_snapshot {
  snapshot_id=$1
  aws --profile $profile ec2 wait snapshot-completed --snapshot-id $snapshot_id
}

function wait_volume {
  volume_id=$1
  aws --profile $profile ec2 wait volume-available --volume-id $volume_id
}

function detach_volume {
  volume_id=$1
  aws --profile $profile ec2 detach-volume --volume-id $volume_id > /dev/null
}

function create_snapshot {
  volume_id=$1
  aws --profile $profile ec2 create-snapshot --volume-id $volume_id --description "${volume_id}-snap"  | sed 's/"//g' | grep -oe 'SnapshotId: [a-z0-9-]*' | cut -d' ' -f 2
}

function create_volume_from_snap {
  snapshot_id=$1
  az=$2
  size=$3
  aws --profile $profile ec2 create-volume --availability-zone $az --snapshot-id $snapshot_id --size $size --volume-type gp2 | sed 's/"//g' | grep -oe 'VolumeId: [a-z0-9-]*' | cut -d' ' -f 2
}

function attach_volume {
  volume_id=$1
  instance_id=$2
  device=$3
  aws --profile $profile ec2 attach-volume --volume-id $volume_id --instance-id $instance_id --device $device > /dev/null
}

function do_resize {
  temp_instance_ip=$1
  temp_file=$( echo "/tmp/$(date +%s).sh" )

  # NAT or direct version
  #ssh_cmd="ssh -o StrictHostKeyChecking=no -i $pemfile $temp_username@$temp_instance_ip "
  ssh_cmd="ssh -o StrictHostKeyChecking=no -i $pemfile $temp_username@localhost -p $tunnel_port "
  $ssh_cmd yum -y install parted 
  partition_start=$($ssh_cmd parted $temp_device_host print | grep -e '^ 1' | tr -s ' '  | cut -d' ' -f3  )
  $ssh_cmd parted -s $temp_device_host rm 1
  $ssh_cmd parted -s $temp_device_host mkpart primary $partition_start 100%
  $ssh_cmd parted -s $temp_device_host set 1 boot on
  # for some reason, this command sometimes doesn't work and you still need to log into the instance when the script has finished and manually resize the filesystem
  $ssh_cmd " e2fsck -p -y -f $( echo ${temp_device_host}1 ) && resize2fs $( echo ${temp_device_host}1 ) "
}

function stop_instance {
  instance_id=$1
  aws --profile $profile ec2 stop-instances --instance-ids $instance_id > /dev/null
}
function start_instance {
  instance_id=$1
  aws --profile $profile ec2 start-instances --instance-ids $instance_id > /dev/null
}

function wait_for_ssh {
  host=$1
  while ! $( nc -w 1 localhost $tunnel_port | grep -q OpenSSH ); do   
  # NAT instance
  #while ! nc -z $host 22 ; do   
    echo - n "."
    sleep 1
  done
}

source_instance_name=$(get_instance_name $source_instance)
echo "Starting resize"
echo "Resizing disk of host $source_instance_name"

echo "Getting info & testing connections"
temp_instance_name=$(get_instance_name $temp_instance) 
source_az=$( get_az $source_instance )
temp_source_az=$( get_az $temp_instance)
source_ip=$( get_ip $source_instance )
temp_ip=$( get_ip $temp_instance )
source_vol_id=$( get_bootvol_id $source_instance )

aws ec2 describe-instances --profile $profile --instance-id $source_instance &> /dev/null || { echo "Source instance not found"; exit 1; }
aws ec2 describe-instances --profile $profile --instance-id $temp_instance &> /dev/null || { echo "Temporary instance not found"; exit 1; }
nc -z localhost $tunnel_port || { echo "Tunnel on port $tunnel_port is closed"; exit 1; }
[[ $source_az == $temp_source_az ]] || { echo "$temp_instance_name is not in the same AZ as $source_instance_name"; exit 1; }
is_stopped $temp_instance || ssh -o StrictHostKeyChecking=no -i $pemfile $temp_username@localhost -p $tunnel_port grep -q 'root=/dev/xvda1' /boot/grub/menu.lst || { echo "Grub is incorrectly configured on $temp_instance_name"; exit 1; }

echo "Stopping instances"
is_stopped $source_instance || stop_instance $source_instance 
is_stopped $temp_instance || stop_instance $temp_instance 
check_stopped $source_instance
check_stopped $temp_instance

echo "Removing disk from $source_instance_name"
detach_volume $source_vol_id

echo "Creating snapshot"
snap_id=$( create_snapshot $source_vol_id )
wait_snapshot $snap_id

echo "Creating volume from snapshot )"
new_vol_id=$( create_volume_from_snap $snap_id $source_az $new_size )
wait_volume $new_vol_id

echo "Attaching volume to intermediate host $temp_instance_name"
attach_volume $new_vol_id $temp_instance $temp_device_aws

echo "Booting intermediate host $temp_instance_name"
start_instance $temp_instance
wait_for_ssh $temp_ip
echo "Resizing disk"
do_resize $temp_ip

echo "Attaching resized volume to $source_instance_name"
stop_instance $temp_instance
check_stopped $temp_instance
detach_volume $new_vol_id
sleep 5
attach_volume $new_vol_id $source_instance /dev/sda1
sleep 5

echo "Starting host $source_instance_name"
start_instance $source_instance
echo "Done"

The post Extend ec2 boot disk appeared first on Cloudar.

• create an aws instance
• install tomcat on it
• deploy a .war file from S3 and edit its contents

While this is a very simple exercise, it does touch on some interesting topics about ansible and aws.
To start, we need something like this:

• an S3 bucket to store our .war files
• a nat host so ansible tower can manage servers from our vpc in the remote vpc
• (and in our case an S3 read-only IAM policy as our version of ansible doesn’t support policy creation yet)

Tower config

To allow our tower to connect through the nat host we need some extra config.
In the root of our project we need an ansible.cfg file containing:

[ssh_connection]
ssh_args = -o ControlMaster=auto -o ControlPersist=16m -F /opt/ansible/ssh/

This allows ansible to use the ssh configuration stored on the tower server. This is what it looks like:

Host [nat instance public ip]
  User [nat instance user]
  IdentityFile [nat instance_pem file]
Host [sandbox private subnet].*
  User [sandbox user]
  StrictHostKeyChecking no
  IdentityFile [sandbox instance pem file]
  ProxyCommand ssh -i [nat instance pem file] -o StrictHostKeyChecking=no [nat_instance_user]@[nat_instance_public_ip] nc %h %p

I won’t detail the tower configuration of the project/inventory/job/… here in detail as it’s quite straightforward.

Ansible playbook

So, we need two playbooks. The first one sets up a node and installs tomcat.

## site.yml
---
- name: Setup EC2
  hosts: tower-via-local
  gather_facts: no
  roles:
    - infrastructure
- name: install tomcat
  remote_user: ec2-user
  sudo: yes
  hosts: launched_servers
  roles:
    - tomcat

The second one deploys the war and edits a file.

### deploy.yml
---
- name: deploy app
  remote_user: ec2-user
  sudo: yes
  hosts: tag_Name_tomcat
  roles:
    - deploy_app

We need to split these up as both playbooks need a different inventory to work with:

• provision & install: ‘tower-via-local’ which is located in our VPC
• configuration: here we get the sandbox vpc ec2 instances and select a specific tagname we gave during provisioning

In theory, you could work with a callback to tower for the second part, but then you need to set up a way for the client to reach the server through the nat instance. As we don’t need this functionality right now, we’re skipping this step.

One of the nice things about tower is you can create simple forms provide variables to our playbooks. In this case I made two forms to provide the number of instances and the name of the war file.

# roles/infrastructure/tasks/main.yml
---
- name: Create the tomcat security group
  ec2_group:
    description: "Open ssh and tomcat ports"
    name: "tomcat-sg"
    region: "{{ region }}"
    rules:
      # only allow ssh access from the nat instance
      - proto: tcp
        from_port: 22
        to_port: 22
        cidr_ip: "{{ nat_ip }}/32"
      # open tomcat to the world
      - proto: tcp
        from_port: 8080
        to_port: 8080
        cidr_ip: 0.0.0.0/0
    purge_rules: yes
    state: present
    vpc_id: "{{ vpc_id }}"
  register: tomcat_sg 

# NOTE: only from ansible > 2.0
# ansible can't create a role from an existing policy, so we're obligated to upload a new one (stored in roles/infrastructure/files)
#- name: Create S3 read-only access
#  iam_policy:
#    iam_name: s3-tomcat
#    iam_type: role
#    policy_document: s3-ro.json
#    policy_name: s3-ro
#    state: present

- name: Launch base server
  ec2:
    assign_public_ip: yes
    group_id: "{{ tomcat_sg.group_id }}"
    image: "{{ tomcat_ami }}"
    instance_type: "{{ tomcat_instance_type }}"
    instance_profile_name: s3-tomcat
    exact_count: "{{ tomcat_instance_count }}"
    count_tag: { "Name": "tomcat" }
    key_name: "{{ tomcat_kp }}"
    region: "{{ region }}"
    vpc_subnet_id: "{{ poc_subnet }}"
    wait: no
    assign_public_ip: yes
    instance_tags: {
      "Name": "tomcat",
    }
  register: base_server 

# because wait_for doesn't use the ssh config we have to delegate this taks to the nat host. Otherwise tower wil try to connect to port 22 directly
- name: Check if we can ssh to the instance
  wait_for:
    host: "{{ item.private_ip }}"
    port: 22
    state: started
  with_items: base_server.instances
  when: item.state != "terminated"
  delegate_to: "{{ nat_ip }}"

# add all servers to a temporary group we can use to install tomcat. We need this group as in the current playbook we have 'tower-via-local' as inventory
- name: Add servers to temporary group
  add_host:
    hostname: "{{ item.private_ip }}"
    groupname: launched_servers
  with_items: base_server.instances
  when: item.state != "terminated"

The next role just installs tomcat

---
- name: Install basic software
  yum:
    name: "{{ item }}"
    state: present
  with_items:
    - java-1.8.0-openjdk
    - tomcat8

Ok, so now we have some servers ready to go. The next playbook uses a different inventory

# roles/deploy_app/tasks/main.yml
---
# our war file is 
- name: Deploy war file
  s3: 
    bucket: "{{ war_bucket }}"
    object: "{{ war_file }}" 
    dest: "{{ war_deploy_path }}/{{ war_file }}"
    mode: get 
    overwrite: no
  register: war_downloaded

- name: Set correct permissions
  file: 
    path: "{{ war_deploy_path }}/{{ war_file }}"
    owner: tomcat
    group: tomcat
  when: war_downloaded.changed
  register: war_deployed

- name: Restart tomcat
  service:
    name: tomcat8
    state: restarted
  when: war_deployed.changed

# here we cheat a little. The sample.war I'm deploying contains an index.html that we want to edit. We just wait untill the war is unpacked and the file is available
- name: Wait until war is deployed
  wait_for:
    path: "{{ war_deploy_path }}/{{ app_name }}/index.html"

- name: Edit index file
  lineinfile:
    dest: "{{ war_deploy_path }}/{{ app_name }}/index.html"
    regexp: '^<title>Sample "Hello, World" Application</title>'
    line: '<title>Sample "Hello, from cloudar" Application</title>'   
  lineinfile:
    dest: "{{ war_deploy_path }}/{{ app_name }}/index.html"
    regexp: '^<h1>Sample "Hello, World" Application</h1>'
    line: '<h1>Sample "Hello, from cloudar" Application</h1>'   
  when: war_deployed.changed

This concludes our little exercise! Feel free to leave comments, I’m sure there is still room for improvement.

The post Ansible tomcat installation + .war deploy appeared first on Cloudar.