Test kitchen opsworks workflow

Posted in AWS Blog
26/09/2015 Wannes Van Causbroeck

As testing new cookbooks directly in opsworks takes a long time, it makes a lot of sense to test them in vagrant first. There is some documentation available on how to do this, but some of it is out of date, broken or unclear.

I’ve tried to cobble something together that’s quick to set up and works good enough (aka: not 100% identical to opsworks, but close enough)

To mimic an opsworks environment in vagrant, we need several tools:

  • vagrant
  • test kitchen
  • berkshelf
  • chef

Below we describe one way to create and test a new cookbook. There is another way (by using the opsworks provisioner) which we’ll use later. For now, the standard chef_zero provisioner works well enough.
The next setup allows us to test one single cookbook. If you need to test several cookbooks at the same time, things have to change slightly. More about this later.

Creating a project

mkdir [project name]
cd !$
mkdir cookbooks 
berks cookbook [cookbook name]  ## create the skeleton for a new cookbook


Modify the .kitchen.yml file:

  name: vagrant
  require_chef_omnibus: 11.10.4  ## opsworks still uses this version of chef
  name: chef_zero  ## chef_solo is deprecated
  environments_path: ./environments  ## this folder contains the json file that mimics the opsworks environment data
  - name: centos-6.7  ## most similar to the current amazon linux
  - name: testsuite
      - recipe[cookbook name::recipe]
    attributes:  ## standard way to provide attributes to cookbooks, but it's better to use the environments functionality, that way we can just copy and paste the json data from opsworks to vagrant
      test: "hello"
   provisioner:  ## Opsworks style attributes
       environment: test ## the name of the actual file containing our json data


To obtain the opsworks attributes, create an opsworks stack + host, log in to that host, and run:

sudo opsworks-agent-cli get_json


This command will provide you the full list of attributes. It’s better to keep only the attributes we need and transfer those over to our environments file. However, the cool thing about opsworks is all hosts can access all data about the stack from these attributes, so it’s worth checking them out in detail to see what’s available.
The example below contains the bare minumum to get things working.

  "default_attributes": {
    "opsworks" : {
      "stack" : {
        "name" : "MyStack",
        "id" : "42dfd151-6766-4f1c-9940-ba79e5220b58"
  ## custom attributes have to be added here and can be accessed with node['custom attribute']
  ## in opsworks you can just specify, in the custom json field, { 'custom attribute': 'value' }
  "chef_type" : "environment",
  "json_class" : "Chef::Environment"


Next, if our module has dependencies to other modules, we need to set up our berksfile.

source "https://supermarket.chef.io"

cookbook 'supermarket_cookbook'


You also need to add the dependencies in the metadata.rb file

name             'cookbook_name'
maintainer       'YOUR_COMPANY_NAME'
maintainer_email 'YOUR_EMAIL'
license          'All rights reserved'
description      'Installs/Configures my cookbook'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version          '0.1.0'
depends 'supermarket_cookbook'


We didn’t actually specify any real tests in this example (check out the test kitchen manual on how to set this up), but we can provision our test machine with the following command:

kitchen converge


Once the machine has converged (or not), we can log into the instance to do some manual debugging with:

kitchen login


Just as a sidenote: if you want to debug on opsworks, all interesting files are under /opt/aws/opsworks. Even better: you can modifie your cookbooks directly under /opt/aws/opsworks/current/site-cookbooks and run the opsworks-agent-cli with the correct options to rerun chef with your modified cookbooks.

Testing multiple cookbooks at once

There is an important difference between how berksfiles are interpreted by opsworks. You need a different Berksfile and metadata per environment
As a reminder, this is our directory layout:

└── project
    ├── Berksfile   # testkitchen
    ├── .kitchen.yml   #  testkitchen
    └── cookbooks
        ├── Berksfile   # opsworks
        ├── cookbook-a
        │   ├── Berksfile.in   # testkitchen
        │   └── metadata.rb   # opsworks + testkitchen
        └── cookbook-b
            ├── Berksfile.in   # testkitchen
            └── metadata.rb   # opsworks + testkitchen


One special remark: Opsworks needs a git repo with only the Berksfile and the two cookbook directories, so take care about how you commit to git!
The main Berksfile for testkitchen is the one under /project/. This calls the Berksfile.in in every cookbook.
I found this code somewhere on the internet, but can’t find the link anymore, so my apologies about not giving credit where credit is due!

source "https://supermarket.chef.io"
# Note the absence of the metadata line.
def dependencies(path)
  berks = "#{path}/Berksfile.in"
  instance_eval(File.read(berks)) if File.exists?(berks)
Dir.glob('./cookbooks/*').each do |path|
  dependencies path
  cookbook File.basename(path), :path => path


The sub Berksfiles only contain the needed cookbooks for that specific cookbook. The metadata.rb file is automatically sourced by the cookbook

cookbook "dependency-a"

cookbook "dependency-b"


Opsworks needs the following file. Again no metadata is referenced as opsworks will find the necessary files automatically.

source "https://supermarket.chef.io"
cookbook "dependency-a"
cookbook "dependency-b"


This is the ‘normal’ Berksfile for cookbook-a. You still need this if you want to test it independently!

source "https://supermarket.chef.io"
cookbook "dependency-a"



http://pixelcog.com/blog/2014/virtualizing-aws-opsworks-with-vagrant/ (this uses the opsworks provisioner, didn’t get it working yet)
http://docs.aws.amazon.com/opsworks/latest/userguide/opsworks-opsworks-mock.html (outdated)
https://www.youtube.com/watch?v=0sPuAb6nB2o (very interesting test kitchen / berkshelf tutorial)


Leave a Reply

Your email address will not be published. Required fields are marked *


Need a hand? Or a high five?
Feel free to visit our offices and come say hi
… or just drop us a message

We are ready when you are

Cloudar NV – Operations

Veldkant 7
2550 Kontich (Antwerp)

info @ cloudar.be

+32 3 450 67 18

Cloudar NV – HQ

Veldkant 33A
2550 Kontich (Antwerp)

VAT BE0564 763 890

    This contact form is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

    • SHARE