Extensive Kitchen-Terraform

A detailed review of the features of Kitchen-Terraform.

Author: Aaron Lane

1. Introduction 2. Configure Amazon Web Services 3. Create InSpec Profile 4. Generate SSH Key 5. Create Test Fixture Terraform Configuration 6. Create Terraform Module 7. Create Test Kitchen Configuration 8. Test Terraform Module

This tutorial provides a detailed review of the features of Kitchen-Terraform by developing a Terraform module which configures resources on the Amazon Web Services (AWS) platform.

Kitchen-Terraform is assumed to be installed on the development system according to the instructions in the Kitchen-Terraform ReadMe.

The files and scripts shown in this tutorial are sourced from a functional example project in the Kitchen-Terraform repository.

Any issues with this tutorial should be reported in the Kitchen-Terraform issue tracker.

This tutorial creates AWS resources using the Terraform AWS Provider so registration of an AWS account is required.

The AWS Command Line Interface (CLI) must be installed and configured with the credentials of the AWS account by following the AWS CLI Quick Configuration instructions.

The JSON filtering tool jq must be installed to enable the parsing of output from the AWS CLI in Bash scripts.

IAM users with API keys must be created in the us-east-1 region and the us-west-2 region. Those IAM users must be authorized to manage a variety of AWS resources, which can be accomplished by adding an inline custom policy like the following example to the IAM users.

{
  "Version": "2012-10-17",
  "Statement":
    [
      {
        "Sid": "Stmt1469773655000",
        "Effect": "Allow",
        "Action": ["ec2:*"],
        "Resource": ["*"]
      }
    ]
}

Prepare the InSpec profile by mimicking the following Bash script.

#! /usr/bin/env bash

mkdir -p test/integration/extensive_suite/controls
touch test/integration/extensive_suite/inspec.yml
touch test/integration/extensive_suite/centos_attributes.yml
touch test/integration/extensive_suite/ubuntu_attributes.yml
touch test/integration/extensive_suite/controls/inspec_attributes.rb
touch test/integration/extensive_suite/controls/operating_system.rb
touch test/integration/extensive_suite/controls/reachable_other_host.rb
touch test/integration/extensive_suite/controls/state_file.rb

Populate the InSpec profile description file,

test/integration/extensive_suite/inspec.yml

, to match the following example.

name: extensive_suite
title: Extensive Kitchen-Terraform
version: 0.1.0
attributes:
  - name: static_terraform_output
    type: string
    required: true
    description: The Terraform configuration under test must define an equivalently named output
  - name: customized_inspec_attribute
    type: string
    required: true
    description: The Test Kitchen configuration must map this attribute to the 'static_terraform_output' output
  - name: instances_ami_operating_system
    type: string
    required: true
    description: The Terraform configuration under test must define the equivalently named output
  - name: reachable_other_host_ip_address
    type: string
    required: true
    description: The Terraform configuration under test must define the equivalently named output
  - name: terraform_state
    type: string
    required: true
    description: The Terraform configuration under test must define the equivalently named output

Populate the InSpec attributes control,

test/integration/extensive_suite/controls/inspec_attributes.rb

, to match the following example.

# frozen_string_literal: true

# Copyright 2016-2021 Copado NCS LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

control "inspec_attributes" do
  desc "A demonstration of how InSpec attributes are mapped to Terraform outputs"

  describe attribute("static_terraform_output") do
    it { should eq "static terraform output" }
  end

  describe attribute("customized_inspec_attribute") do
    it { should eq "static terraform output" }
  end
end

Populate the operating system control,

test/integration/extensive_suite/controls/operating_system.rb

, to match the following example.

# frozen_string_literal: true

# Copyright 2016-2021 Copado NCS LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

control "operating_system" do
  desc "Verifies the name of the operating system on the targeted host"

  describe os.name do
    it { should eq attribute("instances_ami_operating_system_name") }
  end
end

Populate the attributes for the centos test,

test/integration/extensive_suite/centos_attributes.yml

, to match the following example.

instances_ami_operating_system_name: centos

Populate the attributes for the ubuntu test,

test/integration/extensive_suite/ubuntu_attributes.yml

, to match the following example.

instances_ami_operating_system_name: ubuntu

Populate the reachable other host control,

test/integration/extensive_suite/controls/reachable_other_host.rb

, to match the following example.

# frozen_string_literal: true

# Copyright 2016-2021 Copado NCS LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

control "reachable_other_host" do
  desc "Verifies that the other host is reachable from the current host"

  describe host attribute("reachable_other_host_ip_address") do
    it { should be_reachable }
  end
end

Populate the state file control,

test/integration/extensive_suite/controls/state_file.rb

, to match the following example.

# frozen_string_literal: true

# Copyright 2016-2021 Copado NCS LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

control "state_file" do
  desc "Verifies that the Terraform state file can be used in InSpec controls"

  describe json(attribute("terraform_state").chomp).terraform_version do
    it { should match /\d+\.\d+\.\d+/ }
  end
end

Mimic the following Bash script to generate an authentication key for SSH to be used by InSpec when authenticating with the AWS EC2 instances under test.

#!/bin/bash

# Make a directory to contain the key
mkdir -p test/assets

# Generate a 4096 bit RSA key with a blank passphrase in the directory
ssh-keygen \
  -b 4096 \
  -C "Kitchen-Terraform AWS provider tutorial" \
  -f test/assets/key_pair \
  -N "" \
  -t rsa

Using a test fixture Terraform configuration to invoke the Terraform module under test provides two advantages. First, it enables the expansion of the variable interface and the output interface of the module to better support testing without polluting the core logic. Second, it provides an opportunity to test the user experience of the module and identify areas of the design that can be improved or simplified.

Prepare the configuration by mimicking the following Bash script.

#! /usr/bin/env bash

mkdir -p test/fixtures/wrapper
touch test/fixtures/wrapper/main.tf test/fixtures/wrapper/outputs.tf \
  test/fixtures/wrapper/variables.tf

Populate the variables file,

test/fixtures/wrapper/variables.tf

, to match the following example.

variable "instances_ami" {
  description = <<EOD
The Amazon Machine Image (AMI) to use for the AWS EC2 instances of the module
EOD

  type = "string"
}

variable "subnet_availability_zone" {
  description = <<EOD
The isolated, regional location in which to place the subnet of the module
EOD

  type = "string"
}

Populate the main file,

test/fixtures/wrapper/main.tf

, to match the following example.

module "extensive_kitchen_terraform" {
  instances_ami = "${var.instances_ami}"

  # The generated key pair will be used to configure SSH authentication
  key_pair_public_key = "${file("${path.module}/../../assets/key_pair.pub")}"

  # The source of the module is the root directory of the Terraform project
  source                   = "../../../"
  subnet_availability_zone = "${var.subnet_availability_zone}"
}

Populate the outputs file,

test/fixtures/wrapper/outputs.tf

, to match the following example.

output "reachable_other_host_ip_address" {
  description = <<EOD
This output is used as an attribute in the reachable_other_host control
EOD

  value = <<EOV
${module.extensive_kitchen_terraform.reachable_other_host_ip_address}
EOV
}

output "static_terraform_output" {
  description = <<EOD
This output is used as an attribute in the inspec_attributes control
EOD

  value = "static terraform output"
}

output "terraform_state" {
  description = "This output is used as an attribute in the state_file control"

  value = <<EOV
${path.cwd}/terraform.tfstate.d/${terraform.workspace}/terraform.tfstate
EOV
}

output "remote_group_public_dns" {
  description = "This output is used to obtain targets for InSpec"

  value = ["${module.extensive_kitchen_terraform.remote_group_public_dns}"]
}

The Terraform module is the actual artifact under test. The design of the module is informed by the InSpec profile and the fixture configuration.

Prepare the module by mimicking the following Bash script.

#! /usr/bin/env bash

touch main.tf variables.tf outputs.tf

Populate the variables file,

variables.tf

, to match the following example.

variable "instances_ami" {
  description = "The Amazon Machine Image (AMI) to use for the AWS EC2 instances"
  type        = "string"
}

variable "key_pair_public_key" {
  description = <<EOD
The public key material to use for SSH authentication with the instances
EOD

  type = "string"
}

variable "subnet_availability_zone" {
  description = "The isolated, regional location in which to place the subnet"
  type        = "string"
}

Populate the main file,

main.tf

, to match the following example.

terraform {
  # The configuration is restricted to Terraform versions supported by
  # Kitchen-Terraform
  required_version = ">= 0.11.4, < 0.12.0"
}

provider "aws" {
  version = "~> 1.31"
}

provider "random" {
  version = "~> 1.3"
}

# These aws_instances will be targeted with the operating_system control and the
# reachable_other_host control
resource "aws_instance" "remote_group" {
  ami           = "${var.instances_ami}"
  count         = 2
  instance_type = "t2.micro"
  key_name      = "${aws_key_pair.extensive_tutorial.key_name}"
  subnet_id     = "${aws_subnet.extensive_tutorial.id}"

  tags {
    Name      = "kitchen-terraform-test-target-${count.index}"
    Terraform = "true"
  }

  vpc_security_group_ids = ["${aws_security_group.extensive_tutorial.id}"]
}

# The reachable_other_host control will attempt to connect to this aws_instance
# from each of the remote_group aws_instances which will verify the configuration
# of the associated aws_security_group
resource "aws_instance" "reachable_other_host" {
  ami                         = "${var.instances_ami}"
  associate_public_ip_address = true
  instance_type               = "t2.micro"
  key_name                    = "${aws_key_pair.extensive_tutorial.key_name}"
  subnet_id                   = "${aws_subnet.extensive_tutorial.id}"

  tags {
    Name      = "kitchen-terraform-reachable-other-host"
    Terraform = "true"
  }

  vpc_security_group_ids = ["${aws_security_group.extensive_tutorial.id}"]
}

resource "aws_key_pair" "extensive_tutorial" {
  key_name = "kitchen-terraform-${random_string.key_name.result}"

  public_key = "${var.key_pair_public_key}"
}

resource "random_string" "key_name" {
  length  = 9
  special = false
}

resource "aws_security_group" "extensive_tutorial" {
  description = "Allow all inbound traffic"

  egress {
    cidr_blocks = ["0.0.0.0/0"]
    from_port   = 0
    protocol    = "-1"
    to_port     = 0
  }

  ingress {
    cidr_blocks = ["0.0.0.0/0"]
    from_port   = 0
    protocol    = "-1"
    to_port     = 0
  }

  name = "kitchen-terraform-extensive_tutorial"

  tags {
    Name      = "kitchen-terraform-extensive_tutorial"
    Terraform = "true"
  }

  vpc_id = "${aws_vpc.extensive_tutorial.id}"
}

resource "aws_route_table_association" "extensive_tutorial" {
  route_table_id = "${aws_route_table.extensive_tutorial.id}"
  subnet_id      = "${aws_subnet.extensive_tutorial.id}"
}

resource "aws_route_table" "extensive_tutorial" {
  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = "${aws_internet_gateway.extensive_tutorial.id}"
  }

  tags {
    Name = "kitchen_terraform_extensive_tutorial"
  }

  vpc_id = "${aws_vpc.extensive_tutorial.id}"
}

resource "aws_internet_gateway" "extensive_tutorial" {
  tags = {
    Name = "kitchen_terraform_extensive_tutorial"
  }

  vpc_id = "${aws_vpc.extensive_tutorial.id}"
}

resource "aws_subnet" "extensive_tutorial" {
  availability_zone       = "${var.subnet_availability_zone}"
  cidr_block              = "192.168.1.0/24"
  map_public_ip_on_launch = "true"

  tags {
    Name = "kitchen_terraform_extensive_tutorial"
  }

  vpc_id = "${aws_vpc.extensive_tutorial.id}"
}

resource "aws_vpc" "extensive_tutorial" {
  cidr_block           = "192.168.0.0/16"
  enable_dns_hostnames = "true"

  tags {
    Name = "kitchen_terraform_extensive_tutorial"
  }
}

Populate the outputs file,

outputs.tf

, to match the following example.

output "reachable_other_host_ip_address" {
  description = "The IP address of the reachable_other_host instance"
  value       = "${aws_instance.reachable_other_host.public_ip}"
}

output "remote_group_public_dns" {
  description = "The list of public DNS names of the remote_group instances"
  value       = ["${aws_instance.remote_group.*.public_dns}"]
}

Populate the Test Kitchen configuration file,

kitchen.yml

, to match the following example.

driver:
  name: terraform

transport:
  name: terraform
  # The test fixture Terraform configuration is configured to be the Terraform
  # root module
  root_module_directory: test/fixtures/wrapper

provisioner:
  name: terraform

verifier:
  name: terraform

# Platforms provide hooks for overriding the global Test Kitchen plugin
# configuration to provide platform-specific values
platforms:
  - name: centos

    driver:
      variables:
        instances_ami: ami-ae7bfdb8

        subnet_availability_zone: us-east-1a

    verifier:
      systems:
        - name: local

          # The customized_inspec_attribute InSpec attribute is configured to
          # satisfy the inspec_attributes control
          attrs_outputs:
            customized_inspec_attribute: static_terraform_output

          backend: local

          # A subset of the controls included in the extensive_suite InSpec
          # profile will be executed
          controls:
            - inspec_attributes
            - state_file

        - name: remote

          attrs:
            - test/integration/extensive_suite/centos_attributes.yml

          backend: ssh

          controls:
            - operating_system
            - reachable_other_host

          # The value of the Terraform output named remote_group_public_dns will
          # be used to obtain the hostnames to target with InSpec
          hosts_output: remote_group_public_dns

          # The generated key pair is configured to be used for the SSH
          # authentication performed by InSpec
          key_files:
            - test/assets/key_pair

          user: centos

  - name: ubuntu

    driver:
      variables:
        instances_ami: ami-1ee65166

        subnet_availability_zone: us-west-2b

    verifier:
      systems:
        - name: local

          # The customized_inspec_attribute InSpec attribute is configured to
          # satisfy the inspec_attributes control
          attrs_outputs:
            customized_inspec_attribute: static_terraform_output

          backend: local

          # A subset of the controls included in the extensive_suite InSpec
          # profile will be executed
          controls:
            - inspec_attributes
            - state_file

        - name: remote

          attrs:
            - test/integration/extensive_suite/ubuntu_attributes.yml

          backend: ssh

          controls:
            - operating_system
            - reachable_other_host

          # The value of the Terraform output named remote_group_public_dns will
          # be used to obtain the hostnames to target with InSpec
          hosts_output: remote_group_public_dns

          # The generated key pair is configured to be used for the SSH
          # authentication performed by InSpec
          key_files:
            - test/assets/key_pair

          user: ubuntu

# Suites include tests and provide additional hooks for overriding the global Test
# Kitchen plugin configuration
suites:
  - # Kitchen-Terraform will assume that the InSpec profile for this suite is
    # located at test/integration/extensive_suite
    name: extensive_suite

Populate the Bundler dependency file,

Gemfile

, to match the following example.

# frozen_string_literal: true

source "https://rubygems.org/" do
  gem "kitchen-terraform", "~> 7.0"
end

Install the declared Ruby gem dependencies by mimicking the following Bash script.

#! /usr/bin/env bash

gem install bundler
bundle install

Test the Terraform module using multiple platforms by mimicking the following Bash script.

#! /usr/bin/env bash

# Unset AWS STS session environment variables
function drop_aws_sts_session {
  unset AWS_ACCESS_KEY_ID
  unset AWS_DEFAULT_REGION
  unset AWS_SECRET_ACCESS_KEY
  unset AWS_SESSION_TOKEN
}

# Export AWS STS session environment variables
function export_aws_sts_session {
  drop_aws_sts_session
  session="$(aws sts get-session-token --output=json)"
  AWS_ACCESS_KEY_ID="$(echo $session | jq -r .Credentials.AccessKeyId)"
  AWS_DEFAULT_REGION="$1"
  AWS_SECRET_ACCESS_KEY="$(echo $session | jq -r .Credentials.SecretAccessKey)"
  AWS_SESSION_TOKEN="$(echo $session | jq -r .Credentials.SessionToken)"
  export AWS_ACCESS_KEY_ID
  export AWS_DEFAULT_REGION
  export AWS_SECRET_ACCESS_KEY
  export AWS_SESSION_TOKEN
}

export_aws_sts_session "us-east-1"

# Destroy any existing Terraform state in us-east-1
bundle exec kitchen destroy centos

# Initialize the Terraform working directory and select a new Terraform workspace
# to test CentOS in us-east-1
bundle exec kitchen create centos

# Apply the Terraform root module to the Terraform state using the Terraform
# fixture configuration
bundle exec kitchen converge centos

# Test the Terraform state using the InSpec controls
bundle exec kitchen verify centos

# Destroy the Terraform state using the Terraform fixture configuration
bundle exec kitchen destroy centos

export_aws_sts_session "us-west-2"

# Perform the same steps for Ubuntu in us-west-2
bundle exec kitchen test ubuntu

drop_aws_sts_session