Systems and Architecture
We had a use case in test-kitchen where one of our end-to-end cookbook tests installed so many packages and support tools, it was filling up the 10Gb root disk of our custom VirtualBox images. Rather than make larger images, since most groups were not having the same issue, it made more sense to add more disk space when needed to the Vagrant/VirtualBox instances.
Potential use cases other than simply needing more disk space include testing recipes that create RAID devices, logical volumes, filesystem creation and manipulation, and so on.
There were two obvious options here: grow the image or add a disk. Growing the existing image would have required resizing the root filesystem or creating and formatting a new filesystem on the root 'disk' within the test-kitchen provision. While this was possible, it requires a recipe that is never needed in actual production systems, and it could be potentially time-consuming.
Adding a second virtual disk made more sense, in large part because we already have an internal Chef cookbook that finds additional, unformatted disks on a host and creates, formats, and mounts them as a RAID device. By prepending that recipe to our test-kitchen suite's
run_list, we would be using existing code and, if anything, mimicking the production environment even more.
However, probably the most important consideration was to implement the solution in a way that disrupted our established Chef cookbook development and test workflow as little as possible. Ideally, after copying over a few generic scripts or configs into the cookbook, an engineer would only need to add an attribute to a suite in their cookbook's
.kitchen.yml file and everything else would be automatic.
The final solution, which is, admittedly, just one way of getting to the same end goal, required a bit more than just that magic attribute, but not by much.
By passing the kitchen-vagrant driver a supplementary
Vagrantfile using the
vagrantfiles option, it was quite easy to create an unformatted virtual disk and add it to the instance. The
Vagrantfile would look something like this:
disk = '/my/super/special/instance/disk.vdi' Vagrant.configure('2') do |c| c.vm.provider :virtualbox do |p| p.customize ['createhd', '--filename', disk, '--size', '10240'] p.customize ['storageattach', :id, '--storagectl', 'IDE Controller', '--port', '1', '--device', '1', '--type', 'hdd', '--medium', disk] end end
One consideration was to make sure only suites needing an additional disk got it, and that the additional disk would be unique for each instance, to make parallel convergence safe. Therefore adding a one-size-fits-all
Vagrantfile via the
vagrantfiles option wasn't viable. (While
Vagrantfiles are interpreted ruby, nothing in the kitchen-vagrant execution was exposing, say, the instance name to the vagrant run, making using the same file for every instance and setting paths dynamically impractical.)
How to get a
Vagrantfile hardcoded with the necessary information in the right place?
kitchen-vagrant also supports a
pre_create_command hook, but it also exposes very little information, other than the Vagrant path of the current instance. As of
kitchen-vagrant-0.15.0, the hook wasn't passing the vagrant root properly (although it is fixed in 0.16), and even after patching, it still posed a few problems, as the suite attributes were not available to the script, preventing the decision of whether or not to create the additional disk from being made intelligently.
As it looked like the best place to do the pre-processing was outside test-kitchen execution,
rake became the obvious solution. We already encourage use of
Rakefiles in our cookbooks for testing, so pre-processing would just require adding a few lines. Yes, this would require an additional setup step, but it did have the benefit of not requiring making a local fork of
kitchen-vagrant to allow hackiness that didn't necessarily make sense for the entire kitchen community.
A few tasks got added to the cookbook
# Integration tests. Kitchen.ci namespace :integration do desc 'Run test-kitchen pre-processing scripts' task :pre_cmds do cmd = Mixlib::ShellOut.new('./scripts/write_vagrantfile.rb') cmd.run_command end end # Clean up namespace :cleanup do desc 'Destroy test-kitchen instances' task :kitchen_destroy do destroy = Kitchen::RakeTasks.new do |obj| def obj.destroy() config.instances.each do |instance| instance.destroy end end end destroy.destroy end desc 'Remove vagrant disks' task :rm_vdi do ::FileUtils.rm_rf('./vagrant_disks') end desc 'Remove Vagrantfiles/ dir' task :rm_vagrantfiles do ::FileUtils.rm_rf('./Vagrantfiles') end desc 'Remove .kitchen.local.yml' task :rm_kitchen_local do ::File.unlink('.kitchen.local.yml') if ::File.exist?('.kitchen.local.yml') end end desc 'Run full integration' task integration: %w(integration:pre_cmds integration:vagrant) desc 'Generate the setup' task setup: %w(integration:pre_cmds) desc 'Clean up generated files' task cleanup: %w(cleanup:kitchen_destroy cleanup:rm_vdi cleanup:rm_kitchen_local cleanup:rm_vagrantfiles)
write_vagrantfile.rb script is admittedly a bit of a hack; it reads and parses the
.kitchen.yml and then adds some glue and writes a
.kitchen.local.yml. Because some of the information it needs comes from private methods in the
kitchen-vagrant classes, some of those method definitions were ganked and pasted into the script.
.kitchen.ymlfile and create the associated
Vagrantfilesfor the Vagrant run for those instances.
.kitchen.local.ymlfile to add the
vagrantfilesdriver option with the appropriate
Vagrantfilepath for each suite.
The solution is perhaps not the most elegant in terms of pretending like we aren't looking under Kitchen's hood or being 100% integrated into kitchen execution, but it's reasonably lightweight and flexible, and most importantly, it fills a need we had.
Example code for this is available on Ooyala's github.