- What is Vanagon?
- Runtime Requirements
- Configuration and Usage
- Engines
- Contributing
- License
- Maintainers
Vanagon is a tool to build a single package out of a project, which can itself
contain one or more components. This tooling is being used to develop the
puppet-agent package, which contains components such as openssl, ruby, and
augeas among others. For a simple example, please see the project in the
examples/ directory.
Vanagon builds up a Makefile and packaging files (specfile for RPM, control/rules/etc for DEB) and copies them to a remote host, where make can be invoked to build all of the components and make a package of the contents.
Vanagon (/ˈvænəgɪn/) sounds like "van again." It does not sound like "van wagon" or "van and gone."
Vanagon carries two sets of requirements: requirements for the local host where Vanagon is run, and for the remote target where compilation and packaging happens.
Also, Vanagon ships with a number of engines which may include additional optional dependencies if you wish to use them. These engines are currently considered experimental, and receive less attention than the Hardware or vmpooler engines do. If you find a bug in these engines, please open a ticket and let us know.
- Ruby (Ruby 2.3.x is the miniumum supported version)
- fustigit
- ruby-git
- docopt
- The command line tool
ssh(homepage) available on the local${PATH}(any modern version should suffice) - The command line tool
rsync(homepage) available on the local${PATH}(At least rsync 2.6.x) - The command line tool
git(homepage) available on the local${PATH}(Vanagon is tested against Git version 1.8.x but should work with any newer version)
- AWS SDK for Ruby, if you're using the EC2 engine
- Docker, if you're using the Docker engine
Note: package installation & builder configuration for the remote target can be customized in the Platform configuration that defines target provisioning instructions.
- GNU Make (homepage) (Vanagon specifically targets the feature set provided by GNU Make 3.81 but newer versions are known to work -- older versions are specifically known to not work!)
- Bash (homepage) is required by the Makefiles that Vanagon generates
- An ssh server (homepage) is required by most engines
- The command line tool
rsync(homepage) (At least rsync 2.6.x)
Vanagon won't be much use without a project to build. Beyond that, you must define any platforms you want to build for. Vanagon ships with some simple binaries to use, but the one you probably care about is named 'build'.
Vanagon is broken down into three core ideas: the project, the component and the platform. The project contains one or more components and is built for a platform. As a quick example, if I had a ruby app and wanted to package it, the project would probably contain a component for ruby and a component for my app. If I wanted to build it for debian wheezy, I would define a platform called wheezy and build my project against it.
For more detailed examples of the DSLs available, please see the examples directory and the YARD documentation for Vanagon.
The vanagon command line tool contains these commands:
buildbuild a package given a project and platformbuild_host_infoprint information about build hostsbuild_requirementsprint external packages required to build projectcompletionoutputs path to tab completion scriptinspecta build dry-run, printing lots of information about the buildlistshows a list of available projects and platformsrendercreate local versions of packaging artifacts for projectsignsign a packageshipupload a package to a distribution serverhelpprint this help
Commands are called with a git-like pattern of vanagon <subcommand>.
For example: vanagon list
Prior to 0.16.0, the vanagon command line contained these commands
buildbuild_host_infobuild_requirementsinspectrenderreposhipsign
With the exception of repo, which calls packaging methods, the remaining commands
have been moved to a git-like pattern of vanagon <subcommand>. vangon build replaced build, vanagon ship replaced ship and so forth.
The older calling usage is deprecated.
The build command has positional arguments and position independent flags.
The name of the project to build; a file named <project_name>.rb must be
present under configs/projects in the working directory.
The name of the target platform to build <project_name> against; a file named
<platform_name>.rb must be present under configs/platforms in the working
directory. This can also be a comma separated list of platforms such as platform1,platform2;
note that there are no spaces after the comma.
Target host is an optional argument to override the host selection. Instead of using
a random VM collected from the pooler (Vanagon's default build engine), the build will
attempt connect to the target host over SSH as the root user.
If building on multiple platforms, multiple targets can also be specified using
a comma separated list such as host1,host2 (note that there are no spaces after
the comma). If less targets are specified than platforms, the default engine
(pooler) will be used for platforms without a target. If more targets are specified
than platforms, the extra platforms will be ignored.
Build machines should be cleaned between builds.
Note: command flags currently can be used anywhere in the command. Recommended usages is to use flagged arguments before the positional arguments.
Specifies a directory on the local host where the sources should be placed and builds performed.
Defaults to a temporary directory created with Ruby's Dir.mktmpdir method.
Explicitly specify a directory on the remote target to place sources and perform
builds. Components can then be rebuilt manually on the build host for faster iteration. Sources may not be correctly updated if this directory already exists.
Defaults to a temporary directory created by running mktemp -d on the remote target.
Specifies where project configuration is found. Defaults to $pwd/configs.
Choose a different virtualization engine to use to select the build target. Currently supported engines are:
base- Pure ssh backend; no teardown currently definedlocal- Build on the local machine; platform name must match the local machinedocker- Builds in a docker containerpooler- Selects a vm from Puppet's internal vmpooler to build onhardware- Build on a specific taget and lock it in redisec2- Build on a specific AWS instance.ABS- Selects a vm from Puppet's internal Always-be-scheduling service to build on
Note: command flags can be used anywhere in the command.
Indicates that the host used for building the project should be left intact after the build instead of destroyed. The host is usually destroyed after a successful build, or left after a failed build.
Display command-line help.
A full path on disk for a private ssh key to be used in ssh and rsync communications. This will be used instead of whatever defaults are configured in .ssh/config.
When set, Vanagon will forward the ssh authentication agent connection.
Used in conjunction with the pooler engine, this is a token to pass to the vmpooler to access the API. Without this token, the default lifetime of vms will be much shorter.
The name of the host where redis is running. Redis is used to handle a lock when using the hardware engine. It defaults to redis, with no domain.
Port of the system where redis is running. Defaults to 6379.
Controls whether component sources are downloaded directly from upstream URLs
or from configured mirrors. Most Puppet projects using Vanagon default to
fetching components from internal mirrors. Set this variable to n or false
when building outside of the Puppet private network to download directly from
upstream sources.
Some phases of compilation support retries. The default value is 1 but
setting to any integer value greater than 1 will causes these components
to retry operations on failure until the VANAGON_RETRY_COUNT limit is reached.
Some phases of compilation can take an indeterminate (but substantial) amount of
time. The default value is 7200 seconds(120 minutes) but setting to any
integer value these components to fail after the VANAGON_TIMEOUT count is reached.
Note that this value is expected to be in seconds.
By default, Vanagon does not fail if extra files signing fails, it just logs an error and continues building the package. This is unwanted behavior in environments where we expect a hard failure when signing cannot proceed. To force Vanagon to fail if extra files signing fails, ensure this variable is set before starting a build.
vanagon build --preserve puppet-agent el-6-i386 will build the puppet-agent project
on the el-6-i386 platform and leave the host intact afterward.
vanagon build --engine=docker puppet-agent el-6-i386 will build the puppet-agent
project on the el-6-i386 platform using the docker engine (the platform must
have a docker_image defined in its config).
The inspect command has positional arguments and position independent flags. It
mirrors the build command, but exits with success after loading and interpolating
all of the components in the given project. No attempt is made to actually build
the given project; instead, a JSON formatted array of hashes is returned and printed
to stdout. This JSON array can be further processed by external tooling, such as jq.
The name of the project to build, and a file named <project_name>.rb must be present in configs/projects in the working directory.
The name of the platform to build against, and a file named <platform_name>.rb must be present in configs/platforms in the working directory.
Platform can also be a comma separated list of platforms such as platform1,platform2.
Note: command flags currently can be used anywhere in the command. Recommended usages is to use flagged arguments before the positional arguments.
Specifies a directory where the sources should be placed and builds performed. Defaults to a temporary directory created with Ruby's Dir.mktmpdir.
Specifies where project configuration is found. Defaults to $pwd/configs.
Choose a different virtualization engine to use to select the build target.
Engines are respected, but only insofar as components and projects are
rendered -- the vanagon inspect command performs no compilation.
Supported engines are the same as the vanagon build command.
Display command-line help.
Environment variables are respected, but only insofar as components and projects are
rendered -- the vanagon inspect command has no behavior to alter.
Supported environment variables are the same as the vanagon build command.
vanagon inspect puppet-agent el-6-i386 will load the puppet-agent project
on the el-6-i386 platform and print the resulting list of dependencies,
build-time configuration, environment variables, and expected artifacts.
Note: If you have the aws_ami setup Vanagon will default to the ec2 engine.
To use the ec2 engine you should have your credentials set either via your ~/.aws/credentials or environment variables.
After this you can setup your configs/platforms/<platform>.rb to use your
ami, instance type, and key_name to setup the instance.
A simple one looks like this
# configs/platforms/el-7-x86_64.rb
platform "el-7-x86_64" do |plat|
plat.aws_ami "your-ami-id-here" # You must set this
plat.aws_instance_type "t2.small" # Defaults to t1.micro
plat.aws_key_name "vanagon" # this is the default but you can use whichever
plat.aws_user_data <<-eos
#cloud-config
runcmds:
- echo #{my_ssh_key} > /root/.ssh/authorized_keys # Most amis block you from logging in as root.
eos
### Rest of your code here
endWhen using the ABS engine, there is a variety of ways you can specify your token:
- the environment variable ABS_TOKEN
- or vanagon token file ~/.vanagon-token (note this is the same file read by the pooler engine)
- or vmfloaty's config file ~/.vmfloaty.yml
In order to modify or track the VM via floaty or bit-bar you can optionally add the vmpooler token (different from the ABS token) via
- VMPOOLER_TOKEN
- or as a second line to the file ~/.vanagon-token
- or by using the existing mechanism in floaty using a vmpooler_fallback
We'd love to get contributions from you! Once you are up and running, take a look at the Contribution Documents to see how to get your changes merged in.
See LICENSE file.
See MAINTAINERS file.