DC/OS Vagrant

Quickly provision a DC/OS cluster on a local machine for development, testing, or demonstration.

Deploying dcos-vagrant involves creating a local cluster of VirtualBox VMs using the dcos-vagrant-box base image and then installing DC/OS.

Issue tracking is moving to the DCOS JIRA (dcos-vagrant component). Issues on Github will be disabled soon.

Table of Contents

• Audience
• Goals
• Requirements
• Deploy
• Configure
• Example Clusters
• Environment Options
• License and Author

Other Docs:

• Architecture
• DC/OS Installation
• Install Ruby
• Repo Structure
• Examples
• DC/OS CLI
• Troubleshooting
• VirtualBox Guest Additions

Audience

• Developers
  • DC/OS
  • DC/OS Services
  • Mesos Frameworks
  • Marathon Apps
• Continuous Integration (testing)
• Sales Engineers (demos)
• Prospective Customers/Users (kick the tires)

Goals

• Enable free, local demonstration of the core capabilities of DC/OS
• Deploy, test, and debug development versions of DC/OS Services, Mesos Frameworks, and Marathon Apps
• Deploy, test, and debug development versions of DC/OS itself
• Decrease the cycle time from local code-change to deployment and testing
• Support multiple use cases to facilitate sharing of pain and gain
• Stay as close to the process of production deployment as possible to reduce maintenance cost of multiple deployment methods
• Facilitate onboarding of new DC/OS users by preferring intuitive usability over complex configuration
• Facilitate customization of virtualized machine resources to emulate diverse environments

Requirements

Hardware

Minimum:

• 5GB free memory (8GB system memory)

Most services cannot be installed on the Minimal cluster.

Recommended (Medium):

• 10GB free memory (16GB system memory)

Most services can be installed on the Medium cluster, but not all at the same time.

Software

• Git - clone repo
• Vagrant (>= 1.8.1) - virtualization orchestration
  • Host Manager Plugin - manage /etc/hosts
  • (Optional) VBGuest Plugin - manage vbox guest additions
• VirtualBox (>= 4.3) - virtualization engine
• (Optional) jq - json parser used by examples

Supported DC/OS Versions

DC/OS Downloads & Release Notes

• 1.7.x
  • Requires dcos-vagrant >= 0.6.0
  • No config changes since 1.6 (e.g. config-1.7.yaml)
• 1.6.x
  • Requires dcos-vagrant >= 0.4.0
  • Requires flattened yaml config (e.g. config-1.6.yaml)
• 1.5.x
  • Requires dcos-vagrant >= 0.3.0
  • Requires yaml config (e.g. config-1.5.yaml)
• CM.4
  • Requires dcos-vagrant v0.3.0

Deploy

1. Install Vagrant & VirtualBox

   For installer links, see Software Requirements.

2. Clone This Repo

   Select where you want the dcos-vagrant repo to be on your local hard drive and cd into it. Then clone the repo using git.

   git clone https://github.com/dcos/dcos-vagrant

   To upgrade to a new version of dcos-vagrant:

   1. Change into the repo directory (e.g. cd ~/workspace/dcos-vagrant)
   2. Fetch the new code (e.g. git fetch)
   3. Checkout the new desired version (e.g. git checkout v0.6.0)

3. Configure VirtualBox Networking

   Configure the host-only vboxnet0 network to use the 192.168.65.0/24 subnet.

   1. Create the vboxnet0 network if it does not exist:

      VBoxManage list hostonlyifs | grep vboxnet0 -q || VBoxManage hostonlyif create

   2. Set the vboxnet0 subnet:

      VBoxManage hostonlyif ipconfig vboxnet0 --ip 192.168.65.1
      

   For more details about the networking of a DC/OS Vagrant cluster, see the Architecture docs.

4. Install Vagrant Host Manager Plugin

   The Host Manager Plugin manages the /etc/hosts on the VMs and host to allow access by hostname.

   vagrant plugin install vagrant-hostmanager

   This will update /etc/hosts every time VMs are created or destroyed.

   To avoid entering your password on vagrant up & vagrant destroy you may enable passwordless sudo.

   On some versions of Mac OS X, installing vagrant plugins may require installing a modern version of Ruby.

5. Download the DC/OS Installer

   If you don't already have a DC/OS Installer downloaded, you'll need to select and download one of the supported versions.

   Once downloaded, move dcos_generate_config.sh to the root of the repo (the repo will be mounted into the vagrant machines as /vagrant).

   If you have multiple dcos_generate_config.sh files downloaded you can name them differently and specify which to use with DCOS_GENERATE_CONFIG_PATH (e.g. export DCOS_GENERATE_CONFIG_PATH=dcos_generate_config-1.5-EA.sh).

   Enterprise edition installers are also supported. Contact your sales representative or [email protected] to obtain right DC/OS installer.

6. Configure the DC/OS Installer

   Select a config file template based on the downloaded version of DC/OS (select one):

   • DC/OS 1.7: export DCOS_CONFIG_PATH=etc/config-1.7.yaml
   • DC/OS 1.6: export DCOS_CONFIG_PATH=etc/config-1.6.yaml
   • DC/OS 1.5: export DCOS_CONFIG_PATH=etc/config-1.5.yaml

   The path to the config file is relative to the repo dir, because the repo dir will be mounted as /vagrant within each VM. Alternate configurations may be added to the <repo>/etc/ dir and configured in a similar manner.

   Alternatively, a URL to an online config can be specified (e.g. export DCOS_CONFIG_PATH=http://example.com/config.yaml).

7. Configure the DC/OS Machine Types

   Copy the example VagrantConfig file:

   cd <repo>
   cp VagrantConfig.yaml.example VagrantConfig.yaml

   See Configure for more details on customizing your cluster.

8. (Optional) Download/Update the VM Base Image

   By default, Vagrant should automatically download the latest VM Base Image (virtualbox box) when you run vagrant up <machines>, but downloading the image takes a while the first time. You may want to trigger the download or update manually.

   vagrant box add https://downloads.dcos.io/dcos-vagrant/metadata.json
   

   If you already have the latest version downloaded, the above command will fail.

   Known Issue: Vagrant's box downloader is known to be slow. If your download is super slow (100-300k/s range), then cancelling the download (Ctrl+C) and restarting it sometimes makes it download faster.

9. (Optional) Configure Other Options

   DC/OS Vagrant supports many other configurable options via environment variables. Skip these for first time use.

10. Deploy DC/OS

    Specify which machines to deploy. For example (requires 5.5GB free memory):

    vagrant up m1 a1 p1 boot

    Many permutations of machines are possible. See Example Clusters for more options.

    Once the the machines are created and provisioned, DC/OS will be installed. Once complete, the Web Interface will be available at http://m1.dcos/.

    See the DC/OS Usage docs for more information on how to use you new DC/OS cluster.

11. (Optional) Authentication

    When installing the Enterprise Edition of DC/OS (>= 1.6) on dcos-vagrant, the cluster will prompt for a username and password when using the DC/OS CLI or the web dashboard.

    If you're using the provided 1.6 or 1.7 installer config file then the superuser credentials are by default admin/admin.

Configure

The number of machines and their resources is configurable, depending on your needs and hardware constraints.

The VagrantConfig.yaml.example includes some preset machine configurations that have been chosen to allow the widest possible use cases within a constrained memory environment (e.g. a laptop with 16GB memory). These presets may or may not fit your use case. If they don't, just modify your VagrantConfig.yaml file to fit your needs.

Deploying multiple VMs takes a lot of memory and Mesos reserves more for overhead on each node. So don't expect to be able to install every DC/OS service or use production-grade configurations. Most services will require reduced configurations in order to fit within the allocated memory. Some services (e.g. Cassandra) may require more nodes/resources than others.

For more information about how the DC/OS installation works and how to debug deployment/installation failure, see DC/OS Installation.

Node Types

Each machine in VagrantConfig.yaml must specify one of the following node types that governs how that machine will be provisioned:

• master - Master node that runs the DC/OS core components (e.g. m1)
• agent-private - Agent node that runs the Mesos agent with the * role (e.g. a1)
• agent-public - Agent node that runs the Mesos agent with the slave_public role (e.g. p1)
• boot - Bootstrap node that runs the installer (e.g. boot)

Cluster Constraints

Which exact machines are created and provisioned can be specified in one of two ways:

1. Specify the machines by name when deploying (e.g. vagrant up m1 a1 p1 boot)
2. Remove the unwanted machines from the VagrantConfig.yaml file and deploy them all with vagrant up

Generally option 1 is recommended to avoid having to modify the VagrantConfig.yaml file.

When selecting which machines to deploy, the following constraints must be observed:

• An odd number of master nodes is required (usually 1, 3, or 5)
• Any number of public and/or private agent nodes is allowed
• Exactly one bootstrap node is required
• The bootstrap node must be provisioned last

Resource Constraints

Mesos agent nodes are where DC/OS services will be installed. Mesos will auto-detect the amount of resources available on these machines, with the following constraint:

• Mesos reserves half or 1 GB of each machine's memory for overhead (whichever is least)

For example, m1 has 3328 MB memory by default. Some of that memory will be taken by OS and DC/OS component processes (~ MB). 1 GB will be reserved by Mesos as overhead. The rest will be offered to Mesos frameworks for launching tasks (~ MB).

IMPORTANT: Make sure your local machine has enough memory to launch all your desired VMs, otherwise your machine may lock up as all the memory is consumed.

Example Clusters

Any permutation of machines that fits the above constraints is possible. Below are a few options to try.

Minimal Cluster

A minimal cluster supports launching small Marathon apps. Most other services will fail to install, because they require more than one agent node.

Requires > 4.5GB free memory (using the example VagrantConfig).

vagrant up m1 a1 boot

Small Cluster

A small cluster supports running tasks on multiple nodes.

Requires > 7.25GB free memory (using the example VagrantConfig).

vagrant up m1 a1 a2 p1 boot

Medium Cluster

A medium cluster supports the installation of a minimally configured Cassandra.

Requires > 10GB free memory (using the example VagrantConfig).

vagrant up m1 a1 a2 a3 a4 p1 boot

Large Cluster

Requires > 17GB free memory (using the example VagrantConfig).

A large cluster supports master node fail over, multiple framework installs, and multiple public load balancers.

vagrant up m1 m2 m3 a1 a2 a3 a4 a5 a6 p1 p2 p3 boot

Environment Options

There are several configurable options when deploying a cluster and installing DC/OS on it. Most of them are configurable via environment variables:

• DCOS_BOX - VirtualBox box image name (default: mesosphere/dcos-centos-virtualbox)
• DCOS_BOX_URL - VirtualBox box image url or vagrant-cloud style image repo (default: https://downloads.dcos.io/dcos-vagrant/metadata.json)
• DCOS_BOX_VERSION - VirtualBox box image version (default: ~> 0.5.0)
• DCOS_MACHINE_CONFIG_PATH - Path to virtual machine configuration manifest (default: VagrantConfig.yaml)
  • Must contain at least one boot type machine, one master type machine, and one agent or agent-public type machine.
• DCOS_CONFIG_PATH - Path to DC/OS configuration template (default: etc/config.yaml)
  • master_list, agent_list, exhibitor_zk_hosts, and bootstrap_url will be overridden.
• DCOS_GENERATE_CONFIG_PATH - Path to DC/OS configuration generation script (default: dcos_generate_config.sh)
• DCOS_INSTALL_METHOD - One of the following methods (default: ssh_pull):
  • ssh_pull - Use the "manual" DC/OS installation method (dcos_install.sh) with a pool of thread workers performing remote SHH installation.
  • ssh_push - Use the "automated" DC/OS installation method (dcos_generate_config.sh --deploy). WARNING: Does not (yet) support agent-public nodes!
• DCOS_JAVA_ENABLED - Boolean to install java on each agent (default: false)
• DCOS_PRIVATE_REGISTRY - Boolean to install an insecure private Docker registry on the boot machine and configure the agents to allow it (default: false)
• DCOS_VAGRANT_MOUNT_METHOD - One of the following methods (default: virtualbox):
  • virtualbox - Use cross-platform VirtualBox shared folders
  • nfs - Use faster NFS shared folders.

Additional advanced configuration may be possible by modifying the Vagrantfile directly, but is not encouraged because the internal APIs may change at any time.

License

Copyright 2016 Mesosphere, Inc.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this repository except in compliance with the License.

The contents of this repository are solely licensed under the terms described in the LICENSE file included in this repository.

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.