This repo is a BOSH release for deploying Diego and associated tasks for testing a Diego deployment. Diego builds out the new runtime architecture for Cloud Foundry, replacing the DEAs and Health Manager.
This release relies on a separate deployment to provide NATS and Loggregator. In practice these come from cf-release.
Learn more about Diego and its components at diego-design-notes.
When working on individual components of Diego, work out of the submodules under src/.
See Initial Setup.
Run the individual component unit tests as you work on them using
ginkgo. To see if everything still works, run
./scripts/run-unit-tests in the root of the release.
When you're ready to commit, run:
./scripts/prepare-to-diego <story-id> <another-story-id>...
This will synchronize submodules, update the BOSH package specs, run all unit tests, all integration tests, and make a commit, bringing up a commit edit dialogue. The story IDs correspond to stories in our Pivotal Tracker backlog. You should simultaneously also build the release and deploy it to a local BOSH-Lite environment, and run the acceptance tests. See Running Smoke Tests & DATs.
If you're introducing a new component (e.g. a new job/errand) or changing the main path
for an existing component, make sure to update ./scripts/sync-package-specs and
./scripts/sync-submodule-config.
This BOSH release doubles as a $GOPATH. It will automatically be set up for
you if you have direnv installed.
# fetch release repo
mkdir -p ~/workspace
cd ~/workspace
git clone https://github.com/cloudfoundry-incubator/diego-release.git
cd diego-release/
# automate $GOPATH and $PATH setup
direnv allow
# switch to develop branch (not master!)
git checkout develop
# initialize and sync submodules
./scripts/update
If you do not wish to use direnv, you can simply source the .envrc file in the root
of the release repo. You may manually need to update your $GOPATH and $PATH variables
as you switch in and out of the directory.
-
Install ginkgo
go install github.com/onsi/ginkgo/ginkgo -
Install gnatsd
go install github.com/apcera/gnatsd -
Install etcd
go install github.com/coreos/etcd -
Install consul
if uname -a | grep Darwin; then os=darwin; else os=linux; fi curl -L -o $TMPDIR/consul-0.5.0.zip "https://dl.bintray.com/mitchellh/consul/0.5.0_${os}_amd64.zip" unzip $TMPDIR/consul-0.5.0.zip -d ~/workspace/diego-release/bin rm $TMPDIR/consul-0.5.0.zip -
Run the unit test script
./scripts/run-unit-tests
-
Install the
flyCLI:# cd to the concourse release repo, cd /path/to/concourse/repo # switch to using the concourse $GOPATH and $PATH setup temporarily direnv allow # install the version of fly from Concourse's release go install github.com/concourse/fly # add the concourse release repo's bin/ directory to your $PATH export PATH=$PWD/bin:$PATH -
Run Inigo.
# cd back to the diego-release release repo cd diego-release/ # run the tests ./scripts/run-inigo
-
Download the latest Warden Trusty Go-Agent stemcell and upload it to BOSH-lite
bosh public stemcells bosh download public stemcell (name) bosh upload stemcell (downloaded filename) -
Checkout cf-release (develop branch) from git
cd ~/workspace git clone [email protected]:cloudfoundry/cf-release.git cd ~/workspace/cf-release git checkout develop ./update -
Checkout diego-release (develop branch) from git
cd ~/workspace git clone [email protected]:cloudfoundry-incubator/diego-release.git cd ~/workspace/diego-release git checkout develop ./scripts/update -
Install spiff, a tool for generating BOSH manifests. spiff is required for running the scripts in later steps. The following installation method assumes that go is installed. For other ways of installing
spiff, see the spiff README.go get github.com/cloudfoundry-incubator/spiff -
Generate a deployment stub with the BOSH director UUID
mkdir -p ~/deployments/bosh-lite cd ~/workspace/diego-release ./scripts/print-director-stub > ~/deployments/bosh-lite/director.yml -
Generate and target cf-release manifest:
cd ~/workspace/cf-release ./generate_deployment_manifest warden \ ~/deployments/bosh-lite/director.yml \ ~/workspace/diego-release/stubs-for-cf-release/enable_diego_docker_in_cc.yml \ ~/workspace/diego-release/stubs-for-cf-release/enable_consul_with_cf.yml \ > ~/deployments/bosh-lite/cf.yml bosh deployment ~/deployments/bosh-lite/cf.yml -
Do the BOSH dance:
cd ~/workspace/cf-release bosh create release --force bosh -n upload release bosh -n deploy -
Generate and target diego's manifest:
cd ~/workspace/diego-release ./scripts/generate-deployment-manifest \ ~/deployments/bosh-lite/director.yml \ manifest-generation/bosh-lite-stubs/property-overrides.yml \ manifest-generation/bosh-lite-stubs/instance-count-overrides.yml \ manifest-generation/bosh-lite-stubs/persistent-disk-overrides.yml \ manifest-generation/bosh-lite-stubs/iaas-settings.yml \ manifest-generation/bosh-lite-stubs/additional-jobs.yml \ ~/deployments/bosh-lite \ > ~/deployments/bosh-lite/diego.yml bosh deployment ~/deployments/bosh-lite/diego.yml -
Dance some more:
bosh create release --force bosh -n upload release bosh -n deploy
Now you can either run the DATs or deploy your own app.
### Running Smoke Tests & DATs
You can test that your diego-release deployment is working and integrating with cf-release
by running the lightweight diego-smoke-tests or the more thorough diego-acceptance-tests.
Follow the READMEs in their respective repositories:
src/github.com/cloudfoundry-incubator/diego-smoke-testssrc/github.com/cloudfoundry-incubator/diego-acceptance-tests
-
Create new CF Org & Space:
cf api --skip-ssl-validation api.10.244.0.34.xip.io cf auth admin admin cf create-org diego cf target -o diego cf create-space diego cf target -s diego -
Push your application without starting it:
cf push my-app --no-start -
Enable Diego for your application.
-
Start your application:
cf start my-app