Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PLINT-469] Add Octopus Deploy integration #18488

Open
wants to merge 53 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
26c34b0
Add scaffolding
sarah-witt Aug 21, 2024
8237b43
add link to readme
sarah-witt Aug 30, 2024
87f6280
clean up manifest
sarah-witt Sep 3, 2024
152e2bd
validate ci
sarah-witt Sep 3, 2024
fcf2fdc
validate models and config;
sarah-witt Sep 3, 2024
f77c3d7
add labeler
sarah-witt Sep 3, 2024
fc9e61c
remove dash
sarah-witt Sep 3, 2024
23a6260
fix manifest
sarah-witt Sep 3, 2024
f720cc9
clean scaffolding and add base fixtures
sarah-witt Sep 3, 2024
c802b28
add base code and test
sarah-witt Sep 3, 2024
29c0191
fix changelog
sarah-witt Sep 3, 2024
a77d728
update and validate config
sarah-witt Sep 3, 2024
bbb52af
add metric to check
sarah-witt Sep 3, 2024
c180bef
Add mocks and make true unit tests
sarah-witt Sep 3, 2024
3c2c9fa
Add support for first level space autodiscovery
sarah-witt Sep 4, 2024
a7bf9b9
Add support fot project groups discovery
sarah-witt Sep 9, 2024
1ba6632
Add support for projects
sarah-witt Sep 13, 2024
bfc6d01
remove spaces discovery and add it as configuration option
sarah-witt Sep 13, 2024
7cd3ef3
clean up code and tests
sarah-witt Sep 13, 2024
f716ef6
move initializations to check init
sarah-witt Sep 17, 2024
4f007c3
add support for task count metrics
sarah-witt Sep 27, 2024
03393b3
refactor to reference project_group
sarah-witt Sep 30, 2024
d22f5fa
submit project metrics in check method
sarah-witt Sep 30, 2024
299f5ee
Merge branch 'master' into sarah/add-octopus-integration
sarah-witt Sep 30, 2024
37e6752
Add handle_error wrapper and add more logging
sarah-witt Sep 30, 2024
c957319
merge
sarah-witt Sep 30, 2024
90990f9
bump python version
sarah-witt Sep 30, 2024
c659018
Add e2e test
sarah-witt Sep 30, 2024
21e8ff8
lint
sarah-witt Sep 30, 2024
83ab4ae
add http template to config spec
sarah-witt Oct 1, 2024
416fb69
refactor test files and add caddy e2e
sarah-witt Oct 1, 2024
14e851a
lint
sarah-witt Oct 1, 2024
67c6303
fix tests
sarah-witt Oct 1, 2024
1e58c47
Add deployment duration metric
sarah-witt Oct 2, 2024
3b74fdd
rename to seconds
sarah-witt Oct 2, 2024
27b2c5e
Add queue time metric
sarah-witt Oct 2, 2024
0209824
Add deploy success metric
sarah-witt Oct 7, 2024
e1abac8
Add can rerun metric
sarah-witt Oct 7, 2024
49e6135
add metric for has warnings or errors
sarah-witt Oct 7, 2024
d84cad9
sort readme
sarah-witt Oct 7, 2024
18b63b7
Merge branch 'master' into sarah/add-octopus-integration
sarah-witt Oct 7, 2024
a88f90a
Sarah/add octopus server metrics (#18868)
sarah-witt Oct 17, 2024
17cd086
remove dead code
sarah-witt Oct 17, 2024
6a88501
update codeowners
sarah-witt Oct 17, 2024
75396e8
add e2e lab and update config
sarah-witt Oct 21, 2024
7a3d2bd
Address comments
sarah-witt Oct 21, 2024
179597c
Update projects logic and refactor (#19087)
sarah-witt Nov 22, 2024
e3d4d48
Make assertions more specific
sarah-witt Dec 2, 2024
b66b970
Add support for events (#19176)
sarah-witt Dec 2, 2024
197843e
[PLINT-484] Add support for deployment logs (#19152)
sarah-witt Dec 3, 2024
ea78cf3
[PLINT-469] Add support for pagination in octopus integration (#19228)
sarah-witt Dec 9, 2024
e0312a0
update config docs (#19243)
sarah-witt Dec 10, 2024
02785fd
[PLINT-543] Add support for environment metrics and tagging by enviro…
sarah-witt Dec 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .codecov.yml
Original file line number Diff line number Diff line change
Expand Up @@ -442,6 +442,10 @@ coverage:
target: 75
flags:
- nvidia_triton
Octopus_Deploy:
target: 75
flags:
- octopus_deploy
OpenLDAP:
target: 75
flags:
Expand Down Expand Up @@ -1290,6 +1294,11 @@ flags:
paths:
- nvidia_triton/datadog_checks/nvidia_triton
- nvidia_triton/tests
octopus_deploy:
carryforward: true
paths:
- octopus_deploy/datadog_checks/octopus_deploy
- octopus_deploy/tests
openldap:
carryforward: true
paths:
Expand Down
3 changes: 3 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,9 @@ manifest.json @DataDog/documentation @DataDog/agent-integrations
/openstack_controller/ @DataDog/platform-integrations @DataDog/agent-integrations
/openstack_controller/manifest.json @DataDog/platform-integrations @DataDog/agent-integrations @DataDog/documentation
/openstack_controller/*.md @DataDog/platform-integrations @DataDog/agent-integrations @DataDog/documentation
/octopus_deploy/ @DataDog/platform-integrations @DataDog/agent-integrations
/octopus_deploy/manifest.json @DataDog/platform-integrations @DataDog/agent-integrations @DataDog/documentation
/octopus_deploy/*.md @DataDog/platform-integrations @DataDog/agent-integrations @DataDog/documentation
/teleport/ @DataDog/platform-integrations @DataDog/agent-integrations
/teleport/manifest.json @DataDog/platform-integrations @DataDog/agent-integrations @DataDog/documentation
/teleport/*.md @DataDog/platform-integrations @DataDog/agent-integrations @DataDog/documentation
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/config/labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -361,6 +361,8 @@ integration/nvidia_jetson:
- nvidia_jetson/**/*
integration/nvidia_triton:
- nvidia_triton/**/*
integration/octopus_deploy:
- octopus_deploy/**/*
integration/oke:
- oke/**/*
integration/oom_kill:
Expand Down
20 changes: 20 additions & 0 deletions .github/workflows/test-all.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2674,6 +2674,26 @@ jobs:
minimum-base-package: ${{ inputs.minimum-base-package }}
pytest-args: ${{ inputs.pytest-args }}
secrets: inherit
jc2a0c60:
uses: ./.github/workflows/test-target.yml
with:
job-name: Octopus Deploy
target: octopus_deploy
platform: linux
runner: '["ubuntu-22.04"]'
repo: "${{ inputs.repo }}"
python-version: "${{ inputs.python-version }}"
standard: ${{ inputs.standard }}
latest: ${{ inputs.latest }}
agent-image: "${{ inputs.agent-image }}"
agent-image-py2: "${{ inputs.agent-image-py2 }}"
agent-image-windows: "${{ inputs.agent-image-windows }}"
agent-image-windows-py2: "${{ inputs.agent-image-windows-py2 }}"
test-py2: ${{ inputs.test-py2 }}
test-py3: ${{ inputs.test-py3 }}
minimum-base-package: ${{ inputs.minimum-base-package }}
pytest-args: ${{ inputs.pytest-args }}
secrets: inherit
j2cf0a0a:
uses: ./.github/workflows/test-target.yml
with:
Expand Down
4 changes: 4 additions & 0 deletions octopus_deploy/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# CHANGELOG - Octopus Deploy

<!-- towncrier release notes start -->

53 changes: 53 additions & 0 deletions octopus_deploy/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Agent Check: Octopus Deploy

## Overview

This check monitors your [Octopus Deploy][1] deployments through the Datadog Agent. Track information such as average deployment time per Environment, and deployment failure rate for a Project.

## Setup

Follow the instructions below to install and configure this check for an Agent running on a host. For containerized environments, see the [Autodiscovery Integration Templates][3] for guidance on applying these instructions.

### Installation

The Octopus Deploy check is included in the [Datadog Agent][2] package.
No additional installation is needed on your server.

### Configuration

1. Edit the `octopus_deploy.d/conf.yaml` file in the `conf.d/` folder at the root of your Agent's configuration directory to start collecting your `octopus_deploy` performance data. See the [sample `octopus_deploy.d/conf.yaml`][4] for all available configuration options.

2. [Restart the Agent][5].

### Validation

[Run the Agent's status subcommand][6] and look for `octopus_deploy` under the Checks section.

## Data Collected

### Metrics

See [metadata.csv][7] for a list of metrics provided by this integration.

### Events

The Octopus Deploy integration does not include any events.

### Service Checks

The Octopus Deploy integration does not include any service checks.

## Troubleshooting

Need help? Contact [Datadog support][9].


[1]: https://octopus.com/
[2]: https://app.datadoghq.com/account/settings/agent/latest
[3]: https://docs.datadoghq.com/agent/kubernetes/integrations/
[4]: https://github.com/DataDog/integrations-core/blob/master/octopus_deploy/datadog_checks/octopus_deploy/data/conf.yaml.example
[5]: https://docs.datadoghq.com/agent/guide/agent-commands/#start-stop-and-restart-the-agent
[6]: https://docs.datadoghq.com/agent/guide/agent-commands/#agent-status-and-information
[7]: https://github.com/DataDog/integrations-core/blob/master/octopus_deploy/metadata.csv
[8]: https://github.com/DataDog/integrations-core/blob/master/octopus_deploy/assets/service_checks.json
[9]: https://docs.datadoghq.com/help/
265 changes: 265 additions & 0 deletions octopus_deploy/assets/configuration/spec.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,265 @@
name: Octopus Deploy
files:
- name: octopus_deploy.yaml
options:
- template: init_config
options:
- template: init_config/default
- template: init_config/http
- template: instances
options:
- name: octopus_endpoint
display_priority: 8
description: |
Octopus API endpoint.
See https://octopus.com/docs/octopus-rest-api/getting-started#authentication for more details.
value:
example: http://localhost:80/api
type: string
required: true
- name: spaces
display_priority: 5
description: |
Optional configuration to indicate the spaces that we want to be processed. If not configured,
all spaces and their corresponding project groups and projects will be processed.

The 'include' key will indicate the regular expressions of the spaces for which metrics are to be reported
and the configuration to be applied to each of them. Each group may have a 'project-groups'-like configuration,
enabling or disabling metric collection for the project groups which match that condition. For further details
see next section 'project-groups'. If no configuration associated with the key is indicated with the regular
expression, they will be processed with the default configuration.

The spaces will be processed in the order indicated in the 'include'.
If a space is matched on an 'include' key, it will only be processed there and not in a later 'include'
that it might match on.

The 'exclude' key will indicate the regular expressions of those spaces for which metrics
are not to be reported.
The excludes will have priority over the includes, that is, if a space matches an exclude, it will not be
processed even if it matches an include. The 'include' key must be used if using the 'exclude' key.

The 'limit' key will allow limiting the number of spaces processed to avoid a combinatorial explosion of tags
associated with a metric.

The 'interval' key will indicate the validity time of the last list of spaces obtained through the endpoint.
If 'interval' is not indicated, the list of spaces will be obtained each time the check is executed
and will not be cached.

In the following example, only the space named "default" will be collected. Additionally, only the project groups
starting with "test" in that space will be collected. All other project groups and spaces will be ignored.
Furthermore, the cache will be valid for 1 minute.

spaces:
limit: 3
include:
- 'default':
project_groups:
- limit: 5
include:
- 'test.*'
interval: 60
interval: 60
value:
type: object
properties:
- name: limit
description: |
Maximum number of spaces to be processed.
type: integer
- name: include
type: array
items:
anyOf:
- type: string
- type: object
- name: exclude
type: array
items:
type: string
- name: interval
type: integer
example: {}
- name: project_groups
display_priority: 5
description: |
Optional configuration to indicate the project groups that we want to be processed. If not configured,
all project groups will be processed.

The 'include' key will indicate the regular expressions of the project groups for which metrics are to be
reported and the configuration to be applied to each of them. Each group may have a 'projects'-like
configuration, enabling or disabling metric collection for the projects which match that condition. For
further details see previous section 'projects'. If no configuration associated with the key is indicated with
the regular expression, they will be processed with the default configuration.

The project groups will be processed in the order indicated in the 'include'.
If a project group is matched on an 'include' key, it will only be processed there and not in a later 'include'
that it might match on.

The 'exclude' key will indicate the regular expressions of those project groups for which metrics
are not to be reported.
The excludes will have priority over the includes, that is, if a project group matches an exclude, it will not be
processed even if it matches an include. The 'include' key must be used if using the 'exclude' key.

The 'limit' key will allow limiting the number of project groups processed to avoid a combinatorial explosion of
tags associated with a metric.

The 'interval' key will indicate the validity time of the last list of project groups obtained through the
endpoint. If 'interval' is not indicated, the list of project groups will be obtained each time the check is
executed and will not be cached.

In the following example, all project groups will be processed except those whose name begins with 'tmp_'
up to a maximum of 10 project groups.
Furthermore, the cache will be valid for 1 minute.

project_groups:
limit: 10
include:
- '.*'
exclude:
- 'tmp_.*'
interval: 60
value:
type: object
properties:
- name: limit
description: |
Maximum number of project groups to be processed.
type: integer
- name: include
type: array
items:
anyOf:
- type: string
- type: object
- name: exclude
type: array
items:
type: string
- name: interval
type: integer
example: {}
- name: projects
display_priority: 5
description: |
Optional configuration to indicate the projects that we want to be processed. If not configured,
all projects will be processed.

The 'include' key will indicate the regular expressions of the projects for which metrics are to be reported.

The projects will be processed in the order indicated in the 'include'.
If a projects is matched on an 'include' key, it will only be processed there and not in a later 'include'
that it might match on.

The 'exclude' key will indicate the regular expressions of those projects for which metrics
are not to be reported.
The excludes will have priority over the includes, that is, if a projects matches an exclude, it will not be
processed even if it matches an include. The 'include' key must be used if using the 'exclude' key.

The 'limit' key will allow limiting the number of projects processed to avoid a combinatorial explosion of tags
associated with a metric.

The 'interval' key will indicate the validity time of the last list of projects obtained through the endpoint.
If 'interval' is not indicated, the list of projects will be obtained each time the check is executed
and will not be cached.

In the following example, only the project named 'my-project' will be collected.

projects:
include:
- 'my-project'
value:
type: object
properties:
- name: limit
description: |
Maximum number of projects to be processed.
type: integer
- name: include
type: array
items:
anyOf:
- type: string
- type: object
- name: exclude
type: array
items:
type: string
- name: interval
type: integer
example: {}
- name: environments
display_priority: 6
description: |
Optional configuration to indicate the environments that we want to be processed. If not configured,
all environments will be processed.

The 'include' key will indicate the regular expressions of the environments for which metrics are
to be reported.

The environments will be processed in the order indicated in the 'include'.
If an environment is matched on an 'include' key, it will only be processed there and not in a later 'include'
that it might match on.

The 'exclude' key will indicate the regular expressions of those environments for which metrics
are not to be reported.
The excludes will have priority over the includes, that is, if an environment matches an exclude, it will not be
processed even if it matches an include. The 'include' key must be used if using the 'exclude' key.

The 'limit' key will allow limiting the number of environments processed to avoid a combinatorial explosion of
tags associated with a metric.

The 'interval' key will indicate the validity time of the last list of environments obtained through the
endpoint. If 'interval' is not indicated, the list of environments will be obtained each time the check
is executed and will not be cached.

In the following example, only the environment named 'prod' will be collected.

environments:
include:
- 'prod'

value:
type: object
properties:
- name: limit
description: |
Maximum number of environments to be processed.
type: integer
- name: include
type: array
items:
anyOf:
- type: string
- type: object
- name: exclude
type: array
items:
type: string
- name: interval
type: integer
example: {}
- name: paginated_limit
description: |
Sets the number of items API calls should return at a time. Default is 30.
value:
example: 30
type: integer
required: false
- template: instances/default
- template: instances/http
overrides:
headers.display_priority: 6
headers.enabled: true
headers.description: |
Headers to use for every request. An Authorization header including the Octopus Deploy API key token is required
for authentication for the REST API.
You can alternatively use the `auth_token` option.

headers.value.example:
X-Octopus-ApiKey: "<OCTOPUS_API_KEY>"
auth_token.display_priority: 4
- template: logs
example:
- type: integration
source: octopus-deploy
service: <SERVICE_NAME>
1 change: 1 addition & 0 deletions octopus_deploy/assets/service_checks.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[]
1 change: 1 addition & 0 deletions octopus_deploy/changelog.d/18488.added
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Initial Release
Loading
Loading