Queries the Docker Engine API for running containers. If you are using Kubernetes, you should use the k8s-api observer instead of this.
Requires Docker API version 1.22+.
Note that you will need permissions to access the Docker engine API. For a Docker domain socket URL, this means that the agent needs to have read permissions on the socket. We don't currently support authentication for HTTP URLs.
You can configure monitors by putting special labels on your Docker containers. You can either specify all of the configuration in container labels, or you can use the more traditional agent configuration with discovery rules and specify configuration overrides with labels.
The config labels are of the form agent.signalfx.com.config.<port number>.<config_key>: <config value>
. The <config value>
must be a
string in a container label, but it will be deserialized as a YAML value to
the most appropriate type when consumed by the agent. For example, if you
have a Redis container and want to monitor it at a higher frequency than
other Redis containers, you could have an agent config that looks like the
following:
observers:
- type: docker
monitors:
- type: collectd/redis
discoveryRule: container_image =~ "redis" && port == 6379
auth: mypassword
intervalSeconds: 10
And then launch the Redis container with the label:
agent.signalfx.com.config.6379.intervalSeconds
: 1
This would cause the config value for intervalSeconds
to be overwritten to
the more frequent 1 second interval.
You can also specify the monitor configuration entirely with Docker labels and completely omit monitor config from the agent config. With the agent config:
observers:
- type: docker
You can then launch a Redis container with the following labels:
agent.signalfx.com.monitorType.6379
:collectd/redis
agent.signalfx.com.config.6379.auth
:mypassword
Which would configure a Redis monitor with the given authentication configuration. No Redis configuration is required in the agent config file.
The distinction is that the monitorType
label was added to the Docker
container. If a monitorType
label is present, no discovery rules will
be considered for this endpoint, and thus, no agent configuration can be
used anyway.
If you want to configure multiple monitors per port, you can specify the
port name in the form <port number>-<port name>
instead of just the port
number. For example, if you had two different Prometheus exporters running
on the same port, but on different paths in a given container, you could
provide labels like the following:
- `agent.signalfx.com.monitorType.8080-app`: `prometheus-exporter`
- `agent.signalfx.com.config.8080-app.metricPath`: `/appMetrics`
- `agent.signalfx.com.monitorType.8080-goruntime`: `prometheus-exporter`
- `agent.signalfx.com.config.8080-goruntime.metricPath`: `/goMetrics`
The name that is given to the port will populate the name
field of the
discovered endpoint and can be used in discovery rules as such. For
example, with the following agent config:
observers:
- type: docker
monitors:
- type: prometheus-exporter
discoveryRule: name == "app" && port == 8080
intervalSeconds: 1
And given docker labels as follows (remember that discovery rules are
irrelevant to endpoints that specify monitorType
labels):
agent.signalfx.com.config.8080-app.metricPath
:/appMetrics
agent.signalfx.com.config.8080-goruntime.metricPath
:/goMetrics
Would result in the app
endpoint getting an interval of 1 second and the
goruntime
endpoint getting the default interval of the agent.
Observer Type: docker
Config option | Required | Type | Description |
---|---|---|---|
dockerURL |
no | string |
(default: unix:///var/run/docker.sock ) |
cacheSyncInterval |
no | int64 |
The time to wait before resyncing the list of containers the monitor maintains through the docker event listener example: cacheSyncInterval: "20m" (default: 60m ) |
labelsToDimensions |
no | map of strings |
A mapping of container label names to dimension names that will get applied to the metrics of all discovered services. The corresponding label values will become the dimension values for the mapped name. E.g. io.kubernetes.container.name: container_spec_name would result in a dimension called container_spec_name that has the value of the io.kubernetes.container.name container label. |
useHostnameIfPresent |
no | bool |
If true, the "Config.Hostname" field (if present) of the docker container will be used as the discovered host that is used to configure monitors. If false or if no hostname is configured, the field NetworkSettings.IPAddress is used instead. (default: false ) |
useHostBindings |
no | bool |
If true, the observer will configure monitors for matching container endpoints using the host bound ip and port. This is useful if containers exist that are not accessible to an instance of the agent running outside of the docker network stack. (default: false ) |
ignoreNonHostBindings |
no | bool |
If true, the observer will ignore discovered container endpoints that are not bound to host ports. This is useful if containers exist that are not accessible to an instance of the agent running outside of the docker network stack. (default: false ) |
The following fields are available on targets generated by this observer and can be used in discovery rules.
Name | Type | Description |
---|---|---|
container_name |
string |
The first and primary name of the container as it is known to the container runtime (e.g. Docker). |
has_port |
string |
Set to true if the endpoint has a port assigned to it. This will be false for endpoints that represent a host/container as a whole. |
ip_address |
string |
The IP address of the endpoint if the host is in the from of an IPv4 address |
network_port |
string |
An alias for port |
private_port |
string |
The port that the service endpoint runs on inside the container |
public_port |
string |
The port exposed outside the container |
alternate_port |
integer |
Used for services that are accessed through some kind of NAT redirection as Docker does. This could be either the public port or the private one. |
container_command |
string |
The command used when running the container exposing the endpoint |
container_id |
string |
The ID of the container exposing the endpoint |
container_image |
string |
The image name of the container exposing the endpoint |
container_labels |
map of string |
A map that contains container label key/value pairs. You can use the Contains and Get helper functions in discovery rules to make use of this. See Endpoint Discovery. For containers managed by Kubernetes, this will be set to the pod's labels, as individual containers do not have labels in Kubernetes proper. |
container_names |
list of string |
A list of container names of the container exposing the endpoint |
container_state |
string |
The container state, will usually be "running" since otherwise the container wouldn't have a port exposed to be discovered. |
discovered_by |
string |
The observer that discovered this endpoint |
host |
string |
The hostname/IP address of the endpoint. If this is an IPv6 address, it will be surrounded by [ and ] . |
id |
string |
|
name |
string |
A observer assigned name of the endpoint. For example, if using the k8s-api observer, name will be the port name in the pod spec, if any. |
orchestrator |
integer |
|
port |
integer |
The TCP/UDP port number of the endpoint |
port_labels |
map of string |
A map of labels on the container port |
port_type |
string |
TCP or UDP |
target |
string |
The type of the thing that this endpoint directly refers to. If the endpoint has a host and port associated with it (most common), the value will be hostport . Other possible values are: pod , container , host . See the docs for the specific observer you are using for more details on what types that observer emits. |
These dimensions are added to all metrics that are emitted for this discovery target. These variables are also available to use as variables in discovery rules.
Name | Description |
---|---|
container_id |
The container id of the container running this endpoint. |
container_image |
The image name (including tags) of the running container |
container_name |
The primary name of the running container -- Docker containers can have multiple names but this will be the first name, if any. |