From bdd669dce89497f666af90d21dc518a029a688f4 Mon Sep 17 00:00:00 2001 From: "Mishra, Sakshi" Date: Tue, 18 Jun 2024 14:10:39 +0530 Subject: [PATCH 1/6] Updated the doc to import a subset of k8s logs Signed-off-by: Mishra, Sakshi --- charts/opentelemetry-collector/README.md | 25 +++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/charts/opentelemetry-collector/README.md b/charts/opentelemetry-collector/README.md index a085a8872..137afc2f0 100644 --- a/charts/opentelemetry-collector/README.md +++ b/charts/opentelemetry-collector/README.md @@ -104,6 +104,29 @@ presets: The way this feature works is it adds a `filelog` receiver on the `logs` pipeline. This receiver is preconfigured to read the files where Kubernetes container runtime writes all containers' console output to. +#### Log collection for a subset of pods or containers + +The `logsCollection` preset will by default ingest the logs of all kubernetes containers. +This is achieved by using an include path of `/var/log/pods/*/*/*.log` for the `filelog`receiver. + +To limit the import to a certain subset of pods or containers, the `filelog` +receivers `include` list can be overwritten by supplying explicit configuration. + +E.g. The following configuration would only import logs for pods within namespace `example-namespace`: + +```yaml +mode: daemonset + +presets: + logsCollection: + enabled: true +config: + receivers: + filelog: + include: + - /var/log/pods/example-namespace_*/*/*.log +``` + #### :warning: Warning: Risk of looping the exported logs back into the receiver, causing "log explosion" The container logs pipeline uses the `debug` exporter by default. @@ -113,7 +136,7 @@ it is easy to accidentally feed the exported logs back into the receiver. Also note that using the `--verbosity=detailed` option for the `debug` exporter causes it to output multiple lines per single received log, which when looped, would amplify the logs exponentially. -To prevent the looping, the default configuration of the receiver excludes logs from the collector's containers. +To prevent the looping, the default configuration of the receiver excludes logs from the collector's containers by setting `presets.logsCollection.includeCollectorLogs` to `false`. If you want to include the collector's logs, make sure to replace the `debug` exporter with an exporter that does not send logs to collector's standard output. From 829e7899ec12e0d182154cba09d333a0c8c5ea1b Mon Sep 17 00:00:00 2001 From: "Mishra, Sakshi" Date: Fri, 26 Jul 2024 19:30:29 +0530 Subject: [PATCH 2/6] Updated README.md as per external reviewer suggestions Signed-off-by: Mishra, Sakshi --- charts/opentelemetry-collector/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/charts/opentelemetry-collector/README.md b/charts/opentelemetry-collector/README.md index 137afc2f0..fe7060f17 100644 --- a/charts/opentelemetry-collector/README.md +++ b/charts/opentelemetry-collector/README.md @@ -136,7 +136,7 @@ it is easy to accidentally feed the exported logs back into the receiver. Also note that using the `--verbosity=detailed` option for the `debug` exporter causes it to output multiple lines per single received log, which when looped, would amplify the logs exponentially. -To prevent the looping, the default configuration of the receiver excludes logs from the collector's containers by setting `presets.logsCollection.includeCollectorLogs` to `false`. +To prevent the looping, the default configuration of the receiver excludes logs from the collector's containers. If you want to include the collector's logs, make sure to replace the `debug` exporter with an exporter that does not send logs to collector's standard output. From 5327f92041ff142102836bd56c3b51e1f1cccad4 Mon Sep 17 00:00:00 2001 From: "Mishra, Sakshi" Date: Tue, 10 Sep 2024 11:07:38 +0530 Subject: [PATCH 3/6] Bumped chart version based on lint test Signed-off-by: Mishra, Sakshi --- charts/opentelemetry-collector/Chart.yaml | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/charts/opentelemetry-collector/Chart.yaml b/charts/opentelemetry-collector/Chart.yaml index 333415f4a..ea403e75c 100644 --- a/charts/opentelemetry-collector/Chart.yaml +++ b/charts/opentelemetry-collector/Chart.yaml @@ -1,15 +1,15 @@ apiVersion: v2 name: opentelemetry-collector -version: 0.104.0 +version: 0.105.0 description: OpenTelemetry Collector Helm chart for Kubernetes type: application home: https://opentelemetry.io/ sources: - - https://github.com/open-telemetry/opentelemetry-collector - - https://github.com/open-telemetry/opentelemetry-collector-contrib +- https://github.com/open-telemetry/opentelemetry-collector +- https://github.com/open-telemetry/opentelemetry-collector-contrib maintainers: - - name: dmitryax - - name: jaronoff97 - - name: TylerHelmuth +- name: dmitryax +- name: jaronoff97 +- name: TylerHelmuth icon: https://opentelemetry.io/img/logos/opentelemetry-logo-nav.png appVersion: 0.108.0 From 41903dac3c6ec60c577254badac57b166cbfa286 Mon Sep 17 00:00:00 2001 From: "Mishra, Sakshi" Date: Tue, 10 Sep 2024 11:42:28 +0530 Subject: [PATCH 4/6] Removing the unwanted space Signed-off-by: Mishra, Sakshi --- charts/opentelemetry-collector/Chart.yaml | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/charts/opentelemetry-collector/Chart.yaml b/charts/opentelemetry-collector/Chart.yaml index ea403e75c..6af57c5bf 100644 --- a/charts/opentelemetry-collector/Chart.yaml +++ b/charts/opentelemetry-collector/Chart.yaml @@ -5,11 +5,11 @@ description: OpenTelemetry Collector Helm chart for Kubernetes type: application home: https://opentelemetry.io/ sources: -- https://github.com/open-telemetry/opentelemetry-collector -- https://github.com/open-telemetry/opentelemetry-collector-contrib + - https://github.com/open-telemetry/opentelemetry-collector + - https://github.com/open-telemetry/opentelemetry-collector-contrib maintainers: -- name: dmitryax -- name: jaronoff97 -- name: TylerHelmuth + - name: dmitryax + - name: jaronoff97 + - name: TylerHelmuth icon: https://opentelemetry.io/img/logos/opentelemetry-logo-nav.png appVersion: 0.108.0 From 945ace908bb69e833153dcdf044db7ef0b7f8fea Mon Sep 17 00:00:00 2001 From: "Mishra, Sakshi" Date: Tue, 10 Sep 2024 11:44:14 +0530 Subject: [PATCH 5/6] Removed unwanted space Signed-off-by: Mishra, Sakshi --- charts/opentelemetry-collector/Chart.yaml | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/charts/opentelemetry-collector/Chart.yaml b/charts/opentelemetry-collector/Chart.yaml index 6af57c5bf..392be3724 100644 --- a/charts/opentelemetry-collector/Chart.yaml +++ b/charts/opentelemetry-collector/Chart.yaml @@ -5,11 +5,11 @@ description: OpenTelemetry Collector Helm chart for Kubernetes type: application home: https://opentelemetry.io/ sources: - - https://github.com/open-telemetry/opentelemetry-collector - - https://github.com/open-telemetry/opentelemetry-collector-contrib + - https://github.com/open-telemetry/opentelemetry-collector + - https://github.com/open-telemetry/opentelemetry-collector-contrib maintainers: - - name: dmitryax - - name: jaronoff97 - - name: TylerHelmuth + - name: dmitryax + - name: jaronoff97 + - name: TylerHelmuth icon: https://opentelemetry.io/img/logos/opentelemetry-logo-nav.png appVersion: 0.108.0 From cb945af10887f0064daac6052dcdbedad113ed48 Mon Sep 17 00:00:00 2001 From: "Mishra, Sakshi" Date: Thu, 19 Sep 2024 12:53:31 +0530 Subject: [PATCH 6/6] Bumped chart version - "0.105.2" Signed-off-by: Mishra, Sakshi --- charts/opentelemetry-collector/Chart.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/charts/opentelemetry-collector/Chart.yaml b/charts/opentelemetry-collector/Chart.yaml index 392be3724..948fd91a3 100644 --- a/charts/opentelemetry-collector/Chart.yaml +++ b/charts/opentelemetry-collector/Chart.yaml @@ -1,6 +1,6 @@ apiVersion: v2 name: opentelemetry-collector -version: 0.105.0 +version: 0.105.2 description: OpenTelemetry Collector Helm chart for Kubernetes type: application home: https://opentelemetry.io/