Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[extension/encoding] support skywalkingencoding extension #36968

Merged
merged 45 commits into from
Jan 16, 2025

Conversation

JaredTan95
Copy link
Member

@JaredTan95 JaredTan95 commented Dec 27, 2024

Description

The kafkareceiver supports jaeger and zipkin protocol traces, and can also receive skywalkng traces via skywalkingreceiver.

From the practical use of skywalking, we often use kafka to host large-scale traces data. However, kafka receiver does not support skywalking encoding currently.

with this PR, we can fectch skywalking traces and parse it into otlp traces:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
  kafka:
    encoding: skywalking_encoding
    topic: skywalking-segments
    brokers:
      - "localhost:9092"
    resolve_canonical_bootstrap_servers_only: true
    client_id: otel-collector
    group_id: otel-collector
    metadata:
      retry:
        max: 10
        backoff: 5s

exporters:
  debug:

processors:
  batch:

extensions:
  health_check:
  pprof:
    endpoint: :1888
  zpages:
    endpoint: :55679
  skywalking_encoding:

service:
  extensions: [pprof, zpages, skywalking_encoding, health_check]
  pipelines:
    traces:
      receivers: [kafka]
      processors: [batch]
      exporters: [debug]

Link to tracking issue

Fixes #28693

Testing

Documentation

@JaredTan95 JaredTan95 changed the title [extension/encoding] suuport Skywalkingencoding extension [extension/encoding] support skywalkingencoding extension Dec 27, 2024
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
@JaredTan95 JaredTan95 force-pushed the skywalkingencoding_extension branch from 8d1b0e5 to b01b7a1 Compare December 27, 2024 06:28
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
.github/CODEOWNERS Outdated Show resolved Hide resolved
@crobert-1
Copy link
Member

Do you know how to fix the check-owners failed lint?

logs show that I need to do some changes:

- extension/encoding/                               @open-telemetry/collector-contrib-approvers @atoulme @dao-jun @dmitryax @MovieStoreGuy @VihasMakwana
+ extension/encoding/                               @open-telemetry/collector-contrib-approvers @atoulme @dao-jun @dmitryax @MovieStoreGuy @VihasMakwana @JaredTan95 

in .github/CODEOWNERS file, but @atoulme told me do it in separate PR, so, I'm confused~

This is the current action failure output:

Generated code is out of date, please apply this diff and commit the changes in this PR.
diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
index 004df01..3baf60[9](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/12665959423/job/35296498228?pr=36968#step:5:10) 100644
--- a/.github/CODEOWNERS
+++ b/.github/CODEOWNERS
@@ -95,6 +95,7 @@ extension/encoding/avrologencodingextension/      @open-telemetry/collector-cont
 extension/encoding/jaegerencodingextension/       @open-telemetry/collector-contrib-approvers @MovieStoreGuy @atoulme
 extension/encoding/jsonlogencodingextension/      @open-telemetry/collector-contrib-approvers @VihasMakwana @atoulme
 extension/encoding/otlpencodingextension/         @open-telemetry/collector-contrib-approvers @dao-jun @VihasMakwana
+extension/encoding/skywalkingencodingextension/   @open-telemetry/collector-contrib-approvers @JaredTan95
 extension/encoding/textencodingextension/         @open-telemetry/collector-contrib-approvers @MovieStoreGuy @atoulme
 extension/encoding/zipkinencodingextension/       @open-telemetry/collector-contrib-approvers @MovieStoreGuy @dao-jun
 extension/googleclientauthextension/              @open-telemetry/collector-contrib-approvers @dashpole @aabmass @jsuereth @punya @psx95
Error: Process completed with exit code 1.

It's simply missing a line for the new extension you're adding, it's no longer complaining about the extension/encoding entry 👍

@JaredTan95 JaredTan95 added the ready to merge Code review completed; ready to merge by maintainers label Jan 10, 2025
Copy link

codecov bot commented Jan 13, 2025

Codecov Report

Attention: Patch coverage is 73.07692% with 7 lines in your changes missing coverage. Please review.

Project coverage is 79.59%. Comparing base (12551d3) to head (0849205).

Files with missing lines Patch % Lines
.../encoding/skywalkingencodingextension/extension.go 33.33% 4 Missing ⚠️
...encoding/skywalkingencodingextension/skywalking.go 57.14% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #36968      +/-   ##
==========================================
- Coverage   79.59%   79.59%   -0.01%     
==========================================
  Files        2252     2255       +3     
  Lines      211914   211940      +26     
==========================================
+ Hits       168682   168699      +17     
- Misses      37562    37571       +9     
  Partials     5670     5670              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@songy23 songy23 merged commit 6f06d5e into open-telemetry:main Jan 16, 2025
164 checks passed
@github-actions github-actions bot added this to the next release milestone Jan 16, 2025
@JaredTan95 JaredTan95 deleted the skywalkingencoding_extension branch January 17, 2025 01:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cmd/otelcontribcol otelcontribcol command extension/encoding ready to merge Code review completed; ready to merge by maintainers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[receiver/kafka] support receive skywalking traces from kafka
9 participants