This repository has been archived by the owner on Feb 15, 2022. It is now read-only.
TST: e2e with trace group propagation #480
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* main: (112 commits) Bump junit in /data-prepper-plugins/otel-trace-group-prepper Bump aws-java-sdk-core in /data-prepper-plugins/elasticsearch Upgraded OpenTelemetry dependencies. Minor update to scaling docs. Bump guava in /data-prepper-plugins/otel-trace-raw-prepper Bump aws-java-sdk-core in /data-prepper-plugins/elasticsearch Limit overall cache size MAINT: remove unused wrapper class MAINT: remove wrapper class MAINT: variable name MAINT: constant variable names Update guava version FIX: integTest MAINT: enhance e2e FIX: wrong import Untrack MapDBTraceIdTraceGroupCache MAINT: use guava cache for traceId to traceGroup map FIX: timeunit RMV: unused APIs TODO ...
wrijeff
reviewed
Apr 5, 2021
...er-core/src/integrationTest/java/com/amazon/dataprepper/integration/EndToEndRawSpanTest.java
Outdated
Show resolved
Hide resolved
...er-core/src/integrationTest/java/com/amazon/dataprepper/integration/EndToEndRawSpanTest.java
Outdated
Show resolved
Hide resolved
...er-core/src/integrationTest/java/com/amazon/dataprepper/integration/EndToEndRawSpanTest.java
Outdated
Show resolved
Hide resolved
...er-core/src/integrationTest/java/com/amazon/dataprepper/integration/EndToEndRawSpanTest.java
Outdated
Show resolved
Hide resolved
...er-core/src/integrationTest/java/com/amazon/dataprepper/integration/EndToEndRawSpanTest.java
Outdated
Show resolved
Hide resolved
...er-core/src/integrationTest/java/com/amazon/dataprepper/integration/EndToEndRawSpanTest.java
Outdated
Show resolved
Hide resolved
...epper-core/src/integrationTest/java/com/amazon/dataprepper/integration/EndToEndTestData.java
Outdated
Show resolved
Hide resolved
wrijeff
approved these changes
Apr 8, 2021
| @@ -154,6 +164,28 @@ public static ExportTraceServiceRequest getExportTraceServiceRequest(List<Resour | |||
| .build(); | |||
| } | |||
|
|
|||
| private List<ResourceSpans> getResourceSpansBatch(final List<EndToEndTestSpan> dataList) { | |||
There was a problem hiding this comment.
nit - Now that the class has been renamed, would suggest something like testSpanList. Also applies to the for loop below.
Comment on lines
+1
to
+7
| entry-pipeline: | ||
| source: | ||
| otel_trace_source: | ||
| ssl: false | ||
| sink: | ||
| - pipeline: | ||
| name: "raw-pipeline" |
There was a problem hiding this comment.
Comment for later - eventually we might consider integ tests per component/feature instead of adding more to the existing. Calling this out because at first glance this pipeline config might look "incomplete" (no peer forwarder), but that is by design to test the traceGroupPrepper.
End state would be tests for:
- Basic test- Single DP
- Peer Forwarder plugin test - Two DP with forwarding enabled
- TraceGroup plugin test - Two DP without forwarding
etc.
Will be easier to isolate breaking changes based on the specific test failing. I'll create a backlog item if we think this is a good idea.
There was a problem hiding this comment.
Sounds good to me. Might even add one with Amazon Elasticseach Service backend.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
Description of changes:
This PR
(1) refactors integrationTest.gradle tasks
(2) separate pipeline configuration for raw-span and service-map e2e tests
(3) tests trace group propagation by replacing random span generation in raw-span e2e test with explicitly constructed trace-group dataset.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.