Bump iglu-scala-client to 4.0.0 #1360
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
spenes
force-pushed
the
iglu-client-3.2.0
branch
4 times, most recently
from
4d86b43
to
31cab3e
Compare
spenes
changed the title
spenes
force-pushed
the
iglu-client-3.2.0
branch
5 times, most recently
from
6de4e2e
to
6ee66d3
Compare
oguzhanunlu
force-pushed
the
develop
branch
from
cd79f64
to
875ebbd
Compare
spenes
force-pushed
the
iglu-client-3.2.0
branch
2 times, most recently
from
6915ddd
to
b588aaa
Compare
spenes
force-pushed
the
iglu-client-3.2.0
branch
from
b588aaa
to
ca3260c
Compare
Iglu Scala Client has new lookupSchemasUntil function that allows to fetch list of schemas until given schema key. If we replace listSchemasLike function with lookupSchemasUntil function, RDB Loader won't rely on the list endpoint of Iglu Server. This commit makes necessary changes to use new lookupSchemasUntil function instead of listSchemasLike function.
Some of the tests are failing since windows don't have expected timestamps. In order to solve this problem, this commit makes necessary changes to read window timestamps from shredded message instead of using hard-coded values.
spenes
force-pushed
the
iglu-client-3.2.0
branch
from
ca3260c
to
b2dbafb
Compare
After starting to use lookupSchemasUntil in fetchSchemasWithSameModel, we are only getting schemas until the given schema key for every schema key. Previously, we were getting all the schemas for the same schema model. This change caused change of behavior when a message contains multiple schema keys for same schema model. When this happens, RDB Loader tries to create same table multiple times. In order to solve this problem, this commit contains the change for creating the migration for only max schema key of the same schema model.
spenes
force-pushed
the
iglu-client-3.2.0
branch
from
9e87b47
to
23c1feb
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Jira ref: PDP-1359
This PR makes necessary changes to use new
lookupSchemasUntilinstead oflistSchemasLike. With this change,listIglu endpoint won't be needed by RDB Loader.Also, some of the Stream Transformer tests were failing since windows didn't have expected timestamps. In order to solve this problem, this PR makes necessary changes to read window timestamps from shredded message instead of using hard-coded values.