sunbird cloud-storage-sdk should support s3 compliant api calls #349
ddevadat
started this conversation in
Contribution
Replies: 1 comment
-
@ddevadat - The implementation to support OCI cloud storage is available with the below PR. The artefact is pushed to Maven central staging environment for the testing. Please consider using the below repository link and dependency to integrate and test it with the Flink jobs and microservices. Repository URL: https://oss.sonatype.org/content/repositories/orgsunbird-1144/ <dependency>
<groupId>org.sunbird</groupId>
<artifactId>cloud-store-sdk_2.12</artifactId>
<version>1.4.6</version>
</dependency> Note: The changes are available with Scala 2.12. We make the required changes to support Scala 2.11 and share the details. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
sunbird should support cloud service provider which has s3 compliant object storage.
sunbird code cloud-storage-sdk package from maven central in many of its build job. cloud-storage-sdk doesnt support s3 compatible api calls.
If we take the latest version for reference
Under its hood, it uses apache jclouds ( 2.2.1 ) to connect to object storage. It also uses guava version 19.0
The problem is inorder to use apache jclouds to support s3 compliant api calls, the minimum version has to be 2.5.0. Else it wont support v4 signing
Refer:
https://jclouds.apache.org/releasenotes/2.5.0/
https://issues.apache.org/jira/browse/JCLOUDS-1594
To use jclouds ( 2.5.0 ), we need to upgrade guava version from 19.0.
I tried 31.1-jre, 27.1-jre, both works, but it breaks other build jobs
e.g
Build/KnowledgePlatform/Learning uses cassandra-driver-core version 3.1.2 which doesnt work with guava version > 19
This is just one of the example that i encountered.
A workaround i followed was to build cloud-store-sdk with jcloud-all: 2.5.0 , guava: 20.0, then change the version of cassandra-driver-core:3.2.0
Whats is the most efficient way to handle this. This is a major show stopper for us.
Beta Was this translation helpful? Give feedback.
All reactions