java.lang.IllegalStateException: The service request was not made within 10 seconds of doBlockingWrite being invoked. Make sure to invoke the service request BEFORE invoking doBlockingWrite if your caller is single-threaded.ription)

[ramakrishna-g1]

Describe the bug

The service request was not made within 10 seconds of doBlockingWrite being invoked. Make sure to invoke the service request BEFORE invoking doBlockingWrite if your caller is single-threaded.
at software.amazon.awssdk.core.async.BlockingInputStreamAsyncRequestBody.waitForSubscriptionIfNeeded(BlockingInputStreamAsyncRequestBody.java:110) ~[sdk-core-2.22.2.jar!/:na]
at software.amazon.awssdk.core.async.BlockingInputStreamAsyncRequestBody.writeInputStream(BlockingInputStreamAsyncRequestBody.java:74) ~[sdk-core-2.22.2.jar!/:na]

Expected Behavior

We are experiencing these failures very often even after using latest aws crt client "aws-crt-client" 2.23.12.
We expect this to wait for longer time / should have option to increase the time out which would be helpful when there is huge data with large files.

Current Behavior

We are trying to steam large number of files from source system to AWS s3 using Transfer manager by reading the stream using HttpURLConnection, below is sample code -

URL targetURL = new URL("URL");
HttpURLConnection urlConnection = (HttpURLConnection) targetURL.openConnection();
urlConnection.setRequestMethod(HttpMethod.GET.toString());
urlConnection.setRequestProperty(HttpHeaders.ACCEPT, MediaType.ALL_VALUE);

if (urlConnection.getResponseCode() == HttpStatus.OK.value()) {
BlockingInputStreamAsyncRequestBody body = AsyncRequestBody.forBlockingInputStream(null);

Upload upload = transferManager.upload(builder -> builder
		.requestBody(body)
		.addTransferListener(UploadProcessListener.create(fileTracker.getPath()))
		.putObjectRequest(req -> req.bucket(s3BucketName).key("v3/" + s3Key + "/" + fileTracker.getPath()))
		.build());

    long totalBytes = body.writeInputStream(urlConnection.getInputStream());

}

Reproduction Steps

URL targetURL = new URL("URL");
HttpURLConnection urlConnection = (HttpURLConnection) targetURL.openConnection();
urlConnection.setRequestMethod(HttpMethod.GET.toString());
urlConnection.setRequestProperty(HttpHeaders.ACCEPT, MediaType.ALL_VALUE);

if (urlConnection.getResponseCode() == HttpStatus.OK.value()) {
BlockingInputStreamAsyncRequestBody body = AsyncRequestBody.forBlockingInputStream(null);

Upload upload = transferManager.upload(builder -> builder
		.requestBody(body)
		.addTransferListener(UploadProcessListener.create(fileTracker.getPath()))
		.putObjectRequest(req -> req.bucket(s3BucketName).key("v3/" + s3Key + "/" + fileTracker.getPath()))
		.build());

    long totalBytes = body.writeInputStream(urlConnection.getInputStream());

}

Possible Solution

No response

Additional Information/Context

No response

aws-crt-java version used

2.23.12

Java version used

11

Operating System and version

window / linux

[waahm7]

Hi, Thank you for creating the issue. Can you please create the issue at https://github.com/aws/aws-sdk-java-v2? This seems like an SDK Java issue instead of a crt-java specific issue. It would be easier to track and triage this issue in the SDK.

[ramakrishna-g1]

Hi, Thank you for creating the issue. Can you please create the issue at https://github.com/aws/aws-sdk-java-v2? This seems like an SDK Java issue instead of a crt-java specific issue. It would be easier to track and triage this issue in the SDK.

Thanks for your response, I have created another ticket under aws-sdk-java-v2 - aws/aws-sdk-java-v2#4893

[vswamy3]

I am also getting similar exceptions very often. even though the size of the file is very small. it is occurring only in the US region. The same code is running in other regions like EU and CN. but I have not encountered any issues.

[vswamy3]

any updates?

[TingDaoK]

Close this prefer to aws/aws-sdk-java-v2#4893