-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add timeout to channel builder #710
base: development
Are you sure you want to change the base?
Conversation
PR Review ChecklistDo not edit the content of this comment. The PR reviewer should simply update this comment by ticking each review item below, as they get completed. Trivial Change
Code
Architecture
|
43379ba
to
b879753
Compare
@@ -49,8 +52,13 @@ impl GRPCChannel for PlainTextChannel {} | |||
|
|||
impl GRPCChannel for CallCredChannel {} | |||
|
|||
const TIMEOUT: Duration = Duration::from_secs(60); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Chosen arbitrarily. Anything less than a second or so is too strict for slow connections, but going significantly over a minute would probably be too long.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's verify that this timeout doesn't drop query requests with complicated, long-running queries. For example, write a simple read query, put a sleep on the server side for 2 minutes, and let it run. This should not fail. But if you put your server sleep on connection, it should fail after 1 minute.
If this works, everything should be fine.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The query does time out if the server does not respond within this time (tested by adding a sleep(60000)
in TransactionService::respond()
on the server side).
The default transaction timeout is 5 minutes. I'd extend this timeout to 10 minutes or even an hour.
Ideally we'd have a connection builder that would let the user specify the desired request timeout, but that would take a lot longer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm afraid that we'll just destroy some of the usages of analytics queries with this update. I'd suggest giving the query at least 2 hours to execute. And, ideally, highlight this behavior somewhere in the docs.
Later, it will be ideal if this timeout is configurable. Let's create a good task for it if we're going to close the original bug.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's test queries
@@ -49,8 +52,13 @@ impl GRPCChannel for PlainTextChannel {} | |||
|
|||
impl GRPCChannel for CallCredChannel {} | |||
|
|||
const TIMEOUT: Duration = Duration::from_secs(60); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's verify that this timeout doesn't drop query requests with complicated, long-running queries. For example, write a simple read query, put a sleep on the server side for 2 minutes, and let it run. This should not fail. But if you put your server sleep on connection, it should fail after 1 minute.
If this works, everything should be fine.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Waiting for the timeout value update
Usage and product changes
We introduce a timeout in the gRPC channel so that the driver correctly fails when the server is available but unresponsive.
Resolves #672.
Implementation