Skip to content

Commit

Permalink
Fix #163 by only synchronizing on the probe suspended state, not the
Browse files Browse the repository at this point in the history
entire RPC.

Signed-off-by: Chris Larsen <[email protected]>
  • Loading branch information
manolama committed Jun 9, 2017
1 parent 6ab74f7 commit 3fe5133
Show file tree
Hide file tree
Showing 2 changed files with 17 additions and 19 deletions.
32 changes: 15 additions & 17 deletions src/HBaseClient.java
Original file line number Diff line number Diff line change
Expand Up @@ -2802,26 +2802,24 @@ else if ((exists_rpc = nsred_rpcs.get(0)) != rpc) {

// Stop here if this is a known NSRE and `rpc' is not our probe RPC that
// is not suspended
synchronized (exists_rpc) {
if (known_nsre && exists_rpc != rpc && !exists_rpc.isSuspendedProbe()) {
if (size != nsre_high_watermark && size % NSRE_LOG_EVERY == 0) {
final String msg = "There are now " + size
+ " RPCs pending due to NSRE on " + Bytes.pretty(region_name);
if (size + NSRE_LOG_EVERY < nsre_high_watermark) {
LOG.info(msg); // First message logged at INFO level.
} else {
LOG.warn(msg); // Last message logged with increased severity.
}
}
if (reject) {
rpc.callback(new PleaseThrottleException(size + " RPCs waiting on "
+ Bytes.pretty(region_name) + " to come back online", e, rpc,
exists_rpc.getDeferred()));
if (known_nsre && exists_rpc != rpc && !exists_rpc.isSuspendedProbe()) {
if (size != nsre_high_watermark && size % NSRE_LOG_EVERY == 0) {
final String msg = "There are now " + size
+ " RPCs pending due to NSRE on " + Bytes.pretty(region_name);
if (size + NSRE_LOG_EVERY < nsre_high_watermark) {
LOG.info(msg); // First message logged at INFO level.
} else {
LOG.warn(msg); // Last message logged with increased severity.
}
return; // This NSRE is already known and being handled.
}
exists_rpc.setSuspendedProbe(false);
if (reject) {
rpc.callback(new PleaseThrottleException(size + " RPCs waiting on "
+ Bytes.pretty(region_name) + " to come back online", e, rpc,
exists_rpc.getDeferred()));
}
return; // This NSRE is already known and being handled.
}
exists_rpc.setSuspendedProbe(false);
}

num_nsres.increment();
Expand Down
4 changes: 2 additions & 2 deletions src/HBaseRpc.java
Original file line number Diff line number Diff line change
Expand Up @@ -496,11 +496,11 @@ public void setProbe(boolean probe) {
*/
private boolean suspended_probe = false;

boolean isSuspendedProbe() {
synchronized boolean isSuspendedProbe() {
return suspended_probe;
}

void setSuspendedProbe(boolean suspended_probe) {
synchronized void setSuspendedProbe(boolean suspended_probe) {
this.suspended_probe = suspended_probe;
}

Expand Down

1 comment on commit 3fe5133

@jake-maloney
Copy link

@jake-maloney jake-maloney commented on 3fe5133 Jun 9, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't this still deadlock? Moving the synchronized onto the method still attempts to lock on the object instance which will lead to the same behavior.

If you make the fields volatile you will have some thread safety.

Please sign in to comment.