Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: etcd watch restart when receive invalid revision #11833

Merged
merged 4 commits into from
Dec 18, 2024

Conversation

Revolyssup
Copy link
Contributor

@Revolyssup Revolyssup commented Dec 16, 2024

Description

 [error] 56#56: *223892756 lua user thread aborted: runtime error: /usr/local/apisix/apisix/core/config_etcd.lua:260: attempt to compare number with nil
stack traceback:
coroutine 0:
    /usr/local/apisix/apisix/core/config_etcd.lua: in function </usr/local/apisix/apisix/core/config_etcd.lua:130>
coroutine 1:
    [C]: in function 'ngx_thread_wait'
    /usr/local/apisix/apisix/core/config_etcd.lua:285: in function </usr/local/apisix/apisix/core/config_etcd.lua:268>, context: ngx.timer
 [error] 56#56: *223892756 lua entry thread aborted: runtime error: /usr/local/apisix/apisix/core/config_etcd.lua:287: attempt to concatenate local 'err' (a boolean value)
stack traceback:
coroutine 0:
    /usr/local/apisix/apisix/core/config_etcd.lua: in function </usr/local/apisix/apisix/core/config_etcd.lua:268>, context: ngx.timer

When the revision contained in the response returned by etcd watch API is empty, it will cause the watch thread to abort. Since the thread exits abnormally, the error object return from ngx.thread.wait is either nil or a boolean value (nil can be reproduced, but boolean cannot, I don't know why yet), causing subsequent log printing to abort too and making it impossible to restart the watch thread.

Fixes # (issue)

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

@dosubot dosubot bot added size:XS This PR changes 0-9 lines, ignoring generated files. bug Something isn't working labels Dec 16, 2024
@@ -284,7 +289,8 @@ local function run_watch(premature)

local ok, err = ngx_thread_wait(run_watch_th, check_worker_th)
if not ok then
log.error("check_worker thread terminates failed, retart checker, error: " .. err)
log.error("run_watch or check_worker thread terminates failed"
.. " restart those threads, error: ".. inspect(err))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code style: bad indentation

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

log.error("run_watch or check_worker thread terminates failed",
                " restart those threads, error: ", inspect(err))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@Revolyssup Revolyssup requested a review from membphis December 17, 2024 20:45
@Revolyssup Revolyssup merged commit 844b396 into apache:master Dec 18, 2024
33 checks passed
@Revolyssup Revolyssup deleted the revolyssup/etcd-watch-fix branch December 18, 2024 08:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working size:XS This PR changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants