diff --git a/.github/workflows/build_and_test_with_resty_events.yml b/.github/workflows/build_and_test_with_resty_events.yml index 2b375342..71434c69 100644 --- a/.github/workflows/build_and_test_with_resty_events.yml +++ b/.github/workflows/build_and_test_with_resty_events.yml @@ -135,4 +135,4 @@ jobs: run: | eval `luarocks path` eval $(perl -I $HOME/perl5/lib/perl5/ -Mlocal::lib) - TEST_NGINX_RANDOMIZE=1 prove -I. -r t/with_resty-events + TEST_NGINX_TIMEOUT=4 TEST_NGINX_RANDOMIZE=1 prove -I. -r t/with_resty-events diff --git a/.github/workflows/build_and_test_with_worker_events.yml b/.github/workflows/build_and_test_with_worker_events.yml index a78f314c..6129706e 100644 --- a/.github/workflows/build_and_test_with_worker_events.yml +++ b/.github/workflows/build_and_test_with_worker_events.yml @@ -8,7 +8,7 @@ jobs: runs-on: ubuntu-20.04 strategy: matrix: - openresty-version: [1.19.9.1, 1.21.4.1] + openresty-version: [1.21.4.1] steps: - name: Update and install OS dependencies @@ -61,4 +61,4 @@ jobs: run: | eval `luarocks path` eval $(perl -I $HOME/perl5/lib/perl5/ -Mlocal::lib) - TEST_NGINX_RANDOMIZE=1 prove -I. -r t/with_worker-events + TEST_NGINX_TIMEOUT=4 TEST_NGINX_RANDOMIZE=1 prove -I. -r t/with_worker-events diff --git a/docs/index.html b/docs/index.html index 52451055..39b82e55 100644 --- a/docs/index.html +++ b/docs/index.html @@ -53,23 +53,25 @@

Topics

Module resty.healthcheck

Healthcheck library for OpenResty.

-

Some notes on the usage of this library:

+

+ + +

Some notes on the usage of this library:

-

Info:

@@ -456,8 +459,8 @@

Healt
Report a health failure. Reports a health failure which will count against the number of occurrences - required to make a target “fall”. The type of healthchecker, - “tcp” or “http” (see new) determines against which counter the occurence goes. + required to make a target "fall". The type of healthchecker, + "tcp" or "http" (see new) determines against which counter the occurence goes. If unhealthy.tcp_failures (for TCP failures) or unhealthy.http_failures is set to zero in the configuration, this function is a no-op and returns true. @@ -475,7 +478,7 @@

Parameters:

(optional) hostname of the target being checked.
  • check - (optional) the type of check, either “passive” or “active”, default “passive”. + (optional) the type of check, either "passive" or "active", default "passive".
  • @@ -518,7 +521,7 @@

    Parameters:

    the http statuscode, or nil to report an invalid http response.
  • check - (optional) the type of check, either “passive” or “active”, default “passive”. + (optional) the type of check, either "passive" or "active", default "passive".
  • @@ -540,7 +543,7 @@

    Returns:

    Report a health success. Reports a health success which will count against the number of occurrences - required to make a target “rise”. + required to make a target "rise". If healthy.successes is set to zero in the configuration, this function is a no-op and returns true. @@ -557,7 +560,7 @@

    Parameters:

    (optional) hostname of the target being checked.
  • check - (optional) the type of check, either “passive” or “active”, default “passive”. + (optional) the type of check, either "passive" or "active", default "passive".
  • @@ -594,13 +597,13 @@

    Parameters:

  • operation The socket operation that failed: - “connect”, “send” or “receive”. + "connect", "send" or "receive". TODO check what kind of information we get from the OpenResty layer in order to tell these error conditions apart - https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/balancer.md#get_last_failure + https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/balancer.md#getlastfailure
  • check - (optional) the type of check, either “passive” or “active”, default “passive”. + (optional) the type of check, either "passive" or "active", default "passive".
  • @@ -636,7 +639,7 @@

    Parameters:

    (optional) hostname of the target being checked.
  • check - (optional) the type of check, either “passive” or “active”, default “passive”. + (optional) the type of check, either "passive" or "active", default "passive".
  • @@ -769,7 +772,7 @@

    Returns:

    It will be started upon creation.

    NOTE: the returned checker object must be anchored, if not it will be - removed by Lua’s garbage collector and the healthchecks will cease to run. + removed by Lua's garbage collector and the healthchecks will cease to run.

    Parameters:

    @@ -778,35 +781,34 @@

    Parameters:

    table with checker options. Options are:

    -

    If any of the health counters above (e.g. checks.passive.unhealthy.timeouts) is set to zero, the according category of checks is not taken to account. This way active or passive health checks can be disabled selectively. @@ -830,7 +832,7 @@

    Returns:

    generated by LDoc 1.5.0 -Last updated 2023-09-06 09:49:32 +Last updated 2023-12-27 15:06:12
    diff --git a/docs/topics/readme.md.html b/docs/topics/readme.md.html index 5863ee85..a21a310e 100644 --- a/docs/topics/readme.md.html +++ b/docs/topics/readme.md.html @@ -50,19 +50,20 @@

    Modules

    -

    lua-resty-healthcheck

    -

    legacy version -Release 1.6.x -License -Twitter Follow

    +

    lua-resty-healthcheck

    + +

    legacy version +Release 1.6.x +License +Twitter Follow

    A health check library for OpenResty.

    -

    Synopsis

    +
     http {
         lua_shared_dict test_shm 8m;
    @@ -125,7 +126,6 @@ 

    Synopsis

    -

    Description

    This library supports performing active and passive health checks on arbitrary hosts.

    @@ -134,7 +134,7 @@

    Description

    happens via the lua-resty-worker-events library.

    Targets are added using checker:add_target(host, port). -Changes in status (“healthy” or “unhealthy”) are broadcasted via worker-events.

    +Changes in status ("healthy" or "unhealthy") are broadcasted via worker-events.

    Active checks are executed in the background based on the specified timer intervals.

    @@ -145,238 +145,225 @@

    Description

    for the complete API.

    -

    History

    Versioning is strictly based on Semantic Versioning

    -

    1.6.2 (17-Nov-2022)

    +

    1.6.4 (22-Dec-2023)

      -
    • Fix: avoid raising worker events for new targets that were marked for delayed -removal, i.e. targets that already exist in memory only need the removal flag -cleared when added back. #122
    • +
    • Fix: fix delay clean logic when multiple healthchecker was started #146
    +

    1.6.3 (06-Sep-2023)

    -

    1.6.1 (25-Jul-2022)

    +
      +
    • Feature: Added support for https_sni #49 (backport)
    • +
    • Fix: Use OpenResty API for mTLS #99 (backport)
    • +
    + +

    1.6.2 (17-Nov-2022)

      -
    • Fix: improvements to ensure the proper securing of shared resources to avoid -race conditions and clearly report failure states. -#112, -#113, -#114.
    • -
    • Fix: reduce the frequency of checking for unused targets, reducing the number -of locks created. #116
    • -
    • Fix accept any lua-resty-events -0.1.x release. #118
    • +
    • Fix: avoid raising worker events for new targets that were marked for delayed + removal, i.e. targets that already exist in memory only need the removal flag + cleared when added back. #122
    +

    1.6.1 (25-Jul-2022)

    + +
      +
    • Fix: improvements to ensure the proper securing of shared resources to avoid + race conditions and clearly report failure states. + #112, + #113, + #114.
    • +
    • Fix: reduce the frequency of checking for unused targets, reducing the number + of locks created. #116
    • +
    • Fix accept any lua-resty-events + 0.1.x release. #118
    • +

    1.6.0 (27-Jun-2022)

      -
    • Feature: introduce support to lua-resty-events -module in addition to lua-resty-worker-events -support. With this addition, the lua-resty-healthcheck luarocks package does -not require a specific event-sharing module anymore, but you are still -required to provide either lua-resty-worker-events or lua-resty-events. -#105
    • -
    • Change: if available, lua-resty-healthcheck now uses string.buffer, the new LuaJIT’s -serialization API. If it is unavailable, lua-resty-healthcheck fallbacks to -cjson. #109
    • +
    • Feature: introduce support to lua-resty-events + module in addition to lua-resty-worker-events + support. With this addition, the lua-resty-healthcheck luarocks package does + not require a specific event-sharing module anymore, but you are still + required to provide either lua-resty-worker-events or lua-resty-events. + #105
    • +
    • Change: if available, lua-resty-healthcheck now uses string.buffer, the new LuaJIT's + serialization API. If it is unavailable, lua-resty-healthcheck fallbacks to + cjson. #109
    -

    1.5.1 (23-Mar-2022)

      -
    • Fix: avoid breaking active health checks when adding or removing targets. -#93
    • +
    • Fix: avoid breaking active health checks when adding or removing targets. + #93
    -

    1.5.0 (09-Feb-2022)

      -
    • New option checks.active.headers supports one or more lists of values indexed by -header name. #87
    • -
    • Introduce dealyed_clear() function, used to remove addresses after a time interval. -This function may be used when an address is being removed but may be added again -before the interval expires, keeping its health status. -#88
    • +
    • New option checks.active.headers supports one or more lists of values indexed by + header name. #87
    • +
    • Introduce dealyed_clear() function, used to remove addresses after a time interval. + This function may be used when an address is being removed but may be added again + before the interval expires, keeping its health status. + #88
    -

    1.4.3 (31-Mar-2022)

      -
    • Fix: avoid breaking active health checks when adding or removing targets. -#100
    • +
    • Fix: avoid breaking active health checks when adding or removing targets. + #100
    -

    1.4.2 (29-Jun-2021)

      -
    • Fix: prevent new active checks being scheduled while a health check is running. -#72
    • -
    • Fix: remove event watcher when stopping an active health check. -#74; fixes Kong issue -#7406
    • +
    • Fix: prevent new active checks being scheduled while a health check is running. + #72
    • +
    • Fix: remove event watcher when stopping an active health check. + #74; fixes Kong issue + #7406
    -

    1.4.1 (17-Feb-2021)

      -
    • Fix: make sure that a single worker will actively check hosts' statuses. -#67
    • +
    • Fix: make sure that a single worker will actively check hosts' statuses. + #67
    -

    1.4.0 (07-Jan-2021)

      -
    • Use a single timer to actively health check targets. This reduces the number -of timers used by health checkers, as they used to use two timers by each -target. #62
    • +
    • Use a single timer to actively health check targets. This reduces the number + of timers used by health checkers, as they used to use two timers by each + target. #62
    -

    1.3.0 (17-Jun-2020)

      -
    • Adds support to mTLS to active healthchecks. This feature can be used adding -the fields ssl_cert and ssl_key, with certificate and key respectively, -when creating a new healthcheck object. -#41
    • +
    • Adds support to mTLS to active healthchecks. This feature can be used adding + the fields ssl_cert and ssl_key, with certificate and key respectively, + when creating a new healthcheck object. + #41
    -

    1.2.0 (13-Feb-2020)

    -

    1.1.2 (19-Dec-2019)

      -
    • Fix: when ngx.sleep API is not available (e.g. in the log phase) it is not -possible to lock using lua-resty-lock and any function that needs exclusive -access would fail. This fix adds a retry method that starts a new light -thread, which has access to ngx.sleep, to lock the critical path. -#37;
    • +
    • Fix: when ngx.sleep API is not available (e.g. in the log phase) it is not + possible to lock using lua-resty-lock and any function that needs exclusive + access would fail. This fix adds a retry method that starts a new light + thread, which has access to ngx.sleep, to lock the critical path. + #37;
    -

    1.1.1 (14-Nov-2019)

      -
    • Fix: fail when it is not possible to get exclusive access to the list of -targets. This fix prevents that workers get to an inconsistent state. -#34;
    • +
    • Fix: fail when it is not possible to get exclusive access to the list of + targets. This fix prevents that workers get to an inconsistent state. + #34;
    -

    1.1.0 (30-Sep-2019)

      -
    • Add support for setting the custom Host header to be used for active checks.
    • -
    • Fix: log error on SSL Handshake failure -#28;
    • +
    • Add support for setting the custom Host header to be used for active checks.
    • +
    • Fix: log error on SSL Handshake failure + #28;
    -

    1.0.0 (05-Jul-2019)

      -
    • BREAKING: all API functions related to hosts require a hostname argument -now. This way different hostnames listening on the same IP and ports -combination do not have an effect on each other.
    • -
    • Fix: fix reporting active TCP probe successes -#20; -fixes issue #19
    • +
    • BREAKING: all API functions related to hosts require a hostname argument + now. This way different hostnames listening on the same IP and ports + combination do not have an effect on each other.
    • +
    • Fix: fix reporting active TCP probe successes + #20; + fixes issue #19
    -

    0.6.1 (04-Apr-2019)

      -
    • Fix: set up event callback only after target list is loaded -#18; -fixes Kong issue #4453
    • +
    • Fix: set up event callback only after target list is loaded + #18; + fixes Kong issue #4453
    -

    0.6.0 (26-Sep-2018)

      -
    • Introduce checks.active.https_verify_certificate field. -It is true by default; setting it to false disables certificate -verification in active healthchecks over HTTPS.
    • +
    • Introduce checks.active.https_verify_certificate field. + It is true by default; setting it to false disables certificate + verification in active healthchecks over HTTPS.
    -

    0.5.0 (25-Jul-2018)

      -
    • Add support for https — thanks @gaetanfl for the PR!
    • -
    • Introduce separate checks.active.type and checks.passive.type fields; -the top-level type field is still supported as a fallback but is now -deprecated.
    • +
    • Add support for https -- thanks @gaetanfl for the PR!
    • +
    • Introduce separate checks.active.type and checks.passive.type fields; + the top-level type field is still supported as a fallback but is now + deprecated.
    -

    0.4.2 (23-May-2018)

      -
    • Fix Host header in active healthchecks
    • +
    • Fix Host header in active healthchecks
    -

    0.4.1 (21-May-2018)

      -
    • Fix internal management of healthcheck counters
    • +
    • Fix internal management of healthcheck counters
    -

    0.4.0 (20-Mar-2018)

      -
    • Correct setting of defaults in http_statuses
    • -
    • Type and bounds checking to checks table
    • +
    • Correct setting of defaults in http_statuses
    • +
    • Type and bounds checking to checks table
    -

    0.3.0 (18-Dec-2017)

      -
    • Disable individual checks by setting their counters to 0
    • +
    • Disable individual checks by setting their counters to 0
    -

    0.2.0 (30-Nov-2017)

    -

    0.1.0 (27-Nov-2017) Initial release

      -
    • Initial upload
    • +
    • Initial upload
    -

    -

    Copyright and License

     Copyright 2017-2022 Kong Inc.
    @@ -395,11 +382,13 @@ 

    Copyright and License

    + +
    generated by LDoc 1.5.0 -Last updated 2023-09-06 09:49:32 +Last updated 2023-12-27 15:06:12
    diff --git a/lib/resty/healthcheck.lua b/lib/resty/healthcheck.lua index c5411366..4d05c9cf 100644 --- a/lib/resty/healthcheck.lua +++ b/lib/resty/healthcheck.lua @@ -1686,11 +1686,13 @@ function _M.new(opts) end local cur_time = ngx_now() + local is_checked = false for _, checker_obj in pairs(hcs) do if (last_cleanup_check + CLEANUP_INTERVAL) < cur_time then -- clear targets marked for delayed removal locking_target_list(checker_obj, function(target_list) + is_checked = true local removed_targets = {} local index = 1 while index <= #target_list do @@ -1717,8 +1719,6 @@ function _M.new(opts) end end end) - - last_cleanup_check = cur_time end if checker_obj.checks.active.healthy.active and @@ -1737,6 +1737,9 @@ function _M.new(opts) checker_callback(checker_obj, "unhealthy") end end + if is_checked then + last_cleanup_check = cur_time + end end, }) if not active_check_timer then diff --git a/readme.md b/readme.md index 78065a1e..cfb2be45 100644 --- a/readme.md +++ b/readme.md @@ -91,6 +91,10 @@ for the complete API. Versioning is strictly based on [Semantic Versioning](https://semver.org/) +### 1.6.4 (22-Dec-2023) + +* Fix: fix delay clean logic when multiple healthchecker was started [#146](https://github.com/Kong/lua-resty-healthcheck/pull/146) + ### 1.6.3 (06-Sep-2023) * Feature: Added support for https_sni [#49](https://github.com/Kong/lua-resty-healthcheck/pull/49) (backport) diff --git a/rockspecs/lua-resty-healthcheck-1.6.4-1.rockspec b/rockspecs/lua-resty-healthcheck-1.6.4-1.rockspec new file mode 100644 index 00000000..60ba817e --- /dev/null +++ b/rockspecs/lua-resty-healthcheck-1.6.4-1.rockspec @@ -0,0 +1,26 @@ +package = "lua-resty-healthcheck" +version = "1.6.4-1" +source = { + url = "git+https://github.com/Kong/lua-resty-healthcheck.git", + tag = "1.6.4" +} +description = { + summary = "Healthchecks for OpenResty to check upstream service status", + detailed = [[ + lua-resty-healthcheck is a module that can check upstream service + availability by sending requests and validating responses at timed + intervals. + ]], + license = "Apache 2.0", + homepage = "https://github.com/Kong/lua-resty-healthcheck" +} +dependencies = { + "penlight >= 1.9.2", + "lua-resty-timer ~> 1", +} +build = { + type = "builtin", + modules = { + ["resty.healthcheck"] = "lib/resty/healthcheck.lua", + } +} diff --git a/t/with_resty-events/11-clear.t b/t/with_resty-events/11-clear.t index 6fce9327..8bb2a5ab 100644 --- a/t/with_resty-events/11-clear.t +++ b/t/with_resty-events/11-clear.t @@ -3,7 +3,7 @@ use Cwd qw(cwd); workers(1); -plan tests => repeat_each() * 27; +plan tests => repeat_each() * 27 +2; my $pwd = cwd(); $ENV{TEST_NGINX_SERVROOT} = server_root(); @@ -296,3 +296,87 @@ false target not found false target not found + +=== TEST 6: delayed_clear() would clear tgt list when we add two checkers +--- http_config eval: $::HttpConfig +--- config + location = /t { + content_by_lua_block { + local healthcheck = require("resty.healthcheck") + local config1 = { + name = "testing", + shm_name = "test_shm", + events_module = "resty.events", + checks = { + active = { + healthy = { + interval = 0.1 + }, + unhealthy = { + interval = 0.1 + } + } + } + } + + local config2 = { + name = "testing2", + shm_name = "test_shm", + events_module = "resty.events", + checks = { + active = { + healthy = { + interval = 0.1 + }, + unhealthy = { + interval = 0.1 + } + } + } + } + + local checker1 = healthcheck.new(config1) + checker1:add_target("127.0.0.1", 10001, nil, true) + checker1:add_target("127.0.0.1", 10002, nil, true) + checker1:add_target("127.0.0.1", 10003, nil, true) + ngx.sleep(0.2) + ngx.say(checker1:get_target_status("127.0.0.1", 10002)) + checker1:delayed_clear(0.2) + + local checker2 = healthcheck.new(config2) + checker2:add_target("127.0.0.1", 10001, nil, true) + checker2:add_target("127.0.0.1", 10002, nil, true) + ngx.sleep(0.2) + ngx.say(checker2:get_target_status("127.0.0.1", 10002)) + checker2:delayed_clear(0.2) + + ngx.sleep(3) -- wait twice the interval + + local status, err = checker1:get_target_status("127.0.0.1", 10001) + if status ~= nil then + ngx.say(status) + else + ngx.say(err) + end + status, err = checker2:get_target_status("127.0.0.1", 10002) + if status ~= nil then + ngx.say(status) + else + ngx.say(err) + end + status, err = checker2:get_target_status("127.0.0.1", 10003) + if status ~= nil then + ngx.say(status) + else + ngx.say(err) + end + } + } +--- request +GET /t +--- response_body +true +true +target not found +target not found +target not found diff --git a/t/with_worker-events/11-clear.t b/t/with_worker-events/11-clear.t index 0ddb02d5..7056246f 100644 --- a/t/with_worker-events/11-clear.t +++ b/t/with_worker-events/11-clear.t @@ -3,7 +3,7 @@ use Cwd qw(cwd); workers(1); -plan tests => repeat_each() * 27; +plan tests => repeat_each() * 27 +2; my $pwd = cwd(); @@ -280,3 +280,86 @@ false target not found false target not found + + +=== TEST 6: delayed_clear() would clear tgt list when we add two checkers +--- http_config eval: $::HttpConfig +--- config + location = /t { + content_by_lua_block { + local we = require "resty.worker.events" + assert(we.configure{ shm = "my_worker_events", interval = 0.1 }) + local healthcheck = require("resty.healthcheck") + local config1 = { + name = "testing", + shm_name = "test_shm", + checks = { + active = { + healthy = { + interval = 0.1 + }, + unhealthy = { + interval = 0.1 + } + } + } + } + + local config2 = { + name = "testing2", + shm_name = "test_shm", + checks = { + active = { + healthy = { + interval = 0.1 + }, + unhealthy = { + interval = 0.1 + } + } + } + } + + local checker1 = healthcheck.new(config1) + checker1:add_target("127.0.0.1", 10001, nil, true) + checker1:add_target("127.0.0.1", 10002, nil, true) + checker1:add_target("127.0.0.1", 10003, nil, true) + ngx.say(checker1:get_target_status("127.0.0.1", 10002)) + checker1:delayed_clear(0.2) + + local checker2 = healthcheck.new(config2) + checker2:add_target("127.0.0.1", 10001, nil, true) + checker2:add_target("127.0.0.1", 10002, nil, true) + ngx.say(checker2:get_target_status("127.0.0.1", 10002)) + checker2:delayed_clear(0.2) + + ngx.sleep(3) -- wait twice the interval + + local status, err = checker1:get_target_status("127.0.0.1", 10001) + if status ~= nil then + ngx.say(status) + else + ngx.say(err) + end + status, err = checker2:get_target_status("127.0.0.1", 10002) + if status ~= nil then + ngx.say(status) + else + ngx.say(err) + end + status, err = checker2:get_target_status("127.0.0.1", 10003) + if status ~= nil then + ngx.say(status) + else + ngx.say(err) + end + } + } +--- request +GET /t +--- response_body +true +true +target not found +target not found +target not found