⚡ ✨ Cache regular match results to improve performance.

CHANGES-ZH-CN.md

@@ -4,6 +4,9 @@
 
 ### 新增
 
+* 新增了模式 `CACHE`，启用此模式后会缓存每次检查的结果，提高性能。
+* 新增了配置项 `waf_cache_size` 用于设置缓存检查结果的内存的大小。
+
 ### 移除
 
 ### 变动

CHANGES.md

@@ -4,6 +4,9 @@
 
 ### Added
 
+* A new mode `CACHE` has been added, enabling this mode will cache the results of each inspection to improve performance.
+* New configuration `waf_cache_size` has been added to set the size of the memory for caching inspection results.
+
 ### Removed
 
 ### Changed

README-ZH-CN.md

@@ -7,7 +7,6 @@
 [![docker](https://github.com/ADD-SP/ngx_waf/actions/workflows/docker.yml/badge.svg)](https://github.com/ADD-SP/ngx_waf/actions/workflows/docker.yml)
 [![Codacy Badge](https://app.codacy.com/project/badge/Grade/aebcf93b4b7a4b4b800ceb962479ee3a?branch=master)](https://www.codacy.com/gh/ADD-SP/ngx_waf/dashboard?utm_source=github.com&utm_medium=referral&utm_content=ADD-SP/ngx_waf&utm_campaign=Badge_Grade)
 [![GitHub release (latest by date including pre-releases)](https://img.shields.io/github/v/release/ADD-SP/ngx_waf?include_prereleases)](https://github.com/ADD-SP/ngx_waf/releases)
-![GitHub](https://img.shields.io/github/license/ADD-SP/ngx_waf?color=blue)
 [![语义化版本 2.0.0](https://img.shields.io/badge/%E8%AF%AD%E4%B9%89%E5%8C%96%E7%89%88%E6%9C%AC-2.0.0-blue)](https://semver.org/lang/zh-CN/)
 
 [English](README.md) | 简体中文
@@ -16,15 +15,15 @@
 
 ## 功能
 
-+ 支持 IPV4 和 IPV6。
-+ CC 防御，超出限制后自动拉黑对应 IP 一段时间。
-+ IP 黑白名单，同时支持类似 `192.168.0.0/16` 和 `fe80::/10`，即支持点分十进制和冒号十六进制表示法和网段划分。
-+ POST 黑名单。
-+ URL 黑白名单
-+ GET 参数黑名单
-+ UserAgent 黑名单。
-+ Cookie 黑名单。
-+ Referer 黑白名单。
+* 支持 IPV4 和 IPV6。
+* CC 防御，超出限制后自动拉黑对应 IP 一段时间。
+* IP 黑白名单，同时支持类似 `192.168.0.0/16` 和 `fe80::/10`，即支持点分十进制和冒号十六进制表示法和网段划分。
+* POST 黑名单。
+* URL 黑白名单
+* GET 参数黑名单
+* UserAgent 黑名单。
+* Cookie 黑名单。
+* Referer 黑白名单。
 
 ## 使用文档
 
@@ -38,6 +37,7 @@
 
 ## 感谢
 
-+ [ngx_lua_waf](https://github.com/loveshell/ngx_lua_waf): 本模块的默认规则大多来自于此。
-+ [nginx-book](https://github.com/taobao/nginx-book): 感谢作者提供的教程。
-+ [nginx-development-guide](https://github.com/baishancloud/nginx-development-guide): 感谢作者提供的教程。
+* [uthash](https://github.com/troydhanson/uthash): 本项目使用 uthash 的两个数据结构，即 `uthash` 和 `utlist`。
+* [ngx_lua_waf](https://github.com/loveshell/ngx_lua_waf): 本模块的默认规则大多来自于此。
+* [nginx-book](https://github.com/taobao/nginx-book): 感谢作者提供的教程。
+* [nginx-development-guide](https://github.com/baishancloud/nginx-development-guide): 感谢作者提供的教程。

README.md

@@ -7,7 +7,6 @@
 [![docker](https://github.com/ADD-SP/ngx_waf/actions/workflows/docker.yml/badge.svg)](https://github.com/ADD-SP/ngx_waf/actions/workflows/docker.yml)
 [![Codacy Badge](https://app.codacy.com/project/badge/Grade/aebcf93b4b7a4b4b800ceb962479ee3a?branch=master)](https://www.codacy.com/gh/ADD-SP/ngx_waf/dashboard?utm_source=github.com&utm_medium=referral&utm_content=ADD-SP/ngx_waf&utm_campaign=Badge_Grade)
 [![GitHub release (latest by date including pre-releases)](https://img.shields.io/github/v/release/ADD-SP/ngx_waf?include_prereleases)](https://github.com/ADD-SP/ngx_waf/releases)
-![GitHub](https://img.shields.io/github/license/ADD-SP/ngx_waf?color=blue)
 [![Semantic Versioning 2.0.0](https://img.shields.io/badge/Semantic%20Versioning-2.0.0-blue)](https://semver.org/)
 
 English | [简体中文](README-ZH-CN.md)
@@ -16,18 +15,18 @@ A web application firewall module for nginx without complex configuration.
 
 ## Function
 
-+ IPV4 and IPV6 support.
-+ Anti Challenge Collapsar, it can automatically block malicious IP.
-+ Exceptional allow on specific IP address.
-+ Block the specified IP address.
-+ Block the specified request body.
-+ Exceptional allow on specific URL.
-+ Block the specified URL.
-+ Block the specified request args.
-+ Block the specified UserAgent.
-+ Block the specified Cookie.
-+ Exceptional allow on specific Referer.
-+ Block the specified Referer.
+* IPV4 and IPV6 support.
+* Anti Challenge Collapsar, it can automatically block malicious IP.
+* Exceptional allow on specific IP address.
+* Block the specified IP address.
+* Block the specified request body.
+* Exceptional allow on specific URL.
+* Block the specified URL.
+* Block the specified request args.
+* Block the specified UserAgent.
+* Block the specified Cookie.
+* Exceptional allow on specific Referer.
+* Block the specified Referer.
 
 ## Docs
 
@@ -41,6 +40,7 @@ A web application firewall module for nginx without complex configuration.
 
 ## Thanks
 
-+ [ngx_lua_waf](https://github.com/loveshell/ngx_lua_waf): Most of the default rules of this module come from this.
-+ [nginx-book](https://github.com/taobao/nginx-book): Thanks for the tutorial provided by the author.
-+ [nginx-development-guide](https://github.com/baishancloud/nginx-development-guide): Thanks for the tutorial provided by the author.
+* [uthash](https://github.com/troydhanson/uthash): This project uses two data structures, `uthash` and `utlist`.
+* [ngx_lua_waf](https://github.com/loveshell/ngx_lua_waf): Most of the default rules of this module come from this.
+* [nginx-book](https://github.com/taobao/nginx-book): Thanks for the tutorial provided by the author.
+* [nginx-development-guide](https://github.com/baishancloud/nginx-development-guide): Thanks for the tutorial provided by the author.

config

@@ -7,7 +7,9 @@ deps="$ngx_addon_dir/inc/ngx_http_waf_module_check.h \
     $ngx_addon_dir/inc/ngx_http_waf_module_type.h \
     $ngx_addon_dir/inc/ngx_http_waf_module_util.h \
     $ngx_addon_dir/inc/ngx_http_waf_module_ip_trie.h \
-    $ngx_addon_dir/inc/ngx_http_waf_module_token_bucket_set.h"
+    $ngx_addon_dir/inc/ngx_http_waf_module_token_bucket_set.h \
+    $ngx_addon_dir/inc/ngx_http_waf_module_mem_pool.h \
+    $ngx_addon_dir/inc/ngx_http_waf_module_lru_cache.h"
 
 srcs="$ngx_addon_dir/src/ngx_http_waf_module_core.c"
 

docker/Dockerfile.alpine

@@ -50,6 +50,7 @@ RUN set -xe \
         --with-perl_modules_path=/usr/lib/perl5/vendor_perl \
         --user=nginx \
         --group=nginx \
+        --with-debug \
         --with-compat \
         --with-file-aio \
         --with-threads \

docker/Dockerfile.debian

@@ -53,6 +53,7 @@ RUN set -xe \
         --with-perl_modules_path=/usr/lib/perl5/vendor_perl \
         --user=nginx \
         --group=nginx \
+        --with-debug \
         --with-compat \
         --with-file-aio \
         --with-threads \

docs/.vuepress/config.js

@@ -29,7 +29,7 @@ module.exports = {
                 sidebar: [
                     {
                         title: "Quick Start",
-                        path: "/guide/",
+                        path: "/guide/overview.html",
                         children: [
                             "/guide/overview.md",
                             "/guide/version.md",
@@ -42,7 +42,7 @@ module.exports = {
                     },
                     {
                         title: "Advanced Guide",
-                        path: "/advance/",
+                        path: "/advance/syntax.html",
                         children: [
                             "/advance/syntax.md",
                             "/advance/rule.md",
@@ -63,7 +63,7 @@ module.exports = {
                 sidebar: [
                     {
                         title: "快速上手",
-                        path: "/zh-cn/guide/",
+                        path: "/zh-cn/guide/overview.html",
                         children: [
                             "/zh-cn/guide/overview.md",
                             "/zh-cn/guide/version.md",
@@ -76,7 +76,7 @@ module.exports = {
                     },
                     {
                         title: "进阶指南",
-                        path: "/zh-cn/advance/",
+                        path: "/zh-cn/advance/syntax.html",
                         children: [
                             "/zh-cn/advance/syntax.md",
                             "/zh-cn/advance/rule.md",

docs/README.md

@@ -3,13 +3,13 @@ home: true
 heroText: ngx_waf
 tagline: A web application firewall module for nginx without complex configuration.
 actionText: Quick Start →
-actionLink: /guide/
+actionLink: /guide/overview.html
 features:
 - title: Complete
   details: Includes the basic functions of an application firewall, such as URL inspection, POST inspection, and CC defense.
 - title: Easy
   details: Configuration files and rules files are simple to write.
-- title: Performance
-  details: Fast IP inspection.
+- title: Fast
+  details: The IP detection is a constant-time operation. Most of the remaining inspections use caching to improve performance.
 footer: BSD 3-Clause License | Copyright © 2020, ADD-SP
 ---

docs/advance/changes.md

@@ -9,6 +9,9 @@ lang: en
 
 ### Added
 
+* A new mode `CACHE` has been added, enabling this mode will cache the results of each inspection to improve performance.
+* New configuration `waf_cache_size` has been added to set the size of the memory for caching inspection results.
+
 ### Removed
 
 ### Changed

docs/advance/log.md

@@ -14,7 +14,7 @@ The format is `ngx_waf: [rule type][specific rule triggered]`.
 You can use the following command to quickly view the blocking log.
 
 ```sh
-cat /path/to/error.log | grep ngx_waf
+cat /path/to/error.log | grep ngx_waf:
 ```
 
 Here are two examples.
@@ -34,7 +34,7 @@ for troubleshooting purposes. The format is `ngx_waf_debug: debug information`.
 You can use the following command to quickly view the debug log.
 
 ```sh
-cat /path/to/error.log | grep ngx_waf_debug
+cat /path/to/error.log | grep ngx_waf_debug:
 ```
 
 Below is a typical modulation log that illustrates the flow of a CC defense detection.
@@ -75,7 +75,7 @@ http {
                         '  request: "$request"\n'
                         '  status: "$status"\n'
                         '  body_bytes_sent: "$body_bytes_sent"\n'
-                        '  http_referer: "http_referer"\n'
+                        '  http_referer: "$http_referer"\n'
                         '  http_user_agent: "$http_user_agent"\n'
                         '  http_x_forwarded_for: "$http_x_forwarded_for"\n'
                         '  waf_blocked: $waf_blocked\n'
@@ -112,7 +112,7 @@ The following is a log in YAML format.
   request: "GET /www.bak HTTP/1.1"
   status: "403"
   body_bytes_sent: "555"
-  http_referer: "http_referer"
+  http_referer: "localhost"
   http_user_agent: "Mozilla/4.0 (Windows; MSIE 6.0; Windows NT 5.2)"
   http_x_forwarded_for: "-"
   waf_blocked: true
@@ -125,7 +125,7 @@ The following is a log in YAML format.
   request: "GET / HTTP/1.1"
   status: "304"
   body_bytes_sent: "0"
-  http_referer: "http_referer"
+  http_referer: "localhost"
   http_user_agent: "Mozilla/4.0 (Windows; MSIE 6.0; Windows NT 5.2)"
   http_x_forwarded_for: "-"
   waf_blocked: false
@@ -138,7 +138,7 @@ The following is a log in YAML format.
   request: "GET / HTTP/1.1"
   status: "304"
   body_bytes_sent: "0"
-  http_referer: "http_referer"
+  http_referer: "localhost"
   http_user_agent: "Mozilla/4.0 (Windows; MSIE 6.0; Windows NT 5.2)"
   http_x_forwarded_for: "-"
   waf_blocked: false
@@ -151,7 +151,7 @@ The following is a log in YAML format.
   request: "GET / HTTP/1.1"
   status: "503"
   body_bytes_sent: "599"
-  http_referer: "http_referer"
+  http_referer: "localhost"
   http_user_agent: "Mozilla/4.0 (Windows; MSIE 6.0; Windows NT 5.2)"
   http_x_forwarded_for: "-"
   waf_blocked: true
@@ -164,7 +164,7 @@ The following is a log in YAML format.
   request: "GET / HTTP/1.1"
   status: "503"
   body_bytes_sent: "599"
-  http_referer: "http_referer"
+  http_referer: "localhost"
   http_user_agent: "Mozilla/4.0 (Windows; MSIE 6.0; Windows NT 5.2)"
   http_x_forwarded_for: "-"
   waf_blocked: true

docs/advance/syntax.md

@@ -47,18 +47,18 @@ Specify the working mode of the firewall, specifying at least one mode and up to
 * TRAC: Start the inspection process when `Http.Method=TRAC`.
 * IP: Enable IP address inspecting rules.
 * URL: Enable URL inspecting rules.
-* RBODY: Enable request body inspecting rules.
+* RBODY: Enable POST request body inspecting rules.
 * ARGS: Enable ARGS inspecting rules.
 * UA: Enable UA inspecting rules.
 * COOKIE: Enable COOKIE inspecting rules.
 * REFERER: Enable REFERER inspecting rules.
-* CC: Enable 'Anti Challenge Collapsar'.
+* CC: Enable 'Anti Challenge Collapsar'. When you enable this mode, you must set [waf_cc_deny_limit](#waf-cc-deny-limit).
 * COMPAT: compatibility mode, used to enable compatibility options with other modules or environments, currently used for compatibility with the ngx_http_rewrite_module, see [compatibility statement](/guide/compatibility.md).
 * STRICT: Strict mode, which sacrifices some performance for more checks, currently only works when `COMPAT` mode is enabled, and performs a full round of inspections before and after the ngx_http_rewrite_module takes effect.
 * STATIC: working mode for static sites, equivalent to `HEAD GET IP URL UA CC`.
-* DYNAMIC: working mode for dynamic sites, equivalent to `HEAD GET POST IP URL ARGS UA RB COOKIE CC`.
-* STD: Equivalent to `IP URL RB ARGS UA HEAD GET POST CC COMPAT`.
-* FULL: In any case, the inspection process will be started and all inspection rules will be enabled.
+* DYNAMIC: working mode for dynamic sites, equivalent to `HEAD GET POST IP URL ARGS UA RBODY COOKIE CC`.
+* STD: Equivalent to `HEAD GET POST IP URL RBODY ARGS UA CC COMPAT`.
+* FULL: Enable all modes.
 
 You can turn off a mode by prefixing a `mode_type` with `! ` prefix to a `mode_type` to turn it off. 
 The following is an example of using the standard working mode, but without inspecting the User-Agent.
@@ -73,11 +73,25 @@ The mode of `CC` is independent of other modes, and whether it takes effect or n
 
 :::
 
+::: tip CHANGES IN THE DEVELOPMENT VERSION
+
+Added a new mode:
+
+* CACHE: Enable caching. Enabling this mode will cache the result of the inspection, so that the next time the same target is inspected, there is no need to repeat the inspection. However, the results of the POST body inspection are not cached. For example, if a URL is not in the blacklist after inspection, the next time the same URL is inspected, the cache can be read directly. When you enable this mode, you must set [waf_cache_size](#waf-cache-size).
+
+The following modes have changed:
+
+* STD: Standard working mode, equivalent to `HEAD GET POST IP URL RBODY ARGS UA CC COMPAT CACHE`.
+* STATIC: working mode for static sites, equivalent to `HEAD GET IP URL UA CC CACHE`.
+* DYNAMIC: working mode for dynamic sites, equivalent to `HEAD GET POST IP URL ARGS UA RBODY COOKIE CC COMPAT CACHE`.
+
+:::
+
 
 ## `waf_cc_deny_limit`
 
 * syntax: `waf_cc_deny_limit <rate> <duration> [buffer_size]`;
-* default: `waf_cc_deny_limit 10000000 1 10m;`
+* default: ——
 * context: server
 
 Set the parameters related to CC protection.
@@ -87,3 +101,34 @@ Set the parameters related to CC protection.
 * `buffer_size`: used to set the size of the memory for recording IP accesses, such as `10m`, `10240k`, must not be less than `10m`, if not specified then the default is `10m`.
 
 
+## `waf_cache_size`
+
+* syntax: `waf_cache_size buffer_size`;
+* default: ——
+* context: server
+
+Set the size of the memory used to cache the inspection results. 
+For example `10m`, `10240k`, must not be less than `10m`, or default to `10m` if not specified.
+
+::: tip NOTE
+
+It is recommended to set the size of the cache space according to the actual situation. If the memory space is not large enough, the cache will be deleted frequently, which will reduce the performance.
+
+You can check if the following line appears frequently by looking at the [debug log](log.md).
+If it appears almost every request, please increase the size of the cache space appropriately.
+
+```
+ngx_slab_alloc() failed: no memory
+```
+
+Translated with www.DeepL.com/Translator (free version)
+
+::: warning WARNING
+
+This configuration is a new feature in the development version, 
+and can only be used in the development version, 
+and will be merged into the stable version when it is stable.
+
+:::
+
+