chore: updates CRS tests to CRS 4.0.0-rc2

[M4tteoP]

• Updates CRS tests to CRS 4.0.0-rc2
  • /post has been added to the emulated httpbin behavior, being used for a couple of tests to match the response body.
  • Notably: 934131-5 and 934131-7 are failing. This is happening because:
    - we are urldecoding the cookie key values before storing them
    - The test comes with a + in the payload: Yy9vbmVycm9yJTNkYWxlcnQoMSk+Cg== (snippet of payload). The rule transformations are: t:none,t:urlDecodeUni,t:jsDecode,t:base64Decode,t:urlDecodeUni,t:jsDecode. Being the urldecode before the base64decode, the + is translated into a space, making the base64 fail. Further investigations (and likely comparisons with ModSec runs) are needed.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (f1cfd13) 82.65% compared to head (2725717) 82.37%.
Report is 1 commits behind head on main.

❗ Current head 2725717 differs from pull request most recent head 399fcc6. Consider uploading reports for the commit 399fcc6 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #899      +/-   ##
==========================================
- Coverage   82.65%   82.37%   -0.28%     
==========================================
  Files         162      160       -2     
  Lines        9028     8988      -40     
==========================================
- Hits         7462     7404      -58     
- Misses       1317     1336      +19     
+ Partials      249      248       -1     

Flag	Coverage Δ
default	77.64% <ø> (-0.13%)	⬇️
examples	26.56% <ø> (+0.06%)	⬆️
ftw	46.78% <ø> (-0.35%)	⬇️
ftw-multiphase	48.99% <ø> (-0.33%)	⬇️
tinygo	75.14% <ø> (-0.17%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[jcchavezs]

cc @anuraaga @fzipi @jptosso

[jcchavezs]

Can we finish this @M4tteoP ?

[M4tteoP]

• The test comes with a + in the payload: Yy9vbmVycm9yJTNkYWxlcnQoMSk+Cg== (snippet of payload). The rule transformations are: t:none,t:urlDecodeUni,t:jsDecode,t:base64Decode,t:urlDecodeUni,t:jsDecode. Being the urldecode before the base64decode, the + is translated into a space, making the base64 fail. Further investigations (and likely comparisons with ModSec runs) are needed.

Further investigations led to these two root causes:

• REQUEST_COOKIES variables gets implicitly URL decoded #920
• t:base64decode is too strict (padding required, no partial decoding) #926

I added the links to the description of the ignored test and also mentioned that the rule that is tested by the two excluded tests is actually quite debated CRS side being very specific to a few payloads. More context: coreruleset/coreruleset#3376

[M4tteoP]

Tests 934131-5 and 934131-7 are now fixed. Overall the PR comes with 3 fewer tests not compatible with Coraza