Process HTTP abs_path and query separately

[krizhanovsky]

Currently we process both the URI parts in the same Req_UriAbsPath parser state and validate input against the same alphabet. However, if we process them separately and explicitly parse splitting ?, then we can mitigate many RCE globbing attacks. Trickier cases like /[[:alpha:]][[:alpha:]][[:alpha:]]/[[:alpha:]]s or /[a-z][a-z][a-z]/[a-z]s must be blocked by array syntax checking.

For the caching (e.g. #1391) and WAF features we need to put the GET parameters to some map, ideally with dynamic size (e.g. using TfwStr if it doesn't impact performance) or choose some reasonable limit say 1024 and put the rest arguments at once to the last item with some flag that the item compound.

TBD: Probably we can tokenize the parameters with SIMD (see discussion in #2 ).

Only strict and log (default) modes of the content_security_mode option (see #2) must be supported for now.

Arrays must be treated as compound, probably duplicate, TfwStrs. Examples:

1. ?a=0&files[0]=1&files[1]=2 produces a TfwStr of 6 TfwStr's (which can be compound in case of chunked data): even indexes are with flag TFW_STR_NAME (a, files[0], files[1]) and odd indexes with TFW_STR_VALUE (0, 1, 2).
2. files[]=1&files[]=2 or files[a][]=1&files[a][]=2 generates one TFW_STR_DUPLICATE duplicate TfwStr with four sub-TfwStrs (2 names and 2 values). Empty [] prevents generation of parameter pollution error.
3. ?myarray=value1&myarray=value2&myarray=value3 since myarray has no [], then the query string isn't accepted in strict mode or generates duplicate TfwStr in log mode.
4. ?cars=Saab&cars=Audi is blocked in strict and duplicate in log.
5. ?cars[a]=Saab,Audi one TfwStr with name cars[a] and value Saab,Audi. The same for foo=1|2|3 or any other custom, but allowed, separator, e.g. in ?fields=-*,+foo,+bar the whole -*,+foo,+bar goes as value. ?fields=kind,items(title,characteristics/length) also keeps the whole right part as value.

The last example also continues to objects. While ?foo={bar:0} (which can be represented as ?foo=%7Bbar%3A0%7D) is typically equal to ?foo.bar=0 or ?foo[bar]=0 or application layer,

It is supposed that the parameters mapping takes place after percent-decoding #2 , so the TfwStr's must point to a contiguous memory location

Documentation

Please update the Wiki with description of the enhanced functionality, reference to the original blog post and example of the attacks.

Tests

• test cases from A New Era of SSRF - Exploiting URL Parser in Trending Programming Languages!
• HTTP parameters pollution
• al the examples above
• test that the URIs are blocked
  http://user:[email protected][email protected]
  http://google.com#user:[email protected]
  http://google.com?user:[email protected]