[Feature] Native adapter based WAF support with Istio addon / Support for Wasm plugin adapters

[pratiksharma-dev]

**Is your feature request related to a problem?
It is crucial that we have some solution in AKS for Native WAF support until WAF and CNI overlay support is available in Application gateway for Containers. Even with AGC, there would be customers favoring an open-source adapter integrated WAF to have parity with their on-premises / multi-cloud environments, cloud native mindset and prevent vendor lock ins. The concern is that today customers must introduce an entire heavy weight Application gateway or Front door in front of AKS, just so that they can apply OWASP CRS rules on their inbound requests as other ingress features of AG / AFD are already available via Istio / Nginx natively in kubernetes.

Describe the solution you'd like
I have successfully done a POC of Coraza WAF (https://coraza.io/) with OSS Istio.
Here are some results from my POC of OSS istio on AKS with coraza WAF:

With WAF disabled the request goes through to the pods and no action taken on the injected script:

pratik@DESKTOP-MCESCEE:/mnt/c/Users/prashar/repos/corazawaf$ curl 'http://52.226.101.205/?id=<script>alert(123)</script>' -IL
HTTP/1.1 500 Internal Server Error
server: istio-envoy
date: Tue, 12 Dec 2023 16:44:01 GMT
content-type: text/html
content-length: 291
x-envoy-upstream-service-time: 1
x-envoy-decorator-operation: azure-vote-front.default.svc.cluster.local:80/*

With WAF enabled the request gets blocked by istio and we can see the respective logs in istio-proxy container:

pratik@DESKTOP-MCESCEE:/mnt/c/Users/prashar/repos/corazawaf$ kubectl apply -f ./corazawaf.yml
wasmplugin.extensions.istio.io/coraza-waf configured
pratik@DESKTOP-MCESCEE:/mnt/c/Users/prashar/repos/corazawaf$ curl 'http://52.226.101.205/?id=<script>alert(123)</script>' -IL
HTTP/1.1 403 Forbidden
date: Tue, 12 Dec 2023 16:44:24 GMT
server: istio-envoy
x-envoy-decorator-operation: azure-vote-front.default.svc.cluster.local:80/*
transfer-encoding: chunked

And we can see the request got blocked in the container logs of istio-proxy container:

2023-12-12T16:28:07.439210Z critical envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1180 wasm log default.coraza-waf: [client "10.224.0.113"] Coraza: Access denied (phase 1). Inbound Anomaly Score Exceeded in phase 1 (Total Score: 23) [file "@owasp_crs/**REQUEST-949-BLOCKING-EVALUATION.**conf"] [line "11347"] [id "949111"] [rev ""] [msg "Inbound Anomaly Score Exceeded in phase 1 (Total Score: 23)"] [data ""] [severity "emergency"] [ver "OWASP_CRS/4.0.0-rc2"] [maturity "0"] [accuracy "0"] [tag "anomaly-evaluation"] [hostname "10.224.0.44"] [uri "/?id=<script>alert(123)</script>"] [unique_id "LRgCxRfVTmaxBoTegHx"] thread=24

Describe alternatives you've considered
Currently we can have Application gateway or Azure front door as a decoupled WAF in front of AKS cluster. This is not an integrated solution with native support and has cost and management overhead implications as well.

[wesleyhales]

May I also propose LeakSignal as a candidate? It is freely available as a WASM filter for Istio and is listed on the Azure marketplace:
https://www.leaksignal.com/docs/
https://github.com/leaksignal/leaksignal/releases
https://azuremarketplace.microsoft.com/en-us/marketplace/apps/leaksignalinc1673983004536.leaksignal_test?tab=Overview

I'm happy to provide a basic policy that checks for OWASP top 10 or other types of incoming threats.

[biefy]

Hey @pratiksharma-dev, we recently did a PoC with Coraza + Istio ingress gateway. Like you mentioned, CRS rules works perfectly. However, on the other hand, we also noticed the memory usage of ingress gateway pods shoot up very quickly. This is mainly due to the default WASM stack used by envoy. That is why we did not move further toward this direction. In the mean time, we are working with respective owners in hope that the memory usage could be addressed.

[pdefreitas]

This is a big limitation since official Istio addon for AKS blocks customization of mesh through custom resources like WasmPlugin.

Even if want to POC on our own proxy-wasm coraza WAF it is blocked. Extensibility possibilities using this addon are currently limited.

[shashankbarsin]

@pdefreitas - WASM plugin is currently in alpha state in Istio OSS - https://istio.io/latest/docs/reference/config/proxy_extensions/wasm-plugin/. Combined with the memory usage issue @biefy mentioned above we are currently not in a state of opening up WasmPlugin since we have to provide official support for any issues arising out of the addon. On WAF, we do have that in our backlog and will share an update here once we have a definitive ETA to it after addressing the performance concerns listed above. In the interim, if you are ok with the performance tradeoffs of WASM plugin, @pdefreitas OSS Istio charts on AKS can be used (even with alpha features of Istio).

[pdefreitas]

@shashankbarsin I do understand that WasmPlugin is still an alpha API but there are other blocked resources that are already stable like Telemetry since 1.22. There could be a preview available e.g. with a feature toggle for such resources.
The memory leak issues might be related with coraza wasm implementation:
corazawaf/coraza-proxy-wasm#249
But this just an example of a Wasm plugin. Users might want to extend and customize in different ways.
I will look forward to use the OSS helm chart instead due to the limitations.

[nshankar13]

@pdefreitas we will be unblocking Telemetry API soon, there were a few remaining implementation and testing blockers thus far preventing us from doing so, which we have addressed. We also opened up Lua EnvoyFilter customization for all users, and in the next AKS release, Local Rate Limiting and GZIP compression filters will be unblocked as well. A caveat here is that EnvoyFilter will be "allowed" but still "unsupported" as per our support policy, so Azure will not be providing support for the resource.

We will have documentation out regarding our support policy for EnvoyFilter and Telemetry API in the near future.

[nshankar13]

@pdefreitas Telemetry API is now available for asm-1-22 onwards: https://learn.microsoft.com/en-us/azure/aks/istio-telemetry.