Automatic OWASP Test generation

[thim81]

This article in API's You Wont Hate triggered me to think about automatically adding some OWASP variation tests by Portman.

Questions:

• Would that be a useful addition to Portman?
• Which security test should/could be added automatically?
• And how should this be configured? A simple boolean "InjectOWASP" or more fine-grained?)

Looking for some input from the community: @nicklloyd @danielkocot @tillig @savage-alex

[danielkocot]

Summon @miriamgreis

[Gdewilde]

Great idea. This will make basic API security testing much more accessible if Portman supports this out-of-the-box.

[Gdewilde]

Is it possible to support the top 10? https://owasp.org/www-project-api-security/

[tillig]

OWASP ZAP already handles this and also takes in an OAS doc. It supports a lot of scan rules, some of which don't necessarily apply to REST APIs (but if you run it and specify it's an API it will adjust).

While I think additional tests are interesting, I would personally probably always keep this off and just run the Docker version of OWASP ZAP.

If this does get added, it may be interesting to consider some sort of plugin model for Portman, where it's not "all built in" but instead has "test generation plugins" - one plugin generates contract tests, one plugin generates security tests... and if you want to add your own plugins, you can. That'd potentially be a pretty big lift from an internals perspective, but I think it'd be valuable going forward - now folks want some other kind of test generated? Add a plugin. Need to test the various test generators in isolation? Add/remove plugins.

[thim81]

@tillig
It does make sense to keep the Portman core focussed on the current use-cases and not expand it in all directions.

A plugin model, where it is possible to extend the Portman core with specific extensions, would be a great way to facilitate the specific needs from the community. It would also mean a large rework on the internals, so something we could take into account when modernising Portman.