Streamline Maester Test output

[spaelling]

I am making the assumption that these tests was written by different people, and the great team behind Maester was learning on the go, so fully understandable of the current state.

When looking at the overall test results there is a lot of variance that makes it all harder to read and navigate.

ex. EIDSCA has a naming convention using a prefix, like EIDSCA.AG03: Authentication Method - General Settings - Report suspicious activity - Included users/groups. - here it is clear this is regarding authentication methods, but MS.AAD does not, ex. MS.AAD.1.1: Legacy authentication SHALL be blocked. does not make it clear if this is something with authentication (it is not, and relates to a CA policy).

MS.AAD and MS.EXO uses SHALL and SHOULD (here I assume that should is a softer recommendation than shall, but not entirely clear), but when looking at the MT tests this is not used, and in MT there is sometimes a postfix in the title that indicates the area, like MT.1024: Entra Recommendation - Do not expire passwords., but not in ex. MT.1027: No Service Principal with Client Secret and permanent role assignment on Control Plane.

This is just the list overview. I guess the individual layout of each test mandates a different discussion, and also requires much more work to implement, but my hope is that at least the list overview could be streamlined so that it becomes easier to read and navigate (especially for novice users).

[soulemike]

Hey @spaelling, I could definitely see some normalization through tagging being beneficial.

Since each of the control sets are different projects in their own right, they will definitely vary.

• The Maester tests would be specific to this project.
• The Entra ID Security Config Analyzer (EIDSCA) tests come from that project.
• The CISA tests come from their SCuBA project. They do use language aligned with RFC 2119 though.
• The CIS tests would come from those benchmarks as well. They will typically use language similar to RFC 2119 as well.
• Then the ORCA tests would come from that project, though these still aren't in prerelease channel.

One challenge I have faced when considering ways to normalize this, is how to have language and test concepts deduplicated without losing clarity and consistency of the parent framework.

A good example, is Calendar Sharing as a control common between CIS and CISA.

• https://github.com/maester365/maester/blob/main/powershell/public/cisa/exchange/Test-MtCisaCalendarSharing.ps1
• https://github.com/maester365/maester/blob/main/powershell/public/cis/Test-MtCisCalendarSharing.ps1

Functionally these are the same tests and the same security control. Though they are completely separate recommendations from the respective entities.