Discovery Performance Enhancements

This release includes a number of updates designed to improve control over discovery, and provide better discovery performance for large environments.

Changes include:

• Improve CacheMode functionality and add DiscoveryMode to enable better control of discovery for IAM and Policy
• Bugfix for Sort-Object error when running New-AzStateDiscovery with multiple types selected
• Improved resource type coverage when generating the Parent, Parents and ParentPath and ResourcePath properties
• Added the following AtScope methods to simplify discovering which IAM and Policy objects are applied directly at the scope of the current AzState object, rather than all inherited objects:
  • [AzState].GetRoleAssignmentsAtScope()
  • [AzState].GetRoleDefinitionsAtScope()
  • [AzState].GetPolicyAssignmentsAtScope()
  • [AzState].GetPolicyDefinitionsAtScope()
  • [AzState].GetPolicySetDefinitionsAtScope()
• General improvements to logging, including the addition of logs in the Information stream to provide better visibility of discovery progress without enabling full Verbose stream logs
• Added new static property [AzState]::ParentMap to act as Cache for storing a list of parent IDs for Management Groups and Subscriptions, providing improved performance by reducing lookup complexity for the GetParent method
• Added case sensitivity control for AzState caches with default setting to ignore case [AzState]::DefaultCacheCaseSenstive = $false - this helps improve discovery performance by reducing the number of false cache misses due to mismatching case being returned from the ARM API, but can be easily set back to case-sensitive if needed for specific use-cases

Improve New-AzStateDiscovery

This release fixes an issue where AzState objects returned from thread jobs in PowerShell return the following error when accessing methods:

OperationStopped: Object reference not set to an instance of an object.

This ensures that the output from New-AzStateDiscovery (and other operations using thread jobs to generate AzState objects) are able to operate as expected for subsequent commands.

As part of this fix, we have also updated the FromScope method to utilise FromIds when processing in [parallel] mode. This helps to keep the code DRY and reduces future test coverage requirements.

Although planned for removal from the class (intent is to move to dedicated set of functions to improve maintainability and reduce the reliance on calss updates), this release also introduces some changes to the built-in Terraform() method to ensure semantically correct code is generated.

The GetAzStateChildren method has been removed, as this is handled by the Get-AzStateChildrenByType function.

Updates have also been made to the IAM and Policy methods, making discovery a manual step. This is to provide a substantial performance boost for scenarios where IAM and Policy discovery is not required. Further updates will be made in this area to improve the process of getting AzState for IAM and Policy objects whilst populating the IAM and Policy attributes in the AzState object for the primary Resource.

Release parallel processing and New-AzStateDiscovery

This release includes a release-ready version of the parallel processing methods within [AzState] and introduces the first release of New-AzStateDiscovery function to perform iterative discovery of child resources form the specified Root ID(s).

New-AzStateDiscovery supports the following input parameters:

• [String[]]$RootId specifies the ID to use for the start of discovery, supports multiple inputs including from pipeline
• [String[]]$ExcludePathIds is used to specify onw or more resource IDs to exclude from discovery, allowing partial discovery
• [Switch]$IncludeManagementGroups if specified will discover child resources of type Microsoft.Management/managementGroups
• [Switch]$IncludeSubscriptions if specified will discover child resources of type Microsoft.Management/managementGroups/subscriptions and Microsoft.Resources/subscriptions
• [Switch]$IncludeResourceGroups if specified will discover child resources of type Microsoft.Resources/resourceGroups
• [Switch]$IncludeResources if specified will discover child resources of Resource Groups
• [Switch]$IncludeIAM if specified will discover all Access control (IAM) settings for supported resources
• [Switch]$IncludePolicy if specified will discover all Policy settings for supported resources
• [Switch]$Recurse if specified will perform a recursive discovery on all discovered resources
• [Int]$ThrottleLimit is specified will change the number of threads to use for parallel processing (default = 4)
• [CacheMode]$CacheMode is specified will control whether the command uses cache, or not (supported values = UseCache and SkipCache

To support the processing of New-AzStateDiscovery, a private function Get-AzStateChildrenByType is included but not exported as this isn't expected to be used directly.

This release also includes the follow minor bug fixes:

• Update AzProviderCache to case insensitive (to support resources returned from API with incorrect case in ID)
• Bugfix set Raw value using DirectFromScope (to ensure AzState is populated with Raw value using this method)

This release skips 0.0.6 and 0.0.7 which were used for internal testing only.

Refactor AzState object and improve CacheMode controls

This release makes a breaking change to the AzState object structure.

• (+) Add attributes:
  • (+) Raw
• (-) Removes attributes:
  • (-) Properties
  • (-) ExtendedProperties

The new Raw attribute is now used to store the original Resource configuration, as downloaded from the Rest API, removing the need to Properties and ExtendedProperties to be stored.

This change is designed to provide better supportability with any API changes, as the Raw block will capture the entire state in original form and make identifying and handling changes easier.

The release also provides further updates to how caching is controlled as we move towards supporting full refresh of dependencies when refreshing an AzState object from the API (SkipCache). This includes updating the New-AzState function to provide control over caching.

Add DirectFromScope Method

To complement existing functionality regarding parallel processing of resource discovery based on a specified resource scope, this release adds a new DirectFromScope Method which can be used to generate new AzState objects from the original scope-level response.

This can be useful in situations where the scope level response contains the full resource definition. An example of this is obtaining Policy Definitions at a given scope.

This release also includes minor updates for the default constructors [AzState]::new(), [AzState]::Update() and [AzState]::Initialize() methods.

This also includes the first stages of providing greater control over "CacheMode" to ensure all Resources used to generate an AzState hierarchy can be refreshed from Cache or API based on user preference.

Optimisations for parallel processing

0.0.3

Push module to release 0.0.3

First release, as uploaded to PSGallery

This is an initial release representing the module state as uploaded to PSGallery using version tag 0.0.1

Only basic functionality is available, and this is intended for internal testing only.