[Modules] Added azureMonitorProfile to allow us to enable the Prometh…

…eus collector (Azure#4329) (Azure#4331)

* Added azureMonitorProfile to aks resource

* Update readme and generate main.json

* Add [[namePrefix]] back into test

---------

Co-authored-by: aadev1 <[email protected]>
Co-authored-by: Asad Arif <[email protected]>

Push updated Readme file(s)

Container Apps Env Upgrade to latest version 2023-05-01 (Azure#4307)

* Upgrade to latest version 2023-05-01

* Formatted

* assign subnet id if provided

* updated arm template and output defaultDomain

* Lint fix

* minor fix for ReservedCidr, ReservedDnsIP and test

Push updated Readme file(s)

ManagedCluster - WebAppRouting - Updates (Azure#4195)

* Update main.bicep

Do not attempt to load Private DNS Zone when `enableDnsZoneContributorRoleAssignment` is false.

Added output `managedCluster.properties.ingressProfile.webAppRouting.identity.objectId` so I can add role assignment for private dns zone

* Update module

* Upgraded bicep, re-ran

Push updated Readme file(s)

Added MOVED-TO-AVM (Azure#4353)

Push updated API Specs file

Added MOVED-TO-AVM for three modules (Azure#4354)

Ported fixed implemented in AVM to reenabled static test issues (Azure#4352)

Adding Moved to AVM for managed-identity/user-assigned-identity (Azure#4339)

* first draft

* updated avm draft

* moved to avm

* conflicts

* final userid

* readme update

Push updated Readme file(s)

migrated module (Azure#4364)

Bugfix for allowed value logic

Updated NSG Module VM (Azure#4365)

Regenerated all docs & json templates (Azure#4366)

[Modules] Cluster - Expose AGIC Identity Object Id (Azure#4349) (Azure#4375)

* Managed Cluster - Expose AGIC Identity Object Id (Azure#4349)

* Add AGIC identity

* Set-Module

* Update to latest

---------

Co-authored-by: Preston Alvarado <[email protected]>

Push updated Readme file(s)

Added MOVED-TO-AVM for four modules (Azure#4379)

Push updated API Specs file

[Module] Implemented workaround for fileShare role assignment (Azure#4391)

* Update to latest

* Update to latest

Push updated Readme file(s)

Users/arnoldna/4409 app managedenvironment (Azure#4410)

* Moved app/managedenvironment to AVM

* Updated Readme

Push updated API Specs file

Push updated API Specs file

[Fixes] Ensure that VirtualMachineImages are removed before MSIs are removed as there's otherwise a high chance for a lock (Azure#4418)

* Updated removal

* Update to latest

* Update to latest

* Update to latest

Added moved md (Azure#4424)

Added MOVED-TO-AVM for recently migrated modules (Azure#4429)

Updated RG readme (Azure#4431)

[Modules] Updates to the Azure Monitor Private Link Service (AMPLS) Module (Azure#4392)

* Added types and upgraded api version

* Added tests

* Uncommented code

* Added documentation to accessModeSettings and added PrivateOnly value

* Testing using condition

* PR feedback

* Removed access mode from default test

* Updated readme and main.json

* PR feedback

* Uncommented code, added description for exclusion

* Updated markdown and json

* Fixed description to adhere to test

* PR feedback

Push updated Readme file(s)

Added MOVED-TO-AVM (Azure#4435)

Added MOVED-TO-AVM (Azure#4439)

Push updated API Specs file

replace ubuntu version (Azure#4437)

[PSRule] Fix Rule Azure.VMSS.AMA (Azure#3918)

* updates

* update readme

* update

---------

Co-authored-by: Fabio Masciotra <[email protected]>

[Modules] Added missing `Moved-to-AVM.md` (Azure#4440)

* Update to latest

* Regenerated docs

[Modules] Removed excess files (Azure#4441)

* Update to latest

* Regenerated docs

* Update to latest

module moved to AVM (Azure#4453)

* module moved to AVM

* update readme

* add notes back

Push updated API Specs file

Migrate container-service/managed-cluster to AVM specs (Azure#4454)

* moved to avm

* Update readme

Push updated Readme file(s)

Added MOVED-TO-AVM (Azure#4455)

Added MOVED-TO-AVM (Azure#4457)

Push updated API Specs file

Moved to AVM (Azure#4463)

[Utilities] Ported updated reference resolution from AVM (Azure#4468)

* Updated script

* Update to latest

Updated module dependency and fixed typo (Azure#4469)

Update to latest (Azure#4470)

[Modules] moved to AVM: local-network-gateway and virtual-network-gateway (Azure#4471)

* updated modules moved to AVM

* updated readme file with AVM migration assertion

---------

Co-authored-by: Fabio Masciotra <[email protected]>

Scaling Plan MOVED-TO-AVM.md (Azure#4483)

* Create MOVED-TO-AVM.md

* updates

Update to latest

Application group MOVED-TO-AVM.md (Azure#4482)

* Create MOVED-TO-AVM.md

* updates

Workspace MOVED-TO-AVM.md (Azure#4485)

* Create MOVED-TO-AVM.md

* updates

Host pool update readme (Azure#4484)

* Create MOVED-TO-AVM.md

* Create MOVED-TO-AVM.md

* updates

[Fixes] Fixed issue with multi-line parameter descriptions (Azure#4488)

* Ported multiline fix from AVM

* Update to latest

Added moved to avm disclaimer to store account

Added moved to avm disclaimer to app container-app

Added Moved to AVM for migrate modules (Azure#4495)

Added MOVED-TO-AVM (Azure#4498)

[Utilities] Enabled Workflow-Trigger function to trigger only for module diff (ported from AVM) (Azure#4499)

* Add support for triggering workflows based on file diff

* Dummy change to test diff

* Dummy change to test diff

* Adding diff module filter

* Removed dummy changes

Moved from CARML to AVM (Azure#4501)

Added missing slash (Azure#4508)

Added MOVED-TO-AVM.md for the machine-learning workspace (Azure#4505)

Added MOVED-TO-AVM.md for the purview account (Azure#4503)

Added MOVED-TO-AVM.md for the DevTest Lab and Network Manager Modules (Azure#4513)

* added dtl

* Update Network Manager modules

Refactor role assignment filtering in Get-RoleAssignmentList.ps1 (Azure#4515)

* Refactor role assignment filtering in Get-RoleAssignmentList.ps1

* fmt

* Add optional parameter to fetch all available roles

* Refactor Get-RoleAssignmentList.ps1 to fetch relevant Role Definitions by default

* Remove unnecessary debug statements in Get-RoleAssignmentList.ps1

Added MOVED-TO-AVM.md for the CDN Profile module (Azure#4512)

Added MOVED-TO-AVM.md for the configuration store (Azure#4506)

Added MOVED-TO-AVM.md for the Signal-R modules (Azure#4517)

[CI Environment] Ported latest AVM pipline changes (e.g. naming) (Azure#4477)

* Ported several changes

* Update to latest

* Update to latest

* Fixed PSRule setup

* Updated pester test file path

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Changed pwsh

* Update to latest

* Update to latest

* Tried alternative path ref

* Added PS Rule ref

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Renamed paths

* Updated to more specific naming to avoid overlaps with e.g., pwsh path var

* Undid unrelated changes

* Added Az.Resources install to PSRule prep

* Update to latest

* GH PS Rule config fix

* Update to latest

* Update to latest

Added MOVED-TO-AVM.md for the recently migrated modules (Azure#4533)

* Added MOVED-TO-AVM for Relay Namespace

* Added MOVED-TO-AVM for Web Connection

Added MOVED-TO-AVM.md for the recently migrated modules (Azure#4538)

* Added MOVED-TO-AVM.md for the Healthcare APIs Workspace

* Update README.md and MOVED-TO-AVM.md for the Application Gateway Web Application Firewall (WAF) Policies module

CARML - AVM module transition complete (Azure#4541)

* readme updates

* readme update

* new template for MOVED-TO-AVM.md

* remove old notice from README files

* updated notice

* update informational notice in the MOVED-TO-AVM.md files

* update moved-to-avm notice

* moved-to-avm, readme and retired files updated

* readme update

* deleting modules' source code

* delete README files

* rename RETIRED.md and MOVED-TO-AVM.md to README.md

* readme update

* readme updates

.azuredevops/pipelineTemplates/jobs.getModuleTestFiles.yml

@@ -5,6 +5,7 @@ parameters:
 
   # Logic-related parameters
   modulePath: '$(modulePath)'
+  psRuleFilterRegex: '(defaults|waf-aligned)'
 
 ##---------------------------------------------##
 ## TEMPLATE LOGIC                              ##
@@ -19,32 +20,58 @@ jobs:
         name: ${{ parameters.poolName }}
     steps:
       - task: PowerShell@2
-        displayName: 'Get parameter files'
+        displayName: 'Get module test file paths'
         name: getModuleTestFilesTask
         inputs:
           targetType: inline
           pwsh: true
           script: |
+            ## ======= ##
+            ##   All   ##
+            ## ======= ##
+
             # Get the list of parameter file paths
             $moduleFolderPath = Join-Path '$(System.DefaultWorkingDirectory)' '${{ parameters.modulePath }}'
-            $testFilePaths = (Get-ChildItem -Path $moduleFolderPath -Recurse -Filter 'main.test.bicep').FullName | Sort-Object
 
-            $deploymentTestPaths = $testFilePaths | ForEach-Object {
+            $testFilePaths = (Get-ChildItem -Path $moduleFolderPath -Recurse -Filter 'main.test.bicep').FullName | Sort-Object
+            $testFilePaths = $testFilePaths | ForEach-Object {
               $_.Replace($moduleFolderPath, '').Trim('\').Trim('/')
             }
 
-            Write-Verbose 'Found module test files' -Verbose
-            $deploymentTestPaths | ForEach-Object { Write-Verbose "- [$_]" -Verbose }
+            Write-Verbose 'Found all module test files' -Verbose
+            $testFilePaths | ForEach-Object { Write-Verbose "- [$_]" -Verbose }
 
             $testTable = @{}
-            foreach ($deploymentTestPath in $deploymentTestPaths) {
-                $deploymentTestFileName = Split-Path (Split-Path $deploymentTestPath -Parent) -Leaf
-                $testTable[$deploymentTestFileName] += @{
-                    moduleTestFilePath = $deploymentTestPath
-                }
-            }
+            $testFilePaths | ForEach-Object {
+              $testFileName = Split-Path (Split-Path $_) -Leaf
+              $testTable[$testFileName] = @{
+                moduleTestFilePath = $_
+                moduleTestFileName = $testFileName
+              }
+            } | ConvertTo-Json -Compress
+            $deployCompressedOutput = $testTable | ConvertTo-Json -Compress
+
+            Write-Verbose "Publishing output: $deployCompressedOutput" -Verbose
+            Write-Host ('##vso[task.setVariable variable=moduleTestFilePaths;isOutput=true]{0}' -f $deployCompressedOutput)
 
-            $deploymentTestPathsOutput = $testTable | ConvertTo-Json -Compress
+            ## =========== ##
+            ##   PS-Rule   ##
+            ## =========== ##
+
+            $psRuleTestFilePaths = $testFilePaths | Where-Object { $_ -match '${{ parameters.psRuleFilterRegex }}' }
+
+            Write-Verbose 'Found PSRule module test files' -Verbose
+            $psRuleTestFilePaths | ForEach-Object { Write-Verbose "- [$_]" -Verbose }
+
+            $psRuleTestTable = @{}
+            $psRuleTestFilePaths | ForEach-Object {
+              $testFileName = Split-Path (Split-Path $_) -Leaf
+              $psRuleTestTable[$testFileName] = @{
+                moduleTestFilePath = $_
+                moduleTestFileName = $testFileName
+              }
+            }
+            $psRuleCompressedOutput = $psRuleTestTable | ConvertTo-Json -Compress
 
-            Write-Host ('##vso[task.setVariable variable=moduleTests;isOutput=true]{0}' -f ($testTable | ConvertTo-Json -Compress))
-            Write-Verbose "Module test files: $deploymentTestPathsOutput" -Verbose
+            Write-Host ('##vso[task.setVariable variable=psRuleModuleTestFilePaths;isOutput=true]{0}' -f $psRuleCompressedOutput)
+            Write-Verbose "PS Rule publishing output: $psRuleCompressedOutput" -Verbose

.azuredevops/pipelineTemplates/jobs.validateModuleDeployment.yml

@@ -29,7 +29,6 @@
 ##   | vmImage                         | '$(vmImage)'                         | You can provide either a [poolname] or [vmImage] to run the job on.                                       | 'ubuntu20.04'                                         |
 ##   | defaultJobTimeoutInMinutes      | 120                                  | The timeout for the job in this pipeline.                                                                 | 120                                                   |
 ##   | removeDeployment                | 'true'                               | Set to [true] to flag resources for removal. If not provided, defaults to true.                           | 'true'                                                |
-##   | templateFilePath                | ''                                   | Path to the template file to deploy.                                                                      | 'modules/analysis-services/servers/main.bicep'        |
 ##   | customTokens                    | ''                                   | Additional token pairs in json format.                                                                    | '{"tokenName":"tokenValue"}'                          |
 ##   | jobDisplayName                  | ''                                   | The display name of the job.                                                                              | 'Deploy module'                                       |
 ##   | modulePath                      | '$(modulePath)'                      | The path to the module to deploy.                                                                         | 'c:/KeyVault'                                         |
@@ -50,7 +49,6 @@ parameters:
   defaultJobTimeoutInMinutes: 120
   # Logic-related parameters
   removeDeployment: false
-  templateFilePath: ''
   customTokens: ''
   modulePath: '$(modulePath)'
   location: '$(location)'
@@ -76,7 +74,7 @@ jobs:
     dependsOn:
       - getModuleTestFiles
     strategy:
-      matrix: $[ dependencies.getModuleTestFiles.outputs['getModuleTestFilesTask.moduleTests'] ]
+      matrix: $[ dependencies.getModuleTestFiles.outputs['getModuleTestFilesTask.moduleTestFilePaths'] ]
     ##---------------------------------------------##
     ## TEMPLATE LOGIC                              ##
     ##---------------------------------------------##
@@ -117,20 +115,16 @@ jobs:
 
       # [Agent] Replace tokens
       #-----------------------
-      - task: AzurePowerShell@5
-        displayName: 'Replace tokens in template file via connection [${{ parameters.serviceConnection }}]'
+      - task: PowerShell@2
+        displayName: 'Replace tokens in template files'
         inputs:
-          azureSubscription: ${{ parameters.serviceConnection }}
-          azurePowerShellVersion: 'latestVersion'
-          preferredAzurePowerShellVersion: ''
-          ScriptType: InlineScript
+          targetType: inline
           pwsh: true
-          inline: |
+          script: |
             # Load used functions
             . (Join-Path '$(System.DefaultWorkingDirectory)' 'utilities' 'pipelines' 'tokensReplacement' 'Convert-TokensInFileList.ps1')
             . (Join-Path '$(System.DefaultWorkingDirectory)' 'utilities' 'pipelines' 'sharedScripts' 'Get-LocallyReferencedFileList.ps1')
 
-
             # Get target files
             $moduleTestFilePath = Join-Path '$(System.DefaultWorkingDirectory)' '$(modulePath)' '$(moduleTestFilePath)'
 

.azuredevops/pipelineTemplates/jobs.validateModulePSRule.yml

@@ -0,0 +1,144 @@
+#########################################################
+## 'Validate module with Pester' Pipeline Template     ##
+#########################################################
+##
+## This pipeline template contains the logic to validate a module using a set of Pester tests
+##
+## Enabled levels of validation
+## - Resource-Group-Level
+## - Subscription-Level
+## - Management-Group-Level
+## - Tenant-Level
+##
+#########################################################
+##
+##---------------------------------------------##
+## TEMPLATE PARAMETERS                         ##
+##---------------------------------------------##
+##
+## By default it uses the variables specified in the below [parameters] section. However, you can overwrite these variables in the
+##    referencing pipeline by providing the parameter explicitly.
+##
+## NOTE: If you don't need to overwrite a shared value, you can IGNORE this section
+##
+##   |==============================================================================================================================================================================================================================================|
+##   | Parameter                  | Default Value                                 | Description                                                                                           | Example                                                 |
+##   |----------------------------|-----------------------------------------------|-------------------------------------------------------------------------------------------------------|---------------------------------------------------------|
+##   | poolName                   | '$(poolName)'                                 | You can provide either a [poolname] or [vmImage] to run the job on                                    | 'Custom Deployment Pool'                                |
+##   | vmImage                    | '$(vmImage)'                                  | You can provide either a [poolname] or [vmImage] to run the job on                                    | 'ubuntu20.04'                                           |
+##   | defaultJobTimeoutInMinutes | 120                                           | The timeout for the job in this pipeline                                                              | 120                                                     |
+##   | modulePath                 | '$(modulePath)'                               | The path to the module to deploy.                                                                     | 'c:/KeyVault'                                           |
+##   | psrulePath                 | 'utilities/pipelines/staticValidation/psrule' | The path to the PS-Rule configuration                                                                 | 'utilities/pipelines/staticValidation/module.tests.ps1' |
+##   | location                   | '$(location)'                                 | The location to validate with                                                                         | 'France Central'                                        |
+##   | subscriptionId             | '$(ARM_SUBSCRIPTION_ID)'                      | The id of the subscription to validate with when using a Management group service connection          | 'aed7c000-6387-412e-bed0-24dfddf4bbc6'                  |
+##   | managementGroupId          | '$(ARM_MGMTGROUP_ID)'                         | The id of the management group to validate with. Required only for Management-Group-Level validations | '477c9620-cb01-454f-9ebc-fc6b1df48c14'                  |
+##   |==============================================================================================================================================================================================================================================|
+##
+##---------------------------------------------##
+
+parameters:
+  # Pipeline-related parameters
+  poolName: '$(poolName)'
+  vmImage: '$(vmImage)'
+  defaultJobTimeoutInMinutes: 120
+  # Logic-related parameters
+  modulePath: '$(modulePath)'
+  psrulePath: 'utilities/pipelines/staticValidation/psrule'
+  location: '$(location)'
+  subscriptionId: '$(ARM_SUBSCRIPTION_ID)'
+  managementGroupId: '$(ARM_MGMTGROUP_ID)'
+
+##---------------------------------------------##
+## TEMPLATE LOGIC                              ##
+##---------------------------------------------##
+jobs:
+  - template: /.azuredevops/pipelineTemplates/jobs.getModuleTestFiles.yml
+  - job:
+    displayName: Run PSRule tests
+    timeoutInMinutes: ${{ parameters.defaultJobTimeoutInMinutes }}
+    pool:
+      ${{ if ne(parameters.vmImage, '') }}:
+        vmImage: ${{ parameters.vmImage }}
+      ${{ if ne(parameters.poolName, '') }}:
+        name: ${{ parameters.poolName }}
+    dependsOn:
+      - getModuleTestFiles
+    strategy:
+      matrix: $[ dependencies.getModuleTestFiles.outputs['getModuleTestFilesTask.psRuleModuleTestFilePaths'] ]
+    steps:
+      # [Agent] Replace tokens
+      #-----------------------
+      - task: PowerShell@2
+        displayName: 'Replace tokens in template files'
+        inputs:
+          targetType: inline
+          pwsh: true
+          script: |
+            # Load used functions
+            . (Join-Path '$(System.DefaultWorkingDirectory)' 'utilities' 'pipelines' 'tokensReplacement' 'Convert-TokensInFileList.ps1')
+            . (Join-Path '$(System.DefaultWorkingDirectory)' 'utilities' 'pipelines' 'sharedScripts' 'Get-LocallyReferencedFileList.ps1')
+
+            # Get target files
+            $moduleTestFilePath = Join-Path '$(System.DefaultWorkingDirectory)' '$(modulePath)' '$(moduleTestFilePath)'
+
+            # Get target files
+            $targetFileList = @(
+              $moduleTestFilePath
+            )
+
+            # Add all module template files as they may contain tokens
+            $targetFileList += (Get-LocallyReferencedFileList -FilePath $moduleTestFilePath)
+            $targetFileList = $targetFileList | Sort-Object -Unique
+
+            # Construct Token Function Input
+            $ConvertTokensInputs = @{
+                FilePathList = $targetFileList
+                Tokens       = @{}
+                TokenPrefix  = '$(tokenPrefix)'
+                TokenSuffix  = '$(tokenSuffix)'
+            }
+
+            # Add enforced tokens
+            $ConvertTokensInputs.Tokens += @{
+              subscriptionId    = '${{ parameters.subscriptionId }}'
+              managementGroupId = '${{ parameters.managementGroupId }}'
+              tenantId          = '$(ARM_TENANT_ID)'
+            }
+
+            # Add local (source control) tokens
+            $tokenMap = @{}
+            foreach ($token in (Get-ChildItem env: | Where-Object -Property Name -Like "localToken_*")) {
+              $tokenMap += @{ $token.Name.Replace('localToken_','','OrdinalIgnoreCase') = $token.value }
+            }
+            Write-Verbose ('Using local tokens [{0}]' -f ($tokenMap.Keys -join ', ')) -Verbose
+            $ConvertTokensInputs.Tokens += $tokenMap
+
+            # Swap 'namePrefix' token if empty and provided as a Azure DevOps variable
+            if([String]::IsNullOrEmpty($ConvertTokensInputs.Tokens['namePrefix'])){
+              Write-Verbose 'Using [namePrefix] token from Azure DevOps Variable Groups' -Verbose
+              $ConvertTokensInputs.Tokens['namePrefix'] = "$(TOKEN_NAMEPREFIX)"
+            }
+
+            # Add custom tokens (passed in via the pipeline)
+            if(-not [String]::IsNullOrEmpty('${{ parameters.customTokens }}')) {
+              $customTokens = '${{ parameters.customTokens }}' | ConvertFrom-Json -AsHashTable
+              Write-Verbose ('Using custom parameter file tokens [{0}]' -f ($customTokens.Keys -join ', ')) -Verbose
+              $ConvertTokensInputs.Tokens += $customTokens
+            }
+
+            Write-Verbose "Convert Tokens Input:`n $($ConvertTokensInputs | ConvertTo-Json -Depth 10)" -Verbose
+
+            # Invoke Token Replacement Functionality [For Module]
+            $null = Convert-TokensInFileList @ConvertTokensInputs
+
+      - task: ps-rule-assert@2
+        displayName: Analyze Azure template files
+        inputs:
+          inputType: inputPath
+          modules: 'PSRule.Rules.Azure'
+          inputPath: '$(System.DefaultWorkingDirectory)/$(modulePath)/$(moduleTestFilePath)'
+          outputFormat: Csv
+          option: '${{ parameters.psrulePath}}/ps-rule.yaml' # Path to PSRule configuration options file
+          source: '${{ parameters.psrulePath}}/.ps-rule/' # Path to folder containing suppression rules to use for analysis.
+          outputPath: '$(System.DefaultWorkingDirectory)/$(modulePath)/$(moduleTestFilePath)-PSRule-output.csv'
+        continueOnError: true