Unable to expand resources. The language expression property 'X' doesn't exist

[nbwdk]

@BernieWhite

Getting an error, when referencing a user assigned managed identity cross modules within the same bicep files scoped at subscription target

ERROR Failed to expand bicep source '/main.test.bicepparam'. Exception calling "GetBicepParamResources" with "2" argument(s): "Unable to expand resources because the source file '/main.test.bicepparam' was not valid. An error occurred evaluating expression '[reference('userAssignedIdentity').outputs.id.value]' line 1414. The language expression property 'id' doesn't exist."

Using PSRule version 1.39.2

containerRegistry.bicep

targetScope = 'resourceGroup'

param containerRegistryName string
@allowed([
  'Basic'
  'Standard'
  'Premium'
])
param containerRegistrySku string
param location string
param logAnalyticsId string
param userAssignedId string
param acrRoleAssignments object[]
param tags object

resource containerRegistry 'Microsoft.ContainerRegistry/registries@2023-11-01-preview' = {
  name: containerRegistryName
  location: location
  identity: {
    type: 'UserAssigned'
    userAssignedIdentities: {
      '${userAssignedId}': {}
    }
  }
  sku: {
    name: containerRegistrySku
  }
  tags:tags
  properties: {
    adminUserEnabled: false
    policies: {
      azureADAuthenticationAsArmPolicy: {
        status: 'enabled'
      }
    }
  }
}

resource acrMonitoring 'Microsoft.Insights/diagnosticSettings@2021-05-01-preview' = {
  name: '${containerRegistryName}-logging'
  scope: containerRegistry
  properties: {
    logs: [
      {
        category: 'ContainerRegistryRepositoryEvents'
        enabled: true
      }
      {
        category: 'ContainerRegistryLoginEvents'
        enabled: true
      }
    ]
    metrics: [
      {
        category: 'AllMetrics'
        enabled: true
      }
    ]
    workspaceId: logAnalyticsId
  }
}

@description('Following will add principals to the key vault Reader')
module acrReaderRoleAssignment 'rgRoleAssignment.bicep' = {

  name: 'acrReaderRoleAssignment'
  params: {
 roleAssignments:acrRoleAssignments
  }
}

output containerRegistryName string = containerRegistry.name
output containerRegistryId string = containerRegistry.id
output containerRegistryUrl string = containerRegistry.properties.loginServer

Common.bicep

targetScope = 'subscription'
// Parameters
@description('')
@allowed([
  'test'
  'prod'
])
@minLength(3)
@maxLength(4)
param environment string

@description('')
@maxLength(3)
param solutionName string

@description('')
@maxLength(5)
param companyInitName string

@description('Deployment Location')
@allowed([
  'westeurope'
  'northeurope'
  'swedencentral'
])
param location string

@description('')
param sequenceNo string
@description('')
param tags object
@description('')
param persistent bool
@description('')
param commonSubnetId string
@description('')
param clusterSubnetId string
@description('')
param logAnalyticsWorkspaceId string
@description('')
param kvRoleAssignments object[]
@description('')
param acrRoleAssignments object[]
@description('')
param appInsightsName string
@description('')
param appiConnectionString string

//Function import
import { naming } from '../../functions/naming.bicep'

// Variables
var sequence = environment == 'prod' ? '' : '${sequenceNo}'
var rgCommonName = naming('rg', 'common', companyInitName, solutionName, environment, sequence).name
var keyVaultName = naming('kv', '', companyInitName, solutionName, environment, sequence).name
var userAssignedIdentityName = naming('id', '', companyInitName, solutionName, environment, sequence).name
var containerRegistryName = naming('acr', '', companyInitName, solutionName, environment, sequence).nameWithoutDash

// Resources

resource rgCommon 'Microsoft.Resources/resourceGroups@2024-07-01' = {
  name: rgCommonName
  location: location
  tags: union(tags, {})
}

module rgCommonPersistentLock '../../core/lockRg.bicep' = if (persistent) {
  name: '${rgCommonName}-Lock-Deployment'
  scope: rgCommon
}

module userAssignedIdentity '../../core/userAssignedIdentities.bicep' = {
  scope: rgCommon
  name: '${userAssignedIdentityName}-Deployment'
  params: {
    userAssignedName: userAssignedIdentityName
    location: location
    tags: tags
  }
}

var kvRoleAssignmentsAutoGenerated = [
  {
    name: 'Key Vault Secrets User: Allow web app Reader role to key vault via User Assigned Identity'
    roleDefinitionResourceId: '/providers/Microsoft.Authorization/roleDefinitions/4633458b-17de-408a-b874-0445c86b69e6'
    principalId: userAssignedIdentity.outputs.principalObjectId
    principalType: 'ServicePrincipal'
  }
]

module keyVault '../../core/keyVault.bicep' = {
  name: '${keyVaultName}-Deployment'
  scope: rgCommon
  params: {
    subnetIds: [
      {
        id: commonSubnetId
      }
      {
        id: clusterSubnetId
      }
    ]
    keyVaultName: keyVaultName
    location: location
    tags: tags
    logAnalyticsWorkspaceId: logAnalyticsWorkspaceId
    kvRoleAssignments: union(kvRoleAssignments, kvRoleAssignmentsAutoGenerated)
  }
  dependsOn: [
    userAssignedIdentity
  ]
}

var acrRoleAssignmentsAutoGenerated = [
  {
    name: 'AcrPull. Pull artifacts from a container registry.'
    roleDefinitionResourceId: '/providers/Microsoft.Authorization/roleDefinitions/7f951dda-4ed3-4680-a7ca-43fe172d538d'
    principalId: userAssignedIdentity.outputs.principalObjectId
    principalType: 'ServicePrincipal'
  }
]

module containerRegistry '../../core/containerRegistry.bicep' = {
  scope: rgCommon
  name: '${containerRegistryName}-Deployment'
  params: {
    location: location
    acrRoleAssignments: union(acrRoleAssignments, acrRoleAssignmentsAutoGenerated)
    containerRegistryName: containerRegistryName
    containerRegistrySku: 'Standard'
    logAnalyticsId: logAnalyticsWorkspaceId
    tags: tags
    userAssignedId: userAssignedIdentity.outputs.id
  }
  dependsOn: [
    userAssignedIdentity
  ]
}

module applicationInsightsConnectionString '../../core/keyVaultCreateSecret.bicep' = {
  name: '${appInsightsName}-Secret-Deployment'
  scope: rgCommon
  params: {
    keyVaultName: keyVaultName
    secretName: '${appInsightsName}ConnectionString'
    secretValue: appiConnectionString
  }
  dependsOn: [
    keyVault
  ]
}

@description('User Assigned Identity Id (scope)')
output userAssignedIdentityId string = userAssignedIdentity.outputs.id
@description('User Assigned Identity Principle Id (Object Id)')
output userAssignedIdentityObjectId string = userAssignedIdentity.outputs.principalObjectId
@description('User Assigned Identity Client Id')
output userAssignedIdentityClientId string = userAssignedIdentity.outputs.clientId
output containerRegistryName string = containerRegistry.outputs.containerRegistryName
output containerRegistryId string = containerRegistry.outputs.containerRegistryId
output containerRegistryUrl string = containerRegistry.outputs.containerRegistryUrl
output keyVaultName string = keyVault.outputs.keyVaultName

Originally posted by @nbwdk in #2850

[BernieWhite]

Thanks for reporting the issue @nbwdk.

Are you able to provide the version you are using? Or was it v1.39.2 from the previous post?

[BernieWhite]

@nbwdk I'm not able to reproduce this issue on v1.39.2, but I'm only guessing at the content of some of these files. Noting v1.39.3 is the latest stable release.

Are you able to provide a cutdown complete set of bicep files?

[riosengineer]

Hey @BernieWhite, maybe I can help with some info on this also. I also get this error too in 1.40.0 although on the azureADOnlyAuthentication eval. Info below, if it helps:

main.bicep

param location string
param rgName string
param sqlDatabase object = {}
@secure()
param sqlAdministrators object = {}
param sqlServerName string


// targetScope (optional)
targetScope = 'subscription'

// Rg
module resourceGroup 'br/public:avm/res/resources/resource-group:0.4.0' = {
  name: '${uniqueString(deployment().name, location)}-rg'
  params: {
    name: rgName
    location: location
  }
}

// Azure SQL DB
module sqlDb 'br/public:avm/res/sql/server:0.11.1' = {
  name: '${uniqueString(deployment().name, location)}-sql'
  scope: resourceGroup(rgName)
  params: {
    name: sqlServerName
    location: location
    minimalTlsVersion: '1.2'
    managedIdentities: {
      systemAssigned: true
    }
    publicNetworkAccess: 'Enabled' 
    administrators: {
      azureADOnlyAuthentication: sqlAdministrators.azureADOnlyAuthentication
      login: sqlAdministrators.login
      sid: sqlAdministrators.sid
      principalType: sqlAdministrators.principalType
    }
    databases: [
      {
        name: sqlDatabase.name
        collation: sqlDatabase.collation
        maxSizeBytes: sqlDatabase.maxSizeBytes
        zoneRedundant: false
        sku: {
          name: sqlDatabase.sku
          tier: sqlDatabase.tier
        }
      }
    ]
    firewallRules: []
      }
    }
  }
}

main.bicepparam

param location = 'uksouth'
param sqlServerName = 'sql-svr-contoso'
// SQL Database
param sqlDatabase = {
    name: 'sqldb-psrule'
    tier: 'Basic'
    sku: 'Basic'
    maxSizeBytes: 2147483648
    collation: 'SQL_Latin1_General_CP1_CI_AS'
}
param sqlAdministrators = {
  azureADOnlyAuthentication: true
  login: 'SG_SomeGroup'
  sid: 'GUID'
  principalType: 'Group'
}

main.test.bicep

// Test with only required parameters
module test '../main.bicep' = {
  scope: subscription('guid')
  name: 'deploy'
  params:{
    location: 'uksouth'
    sqlAdministrators: {
      azureADOnlyAuthentication: true
      login: 'SG_SomeGroup'
      sid: 'guid'
      principalType: 'Group'
    }
    sqlDatabase: {
      name: 'sqldb-contoso'
      tier: 'Basic'
      sku: 'Basic'
    }

Error

 *  Executing task: Assert-PSRule -Format File -Path './.ps-rule/' -InputPath . -Outcome Fail, Error; 

Using PSRule v2.9.0
Using PSRule.Rules.Azure v1.40.0
  ERROR  Failed to expand bicep source '\.tests\main.test.bicep'. Exception calling "GetBicepResources" with "2" argument(s): "Unable to expand resources because the source file '\.tests\main.test.bicep' was not valid. An error occurred evaluating expression '[parameters('sqlAdministrators').azureADOnlyAuthentication]' line 23436. The language expression property 'azureADOnlyAuthentication' doesn't exist."

Happy to do further troubleshooting if need be. Let me know if I can help.

[BernieWhite]

Thanks @riosengineer Your main.test.bicep example was missing maxSizeBytes and collation e.g. below.

However after adding these I can't reproduce the azureADOnlyAuthentication error on v1.40.0. Are you able to provide any Bicep configuration experimental flags that are enabled? Also what version of Bicep CLI are you using?

// Test with only required parameters
module test '../main.bicep' = {
  scope: subscription('guid')
  name: 'deploy'
  params: {
    rgName: 'rg1'
    sqlServerName: 'contoso'
    location: 'uksouth'
    sqlAdministrators: {
      azureADOnlyAuthentication: true
      login: 'SG_SomeGroup'
      sid: 'guid'
      principalType: 'Group'
    }
    sqlDatabase: {
      name: 'sqldb-contoso'
      tier: 'Basic'
      sku: 'Basic'
      maxSizeBytes: 2147483648
      collation: 'SQL_Latin1_General_CP1_CI_AS'
    }
  }
}

[maythamfahmi]

This happened to me as well.

📢 Here is a solution. or maybe let's call it a workaround. @nbwdk @BernieWhite

Before going to the solution here is the version of each component I use

PowerShell Version: 7.4.6
PowerShell Edition: Core
Running in Windows Terminal: Yes
Module Versions:
PSRule: 2.9.0
PSRule.Rules.Azure: 1.40.0
PSRule.Rules.CAF: 0.3.0
Az: 13.0.0
Bicep: Bicep CLI version 0.32.4 (b326faa456)

Problem

Your model userAssignedIdentity which is using Microsoft.ManagedIdentity/userAssignedIdentities@2023-07-31-preview as I assume, needs to be executed before getting consumed. It creates a racing condition at the running time, possibly ending with what you are trying to consume before getting the output values.

Solution

The way I solved it is by moving the userAssignedIdentity to a separate step before common resources and making the steps depend on, something like following in your main bicep

module identity './modules/identity.bicep' {
  name: 'Identity-Deployment'
  params: {
    ...
  }
  dependsOn: [
    previous_resources_if_there_is
  ]
}

module common './modules/common.bicep' {
  name: 'Common-Deployment'
  params: {
    ...
    userAssignedId: identity.outputs.userAssignedIdentityId
    principalObjectId: identity.outputs.userAssignedIdentityPrincipalObjectId
  }
  dependsOn: [
    identity
  ]
}

With this solution, you can PSRule test without issues, you can deploy without issues.

[BernieWhite]

Thanks @maythamfahmi. I'd like to understand the issue more so that I can provide a fix. Thanks for the bicep version.

In your example dependsOn shouldn't be required because this is implicitly generated by Bicep if you reference the symbolic name within the common block. i.e. userAssignedId: identity.outputs.userAssignedIdentityId.

So, would I be right in saying that if userAssignedIdentity is in the main bicep file here it generates an error? Where was the userAssignedIdentity resource in the main bicep file before or after the common block when it was generating an error?

[riosengineer]

Thanks @riosengineer Your main.test.bicep example was missing maxSizeBytes and collation e.g. below.

However after adding these I can't reproduce the azureADOnlyAuthentication error on v1.40.0. Are you able to provide any Bicep configuration experimental flags that are enabled? Also what version of Bicep CLI are you using?

// Test with only required parameters
module test '../main.bicep' = {
scope: subscription('guid')
name: 'deploy'
params: {
rgName: 'rg1'
sqlServerName: 'contoso'
location: 'uksouth'
sqlAdministrators: {
azureADOnlyAuthentication: true
login: 'SG_SomeGroup'
sid: 'guid'
principalType: 'Group'
}
sqlDatabase: {
name: 'sqldb-contoso'
tier: 'Basic'
sku: 'Basic'
maxSizeBytes: 2147483648
collation: 'SQL_Latin1_General_CP1_CI_AS'
}
}
}

Ah yeah, that's my bad, I just didn't paste across the maxSizeBytes property, but the collation was definitely missing from my file. I also cannot replicate now in 1.40.0 with latest Bicep CLI now.

I have one feature enabled just for FYI:

  "experimentalFeaturesEnabled": {
        "symbolicNameCodegen": true
    }

I actually think my particular root cause was:

I had two new test files with SQL resources in the main template. In one of the test files I was missing the sqlAdministrators object as it didn't auto-complete as a required param, so I missed it. When adding it in, no issues. So tldr: user error!

[BernieWhite]

Thanks for the update @riosengineer.

@maythamfahmi Also can you confirm if you are using any experimental features in bicepconfig.json.

[maythamfahmi]

@BernieWhite I can confirm I am using the following

{
  "experimentalFeaturesEnabled": {
    "assertions": true,
    "testFramework": true,
    "extendableParamFiles": true
  }
}

regarding your previous message, I can provide more details if still relevant :)

Thanks @maythamfahmi. I'd like to understand the issue more so that I can provide a fix. Thanks for the bicep version.

[BernieWhite]

@maythamfahmi Absolutely, if there is a race case between resource dependencies then we should fix it. Ideally if you can provide a self-contained reproduction, this will assist significantly.