Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exporting Intune device configurations to a new tenant creates duplicate device configuration entries each time it is run #5447

Open
rick-engle opened this issue Nov 21, 2024 · 36 comments

Comments

@rick-engle
Copy link

Description of the issue

As I was using Microsoft365DSC to migrate settings from my old to my new tenant, I noticed that fortunately the majority of settings imported were not duplicated with each run however Intune Device Configurations are being duplicated:
Image
This should be a screenshot that illustrates the problem in Intune.
The command being used to export my Intune components is:
Export-M365DSCConfiguration -Components @("IntuneAccountProtectionLocalAdministratorPasswordSolutionPolicy", "IntuneAccountProtectionLocalUserGroupMembershipPolicy", "IntuneAccountProtectionPolicy", "IntuneAntivirusPolicyWindows10SettingCatalog", "IntuneAppConfigurationDevicePolicy", "IntuneAppConfigurationPolicy", "IntuneApplicationControlPolicyWindows10", "IntuneAppProtectionPolicyAndroid", "IntuneAppProtectionPolicyiOS", "IntuneASRRulesPolicyWindows10", "IntuneAttackSurfaceReductionRulesPolicyWindows10ConfigManager", "IntuneDeviceAndAppManagementAssignmentFilter", "IntuneDeviceCategory", "IntuneDeviceCleanupRule", "IntuneDeviceCompliancePolicyAndroid", "IntuneDeviceCompliancePolicyAndroidDeviceOwner", "IntuneDeviceCompliancePolicyAndroidWorkProfile", "IntuneDeviceCompliancePolicyiOs", "IntuneDeviceCompliancePolicyMacOS", "IntuneDeviceCompliancePolicyWindows10", "IntuneDeviceConfigurationAdministrativeTemplatePolicyWindows10", "IntuneDeviceConfigurationCustomPolicyWindows10", "IntuneDeviceConfigurationDefenderForEndpointOnboardingPolicyWindows10", "IntuneDeviceConfigurationDeliveryOptimizationPolicyWindows10", "IntuneDeviceConfigurationDomainJoinPolicyWindows10", "IntuneDeviceConfigurationEmailProfilePolicyWindows10", "IntuneDeviceConfigurationEndpointProtectionPolicyWindows10", "IntuneDeviceConfigurationFirmwareInterfacePolicyWindows10", "IntuneDeviceConfigurationHealthMonitoringConfigurationPolicyWindows10", "IntuneDeviceConfigurationIdentityProtectionPolicyWindows10", "IntuneDeviceConfigurationImportedPfxCertificatePolicyWindows10", "IntuneDeviceConfigurationKioskPolicyWindows10", "IntuneDeviceConfigurationNetworkBoundaryPolicyWindows10", "IntuneDeviceConfigurationPkcsCertificatePolicyWindows10", "IntuneDeviceConfigurationPlatformScriptMacOS", "IntuneDeviceConfigurationPlatformScriptWindows", "IntuneDeviceConfigurationPolicyAndroidDeviceAdministrator", "IntuneDeviceConfigurationPolicyAndroidDeviceOwner", "IntuneDeviceConfigurationPolicyAndroidOpenSourceProject", "IntuneDeviceConfigurationPolicyAndroidWorkProfile", "IntuneDeviceConfigurationPolicyiOS", "IntuneDeviceConfigurationPolicyMacOS", "IntuneDeviceConfigurationPolicyWindows10", "IntuneDeviceConfigurationSCEPCertificatePolicyWindows10", "IntuneDeviceConfigurationSecureAssessmentPolicyWindows10", "IntuneDeviceConfigurationSharedMultiDevicePolicyWindows10", "IntuneDeviceConfigurationTrustedCertificatePolicyWindows10", "IntuneDeviceConfigurationVpnPolicyWindows10", "IntuneDeviceConfigurationWindowsTeamPolicyWindows10", "IntuneDeviceConfigurationWiredNetworkPolicyWindows10", "IntuneDeviceEnrollmentLimitRestriction", "IntuneDeviceEnrollmentPlatformRestriction", "IntuneDeviceEnrollmentStatusPageWindows10", "IntuneDeviceRemediation", "IntuneDiskEncryptionMacOS", "IntuneEndpointDetectionAndResponsePolicyWindows10", "IntuneExploitProtectionPolicyWindows10SettingCatalog", "IntunePolicySets", "IntuneRoleAssignment", "IntuneRoleDefinition", "IntuneSettingCatalogASRRulesPolicyWindows10", "IntuneSettingCatalogCustomPolicyWindows10", "IntuneWifiConfigurationPolicyAndroidDeviceAdministrator", "IntuneWifiConfigurationPolicyAndroidEnterpriseDeviceOwner", "IntuneWifiConfigurationPolicyAndroidEnterpriseWorkProfile", "IntuneWifiConfigurationPolicyAndroidForWork", "IntuneWifiConfigurationPolicyAndroidOpenSourceProject", "IntuneWifiConfigurationPolicyIOS", "IntuneWifiConfigurationPolicyMacOS", "IntuneWifiConfigurationPolicyWindows10", "IntuneWindowsAutopilotDeploymentProfileAzureADHybridJoined", "IntuneWindowsAutopilotDeploymentProfileAzureADJoined", "IntuneWindowsInformationProtectionPolicyWindows10MdmEnrolled", "IntuneWindowsUpdateForBusinessDriverUpdateProfileWindows10", "IntuneWindowsUpdateForBusinessFeatureUpdateProfileWindows10", "IntuneWindowsUpdateForBusinessRingUpdateProfileWindows10") -ApplicationId $clientId -TenantId $tenantIdDomainName -ApplicationSecret $clientSecretValue -Path $SavePath -FileName $SaveFileName

Microsoft 365 DSC Version

1.24.1120.1

Which workloads are affected

Intune

The DSC configuration

The command being used to export my Intune components is:
Export-M365DSCConfiguration -Components @("IntuneAccountProtectionLocalAdministratorPasswordSolutionPolicy", "IntuneAccountProtectionLocalUserGroupMembershipPolicy", "IntuneAccountProtectionPolicy", "IntuneAntivirusPolicyWindows10SettingCatalog", "IntuneAppConfigurationDevicePolicy", "IntuneAppConfigurationPolicy", "IntuneApplicationControlPolicyWindows10", "IntuneAppProtectionPolicyAndroid", "IntuneAppProtectionPolicyiOS", "IntuneASRRulesPolicyWindows10", "IntuneAttackSurfaceReductionRulesPolicyWindows10ConfigManager", "IntuneDeviceAndAppManagementAssignmentFilter", "IntuneDeviceCategory", "IntuneDeviceCleanupRule", "IntuneDeviceCompliancePolicyAndroid", "IntuneDeviceCompliancePolicyAndroidDeviceOwner", "IntuneDeviceCompliancePolicyAndroidWorkProfile", "IntuneDeviceCompliancePolicyiOs", "IntuneDeviceCompliancePolicyMacOS", "IntuneDeviceCompliancePolicyWindows10", "IntuneDeviceConfigurationAdministrativeTemplatePolicyWindows10", "IntuneDeviceConfigurationCustomPolicyWindows10", "IntuneDeviceConfigurationDefenderForEndpointOnboardingPolicyWindows10", "IntuneDeviceConfigurationDeliveryOptimizationPolicyWindows10", "IntuneDeviceConfigurationDomainJoinPolicyWindows10", "IntuneDeviceConfigurationEmailProfilePolicyWindows10", "IntuneDeviceConfigurationEndpointProtectionPolicyWindows10", "IntuneDeviceConfigurationFirmwareInterfacePolicyWindows10", "IntuneDeviceConfigurationHealthMonitoringConfigurationPolicyWindows10", "IntuneDeviceConfigurationIdentityProtectionPolicyWindows10", "IntuneDeviceConfigurationImportedPfxCertificatePolicyWindows10", "IntuneDeviceConfigurationKioskPolicyWindows10", "IntuneDeviceConfigurationNetworkBoundaryPolicyWindows10", "IntuneDeviceConfigurationPkcsCertificatePolicyWindows10", "IntuneDeviceConfigurationPlatformScriptMacOS", "IntuneDeviceConfigurationPlatformScriptWindows", "IntuneDeviceConfigurationPolicyAndroidDeviceAdministrator", "IntuneDeviceConfigurationPolicyAndroidDeviceOwner", "IntuneDeviceConfigurationPolicyAndroidOpenSourceProject", "IntuneDeviceConfigurationPolicyAndroidWorkProfile", "IntuneDeviceConfigurationPolicyiOS", "IntuneDeviceConfigurationPolicyMacOS", "IntuneDeviceConfigurationPolicyWindows10", "IntuneDeviceConfigurationSCEPCertificatePolicyWindows10", "IntuneDeviceConfigurationSecureAssessmentPolicyWindows10", "IntuneDeviceConfigurationSharedMultiDevicePolicyWindows10", "IntuneDeviceConfigurationTrustedCertificatePolicyWindows10", "IntuneDeviceConfigurationVpnPolicyWindows10", "IntuneDeviceConfigurationWindowsTeamPolicyWindows10", "IntuneDeviceConfigurationWiredNetworkPolicyWindows10", "IntuneDeviceEnrollmentLimitRestriction", "IntuneDeviceEnrollmentPlatformRestriction", "IntuneDeviceEnrollmentStatusPageWindows10", "IntuneDeviceRemediation", "IntuneDiskEncryptionMacOS", "IntuneEndpointDetectionAndResponsePolicyWindows10", "IntuneExploitProtectionPolicyWindows10SettingCatalog", "IntunePolicySets", "IntuneRoleAssignment", "IntuneRoleDefinition", "IntuneSettingCatalogASRRulesPolicyWindows10", "IntuneSettingCatalogCustomPolicyWindows10", "IntuneWifiConfigurationPolicyAndroidDeviceAdministrator", "IntuneWifiConfigurationPolicyAndroidEnterpriseDeviceOwner", "IntuneWifiConfigurationPolicyAndroidEnterpriseWorkProfile", "IntuneWifiConfigurationPolicyAndroidForWork", "IntuneWifiConfigurationPolicyAndroidOpenSourceProject", "IntuneWifiConfigurationPolicyIOS", "IntuneWifiConfigurationPolicyMacOS", "IntuneWifiConfigurationPolicyWindows10", "IntuneWindowsAutopilotDeploymentProfileAzureADHybridJoined", "IntuneWindowsAutopilotDeploymentProfileAzureADJoined", "IntuneWindowsInformationProtectionPolicyWindows10MdmEnrolled", "IntuneWindowsUpdateForBusinessDriverUpdateProfileWindows10", "IntuneWindowsUpdateForBusinessFeatureUpdateProfileWindows10", "IntuneWindowsUpdateForBusinessRingUpdateProfileWindows10") -ApplicationId $clientId -TenantId $tenantIdDomainName -ApplicationSecret $clientSecretValue -Path $SavePath -FileName $SaveFileName

Verbose logs showing the problem

No error is presented in the logs.

Environment Information + PowerShell Version

OsName               : Microsoft Windows 11 Enterprise
OsOperatingSystemSKU : EnterpriseEdition
OsArchitecture       : 64-bit
WindowsVersion       : 2009
WindowsBuildLabEx    : 26100.1.amd64fre.ge_release.240331-1435
OsLanguage           : en-US
OsMuiLanguages       : {en-US}

Key   : PSVersion
Value : 5.1.26100.2161
Name  : PSVersion

Key   : PSEdition
Value : Desktop
Name  : PSEdition

Key   : PSCompatibleVersions
Value : {1.0, 2.0, 3.0, 4.0...}
Name  : PSCompatibleVersions

Key   : BuildVersion
Value : 10.0.26100.2161
Name  : BuildVersion

Key   : CLRVersion
Value : 4.0.30319.42000
Name  : CLRVersion

Key   : WSManStackVersion
Value : 3.0
Name  : WSManStackVersion

Key   : PSRemotingProtocolVersion
Value : 2.3
Name  : PSRemotingProtocolVersion

Key   : SerializationVersion
Value : 1.1.0.1
Name  : SerializationVersion
@FabienTschanz
Copy link
Contributor

@rick-engle For any of the policies that are duplicated, do you have verbose logs so we can track what the resource is doing? Do you have a minimal reproducible configuration that always forces the issue?

@rick-engle
Copy link
Author

@FabienTschanz sure thing, let me run the whole thing again for the Intune components, make sure that I then see another row of duplicates in device configurations then share the verbose output here.

@FabienTschanz
Copy link
Contributor

@rick-engle Awesome, thanks a lot.

@rick-engle
Copy link
Author

rick-engle commented Nov 22, 2024

Microsoft365DSC Intune components run logs for GitHub bug report.docx
Ok @FabienTschanz here it is. I did get one duplicate over this entire run which I highlighted below. I wonder if the other duplicates had been generated in earlier builds of Microsoft365DSC as I have been updating a lot to take advantage of bug fixes. I also attached a Word document with the same log info in case the screenshots and RTF formatting is lost here:

<< The logs info I pasted here I guess had a problem as GitHub didn't let me save the comment so check out the attached Word doc instead >>

@FabienTschanz
Copy link
Contributor

@rick-engle Thank you very much for the detailed output. Unfortunately, I can't find anything indicating that the iOS device restriction to block Game Center policy was created another time - The only remnants in the log I found was a call to Test-TargetResource return True, so the updating / creation was skipped. So I'm not sure why that keeps happening. It doesn't seem to be from this DSC run. What happens if you delete all of the duplicate policies and start from zero? What happens after two runs? What are the exact resources that get duplicated?

VERBOSE: [ULTRASBOOK6]: LCM:  [ Start  Resource ]  [[IntuneDeviceConfigurationPolicyIOS]IntuneDeviceConfigurationPolicyiOS-iOS device restriction to block Game Center]
VERBOSE: [ULTRASBOOK6]: LCM:  [ Start  Test     ]  [[IntuneDeviceConfigurationPolicyIOS]IntuneDeviceConfigurationPolicyiOS-iOS device restriction to block Game Center]
VERBOSE: [ULTRASBOOK6]:                            [[IntuneDeviceConfigurationPolicyIOS]IntuneDeviceConfigurationPolicyiOS-iOS device restriction to block Game Center] Testing configuration of {f60c3882-ad7a-4084-9de1-28df20fa1783}
VERBOSE: [ULTRASBOOK6]:                            [[IntuneDeviceConfigurationPolicyIOS]IntuneDeviceConfigurationPolicyiOS-iOS device restriction to block Game Center] Found something with id {f60c3882-ad7a-4084-9de1-28df20fa1783}
VERBOSE: [ULTRASBOOK6]:                            [[IntuneDeviceConfigurationPolicyIOS]IntuneDeviceConfigurationPolicyiOS-iOS device restriction to block Game Center] Count: 1 - 1
VERBOSE: [ULTRASBOOK6]:                            [[IntuneDeviceConfigurationPolicyIOS]IntuneDeviceConfigurationPolicyiOS-iOS device restriction to block Game Center] Group found by groupId or groupDisplayName, checking filters
VERBOSE: [ULTRASBOOK6]:                            [[IntuneDeviceConfigurationPolicyIOS]IntuneDeviceConfigurationPolicyiOS-iOS device restriction to block Game Center] Group and filters match, checking collectionId
....
....
[[IntuneDeviceConfigurationPolicyIOS]IntuneDeviceConfigurationPolicyiOS-iOS device restriction to block Game Center] Test-TargetResource returned True
VERBOSE: [ULTRASBOOK6]: LCM:  [ End    Test     ]  [[IntuneDeviceConfigurationPolicyIOS]IntuneDeviceConfigurationPolicyiOS-iOS device restriction to block Game Center]  in 4.1550 seconds.
VERBOSE: [ULTRASBOOK6]: LCM:  [ Skip   Set      ]  [[IntuneDeviceConfigurationPolicyIOS]IntuneDeviceConfigurationPolicyiOS-iOS device restriction to block Game Center]
VERBOSE: [ULTRASBOOK6]: LCM:  [ End    Resource ]  [[IntuneDeviceConfigurationPolicyIOS]IntuneDeviceConfigurationPolicyiOS-iOS device restriction to block Game Center]

When investigating, I stumbled upon two resources with an issue though. Don't know if you are aware of that, so I'm just putting it here for you:
1)

[[IntuneDeviceConfigurationCustomPolicyWindows10]IntuneDeviceConfigurationCustomPolicyWindows10-USB Device Control (Custom OME-URIs)] Updating the Intune Device Configuration Custom Policy for Windows10 with Id {87e8a692-66cc-49a6-b9d4-1d9b79c180c7}
[BadRequest] : {
  "_version": 3,
  "Message": "XML must be well formed. - Operation ID (for customer support): 00000000-0000-0000-0000-000000000000 - Activity ID: 
b39a25cf-da7f-45a8-b500-2d35967b8d56 - Url: https://fef.amsua0402.manage.microsoft.com/DeviceConfiguration_2411/StatelessDeviceConfigurationFEServi
ce/deviceManagement/deviceConfigurations('87e8a692-66cc-49a6-b9d4-1d9b79c180c7')?api-version=5024-09-04 - CustomApiErrorPhrase: 
DeviceConfigurationValidationErr",
  "CustomApiErrorPhrase": "DeviceConfigurationValidationErr",
  "RetryAfter": null,
  "ErrorSourceService": "",
  "HttpHeaders": "{}"
}
    + CategoryInfo          : InvalidOperation: ({ DeviceConfigu...Configuration }:) [], CimException
    + FullyQualifiedErrorId : BadRequest,Microsoft.Graph.Beta.PowerShell.Cmdlets.UpdateMgBetaDeviceManagementDeviceConfiguration_Update
    + PSComputerName        : localhost

[[IntuneDeviceConfigurationCustomPolicyWindows10]IntuneDeviceConfigurationCustomPolicyWindows10-USB Device Control II] Updating the Intune Device Configuration Custom Policy for Windows10 with Id {3d4df26d-1519-4375-b16c-6a30b9139338}
[BadRequest] : {
  "_version": 3,
  "Message": "XML must be well formed. - Operation ID (for customer support): 00000000-0000-0000-0000-000000000000 - Activity ID: 
30aec5b2-2cb7-440e-8a06-b7b72183f8a1 - Url: https://fef.amsua0402.manage.microsoft.com/DeviceConfiguration_2411/StatelessDeviceConfigurationFEServi
ce/deviceManagement/deviceConfigurations('3d4df26d-1519-4375-b16c-6a30b9139338')?api-version=5024-09-04 - CustomApiErrorPhrase: 
DeviceConfigurationValidationErr",
  "CustomApiErrorPhrase": "DeviceConfigurationValidationErr",
  "RetryAfter": null,
  "ErrorSourceService": "",
  "HttpHeaders": "{}"
}
    + CategoryInfo          : InvalidOperation: ({ DeviceConfigu...Configuration }:) [], CimException
    + FullyQualifiedErrorId : BadRequest,Microsoft.Graph.Beta.PowerShell.Cmdlets.UpdateMgBetaDeviceManagementDeviceConfiguration_Update
    + PSComputerName        : localhost

@rick-engle
Copy link
Author

Thank you @FabienTschanz. I did notice those errors. I did have quite a number of other duplicate policies and some duplicated far more than others as Windows Defender AV Baseline and iOS device restriction to block Game Center had an incredible number of duplicates. I did find this great script below which is helping me easily buly delete the duplicates. I can make sure I have 1 version of each policy left and then see what happens when I do the entire run again:
https://andrewstaylor.com/2022/10/04/bulk-deleting-from-intune/
Again, it could have perhaps been earlier versions of the DSC that had those bugs.
Is there any logic to see if something already exists and if so, what is the decision for if that is overwritten, duplicated or ignored?

Rick

@FabienTschanz
Copy link
Contributor

@rick-engle The logic is the following in all Intune resources:

  • Is there a policy with Id XYZ?
  • If not, is there a policy with display name ABCDE?
  • If not, create the policy
  • If there is either a policy with the id or display name matching, update / remove it (as instructed by the configuration)

That's generally the flow of the resources. Personally, I would start with zero policies from scratch and work my way towards the failure. It could be that there is an incorrect way we're fetching the resources from Graph. That's what we need to find out.

Thank you very much for your testing, really appreciated.

@rick-engle
Copy link
Author

rick-engle commented Nov 22, 2024

Part 2 - Microsoft365DSC Intune components run logs for GitHub bug report.docx
Hello @FabienTschanz, thank you for the details. I removed all of the duplicate policies. My original tenant starts with 20 policies but ends with only 13 policies:
Image

After – still only 13 policies:
Image

The output is again too large so I attached a new Word document.

@FabienTschanz
Copy link
Contributor

Got it. So let's quickly recap the different policies and their respective type:

  • Configure Microsoft Defender Firewall through MDE - Settings catalog
  • Control USB Devices - 04-27-2023 - Settings catalog
  • Deploy Bitlocker to Windows 10 desktops - Settings catalog
  • Device Control Policy - Settings catalog
  • Microsoft Defender Firewall Rules - Settings catalog
  • Microsoft Defender for Endpoint - Win10
  • USB Device Control (Custom OME-URIs) - Custom policy
  • USB Device Control II - Custom policy

One policy exists in the new tenant but not in the old one: iOS device restriction to block Game Center - Device restrictions

Can you tell me how these policies look like in the old tenant? All of them do not show up in the export, that's why they are not being created on your new tenant. A screenshot from one of those policies would be great, then I can rebuild them and test. Also, if you are familiar with your browser's dev tools, you can do the following:

  • Navigate to the policy overview (as in your screenshots)
  • Press F12 to enter the dev tools
  • Click on the Network tab
  • Now select your policy
  • A whole bunch of requests will show up in the network tab - That's what we want
  • Click the "Export HAR" button (looks like a download symbol)
  • Attach the HAR file to your next post

That way, I can take a look at how the policy is structured internally with the request / response from Microsoft Graph. That's the fastest way how we can get to the bottom of these resources and why they are not being exported. Possibly it's a condition in the IntuneSettingCatalogCustomPolicy resource that's now right. We'll have to figure that out.

@FabienTschanz
Copy link
Contributor

@rick-engle Also, I think I know why some resources are being created over and over again. When we switch from one tenant to another, the ID changes, so it does a lookup by display name. But we don't fetch all results from the Graph API, potentially leading to no result, even though it would actually exist. That's what my PR #5456 addresses, but I didn't mention that it is the solution for your issue. Just a part of the puzzle.

@rick-engle
Copy link
Author

Hi @FabienTschanz. Ok I collected all of the information you wanted for 2 of the policies that were being skipped. I'm attaching a Word doc with the text information and screenshots as well as the HAR network captures when opening up the 2 policies. Let me know if Imissed anything. You should have enough info to recreate those policies.Part 3 - Microsoft365DSC - Investigation into Intune policies that are being skipped.docx
intune.microsoft.com_HAR captures.zip
GitHub did not support .HAR files so I zipped them up and attached.

@FabienTschanz
Copy link
Contributor

@rick-engle Awesome, thank you so much for the information. I'll report back with my findings once I'm done checking.

@FabienTschanz
Copy link
Contributor

Okidoki, so let's get started:

  • The first policy Configure Microsoft Defender Firewall through MDE is the resource IntuneFirewallPolicyWindows10. This resource is currently not specified in your components to export.
  • USB Device Control (Custom OME-URIs) is being exported, but can't be imported because of the XML error.

General suggestion: Instead of using the hardcoded list of all components to export, use it like the following:

$resources = Get-M365DSCAllResources | Where-Object -FilterScript { $_-like "Intune*" }
Export-M365DSCConfiguration -Components $resources ...

So if you check the XML (maybe it's also the export that's being done wrong) and fix that, you should be able to export and import all policies. And if one is missing but you got the resources dynamically (with my command), then it is in fact missing and needs a new resource.

Does that help? Hope the export now works for you.

@rick-engle
Copy link
Author

Hi @FabienTschanz that sounds exciting, I'll try your new technique. I've had a number of scenarios where various policies are not being migrated and I wonder if I've missed other resources. I've been using https://export.microsoft365dsc.com/ since the beginning to create the component list that I use for Export-M365DSCConfiguration -Components so maybe its not the best approach. Should I be using something similar to your technique for the other categories of components too to find all of the Active Directory, Security & Compliance components, etc.?
I'll try out your suggestion tomorrow morning!

Thanks!
Rick

@FabienTschanz
Copy link
Contributor

@rick-engle You can try it for the other workloads as well, but there might be even something simpler: Export-M365DSCConfiguration -Workloads @("Intune", "AAD", ...).... This fetches all components dynamically without you having to specify anything.

@rick-engle
Copy link
Author

rick-engle commented Nov 26, 2024

Hi @FabienTschanz. ok I did a new run and sure enough it exported all of the missing policiesM365TenantConfig_M365x648977_Backup.zip
I did get a fail on compile because I had duplicate Device configuration policies. I think those duplicates were incorrectly created in early testing when I didn't update the ConfigurationData.psd1 to have my target tenant domain info. I deleted those duplicates (which oddly had not been detected before) and was able to compile OK. But when I started the Start-DscConfiguration command, it ended with this error which I've never seen before. I'll also upload my .ps1 file:

PS C:\Users\rick\Documents\Azure AD\scripts\Microsoft365DSC_More_AAD> $PathToCompiledMOF = "C:\Users\rick\Azure AD\scripts\Microsoft365DSC_More_AAD\M365TenantConfig_M365x648977_Backup"
Start-DscConfiguration -Path $PathToCompiledMOF -Wait -Verbose -Force
VERBOSE: Perform operation 'Invoke CimMethod' with following parameters, ''methodName' = SendConfigurationApply,'className' = MSFT_DSCLocalConfigura
tionManager,'namespaceName' = root/Microsoft/Windows/DesiredStateConfiguration'.
VERBOSE: An LCM method call arrived from computer U with user sid XXXX.
VERBOSE: The -Force option was specified with the Stop operation. The current configuration has been successfully cancelled.
VERBOSE: An LCM method call arrived from computer computer U with user sid XXXX.
VERBOSE: [U]: LCM: [ Start Set ]
VERBOSE: [U]: LCM: [ End Set ]
The command Test-TargetResource of the PS module MSFT_IntuneDeviceManagementEnrollmentAndroidGooglePlay does not implement the write property
OwnerUserPrincipalName mentioned in the corresponding MOF schema file C:\Program Files\WindowsPowerShell\Modules\Microsoft365DSC\1.24.1120.1\DscRes
ources\MSFT_IntuneDeviceManagementEnrollmentAndroidGooglePlay\MSFT_IntuneDeviceManagementEnrollmentAndroidGooglePlay.schema.mof. All write
paramenters mentioned in the schema file must be implemented by the command Test-TargetResource.
+ CategoryInfo : InvalidOperation: (root/Microsoft/...gurationManager:String) [], CimException
+ FullyQualifiedErrorId : WriteParameterNotImplemented
+ PSComputerName : localhost

VERBOSE: Operation 'Invoke CimMethod' complete.
VERBOSE: Time taken for configuration job to complete is 6.76 seconds

@ricmestre
Copy link
Contributor

This resource IntuneDeviceManagementEnrollmentAndroidGooglePlay was merged without having all the properties defined in its schema also defined in the Test method of the module, they're also missing from Get and Set and are all commented out.

@ykuijs @NikCharlebois And this is why I'm very concerned, and frustrated, of seeing all these new resources popping up which are being merged without actually being thoroughly tested and then trivial problems like these are randomly discovered because of something else.

@FabienTschanz
Copy link
Contributor

@ricmestre That's what I was just about to write, stumbled upon it. I guess this was never executed, otherwise it would have been detected right away.

@rick-engle
Copy link
Author

@ricmestre , @FabienTschanz , should I then just edit my .ps1 file and remove this block for now?
IntuneDeviceManagementEnrollmentAndroidGooglePlay "IntuneDeviceManagementEnrollmentAndroidGooglePlay-androidManagedStoreAccountEnterpriseSettings"
{
ApplicationId = $ConfigurationData.NonNodeData.ApplicationId;
ApplicationSecret = New-Object System.Management.Automation.PSCredential ('ApplicationSecret', (ConvertTo-SecureString $ConfigurationData.NonNodeData.ApplicationSecret -AsPlainText -Force));
BindStatus = "boundAndValidated";
Ensure = "Present";
Id = "androidManagedStoreAccountEnterpriseSettings";
TenantId = $OrganizationName;
}

@ricmestre
Copy link
Contributor

@rick-engle Yes please, let's see how much far you can get into it, you might find other issues beware, here be dragons

@FabienTschanz
Copy link
Contributor

@rick-engle Until we have a solution for that, yes. Not quite sure how we should proceed. @ricmestre Do you want me to simply remove the properties from the schema.mof? That would be the easiest solution, but might trigger a breaking change... Although the properties would have been ignored anyway.

@ricmestre
Copy link
Contributor

@FabienTschanz My opinion is that this resource should not be even available until it works

@rick-engle
Copy link
Author

@FabienTschanz this seems like it could be a side effect of using $resources = Get-M365DSCAllResources | Where-Object -FilterScript { $_-like "Intune*" } as it may populate components that are not yet ready? Can that FilterScript syntax be modified so that it has -like "Intune*" but also has an exception for "IntuneDeviceManagementEnrollmentAndroidGooglePlay"? That way I can just run the export again filtering out that bad component and generate all of the successive files without a need to manually edit any of the output files?

@FabienTschanz
Copy link
Contributor

@rick-engle The issue would appear also if you were using -Workload @("Intune"). What you can do is define an exclude array with components and exclude the components from the final ones like the following:

$resources = Get-M365DSCAllResources | Where-Object -FilterScript { $_-like "Intune*" }
$excludeResources = @("IntuneDeviceManagementEnrollmentAndroidGooglePlay")
$finalResources = $resources | Where-Object { $_ -notin $excludeResources }

@rick-engle
Copy link
Author

rick-engle commented Nov 26, 2024

Ok @FabienTschanz good progress! I used this code before seeing your suggestion:

$resources = Get-M365DSCAllResources | Where-Object -FilterScript { $_ -like "Intune*" -and $_ -notlike "IntuneDeviceManagementEnrollmentAndroidGooglePlay" }
Export-M365DSCConfiguration -Components $resources -ApplicationId $clientId -TenantId $tenantIdDomainName -ApplicationSecret $clientSecretValue -Path $SavePath -FileName $SaveFileName

And then after the migration steps. I went from 20 policies in the original tenant to 18 in the new.
There were all of the ones missed:
Image

That actually makes 18 + 3 = 21 policies but there was 1 policy for iOS that was in the new but not in the original tenant which makes sense.

I attached the verbose output with errors highlighted.
By the way the logic definitely doesn't like policies with the same name. There are several Device Enrollment Platform policies, each for a different platform but they have the same name and therefor throw errors like these:
The PowerShell DSC resource '[IntuneDeviceEnrollmentPlatformRestriction]IntuneDeviceEnrollmentPlatformRestriction-Block personal devices from
enrolling-3' with SourceInfo 'C:\Users\rick\Azure
AD\scripts\Microsoft365DSC_More_AAD\M365TenantConfig_M365x648977_Backup.ps1::2177::9::IntuneDeviceEnrollmentPlatformRestriction' threw one or more
non-terminating errors while running the Set-TargetResource functionality. These errors are logged to the ETW channel called
Microsoft-Windows-DSC/Operational. Refer to this channel for more details.
+ CategoryInfo : InvalidOperation: (:) [], CimException
+ FullyQualifiedErrorId : NonTerminatingErrorFromProvider
+ PSComputerName : localhost

Here is a combo screenshot:
Image

Part 4 - Verbose output with errors from Start-DscConfiguration command.docx

@FabienTschanz
Copy link
Contributor

@rick-engle Yes, the display name is the key property to make policies unique per tenant. That's the only way we can guarantee that they can also be created in another tenant, where the id of the policy would always be different.

Am I right that all resources you now used are in your previous .ps1 script which you shared? Some resources (e.g. the IntuneWindowsUpdateForBusinessFeatureUpdateProfileWindows10 resource) have an incorrect value specified. That's something you can see and update for yourself, not really an issue from the logic. Some things can't be checked up front and will throw a runtime issue. For the other errors, I'll take a look at it in the future.

@rick-engle
Copy link
Author

@FabienTschanz , ok thanks. I see at least one workaround by renaming my policies with duplicative names. I wish the verbose output provided self-service hints as to what we might do or look at fixing in the source policies. Intune didn't complain when saving those original policies so it's interesting that the export uncovers issues not seen by the Intune console.
By the way if I run the Start-DscConfiguration command without Verbose, will I still see the errors and which policy/component threw that error? That would make it a lot easier to sift through all of the output.

@FabienTschanz
Copy link
Contributor

You should still see the errors even without -Verbose. They are still errors 😄 The Intune portal itself does a bunch of validation and prevents saving invalid input, something Microsoft365DSC doesn't (or better said: in selected places).

@ricmestre
Copy link
Contributor

The android restrictions are split in 2 and they MUST have the same name, then the Windows restriction has another separate one, that's why the error message appears on Block personal devices from enrolling-3 with the ending "-3", just change the Windows restriction policy name to fix it.

@rick-engle
Copy link
Author

Hi @ricmestre that took me a bit to realize that the -number usage in the verbose update was not part of the actual name of policies but a hint that those policies were duplicates and the numbers were added as a distinguisher.

@rick-engle
Copy link
Author

I guess combing through the verbose output, BadRequests are errors based on a faulty value/configuration on the Intune side while "XXX threw one or more non-terminating errors while running XXX" are errors caught trying to process the job. The first can be corrected by fixing the source policy in Intune (if you can figure out what) and the 2nd may not be fixed due to a bug?
@FabienTschanz , @ricmestre

@FabienTschanz
Copy link
Contributor

@rick-engle BadRequests simply means that a provided value was not correct, either one from the configuration or through the code which connects to Microsoft Graph or any other service. XXX threw one or more non-terminating errors is the subsequent error message from the DSC engine itself that an error occurred. This can be because of a BadRequest, but it's not limited to such errors,

@FabienTschanz
Copy link
Contributor

FabienTschanz commented Nov 27, 2024

@rick-engle I had a look at your configuration:

  • iOS apps data leak prevention policy (IntuneAppProtectionPolicyiOS) - Is it possible that the assignment is no longer valid? It exported a GroupId, but no corresponding display name. Does the referenced group still exist?
  • USB Device Control (Custom OME-URIs) - Seems like the XML in the policy is invalid (or not exported correctly). Can you share the export of it?
  • USB Device Control II - Same as above probaby
  • Microsoft Defender for Endpoint - Win10 (IntuneDeviceConfigurationDefenderForEndpointOnboardingPolicyWindows10) - I think the onboarding blob specified is too old, thus throwing the error
  • Any IntuneDeviceEnrollmentPlatformRestriction currently throws an exception when trying to create or update. This is due to a bug at Microsoft for which I opened a ticket, see [IntuneDeviceEnrollmentPlatformRestriction] Tenant is not Global Admin or Intune Service Admin. Operation is restricted. #5127
  • RicksAutopilotEnrollmentStatusPage (IntuneDeviceEnrollmentStatusPageWindows10) - Unable to reproduce. Please share your configuration.
  • Update stale Group Policies (IntuneDeviceRemediation) - Limit of 200 device remediations reached. Please check and delete the duplicates.
  • Office 365 Desktop (IntuneMobileAppsWindowsOfficeSuiteApp) - You have to specify a "valid office default file format version". No idea what that means, can you share your configuration? Or create a new policy, export and import again? Maybe something changed?
  • MobileThreatDefenseConnector - PR Fix NotFound error on Get and comparison #5471 is open. Another one where the resource probably was never executed
  • Edge Policies-Edge for iOS & Android 1.16.2020_12:21:11 (IntunePolicySets) - Don't know, the configuration would help. Although personally I would refrain from using policy sets alltogether, they aren't being developed anymore.
  • Assign to remote help (IntuneRoleAssignment) - EDIT 2 --> Fix Intune role assignment and cloud login variable #5473
  • Autopilot Ultra (IntuneWindowsAutopilotDeploymentProfileAzureADJoined) - There seems to be some invalid input. What's the configuration?
  • Restrict Windows 10 updates to 21H2 (IntuneWindowsUpdateForBusinessFeatureUpdateProfileWindows10) - Windows 10, version 21H2 is an unsupported value. You cannot create a policy from a previous Windows feature update. Remove the policy or specify a correct Windows version.

@ricmestre
Copy link
Contributor

@rick-engle @FabienTschanz The issue with IntuneMobileAppsWindowsOfficeSuiteApp is easy, I also stumbled upon this issue myself, see #5253, this is because the policy Office 365 Desktop is old and was created at a time when property officeSuiteAppDefaultFileFormat was still not mandatory, now that it is you just need to define it as officeOpenXMLFormat, recommended, or officeOpenDocumentFormat

@FabienTschanz
Copy link
Contributor

Can't believe it, just because @rick-engle did an export and reimport of his settings, I created two PRs to address a number of issues... We absolutely need a better pipeline than what currently is implemented. Many of those things would be caught if some real examples would be run.

@ricmestre
Copy link
Contributor

@FabienTschanz That's why I currently have created more than 600 individual examples which I use for testing my parsers, which indirectly also test ReverseDSC, DSCParser and Get-M365DSCExportContentForResource from M365DSCUtil, and I use those same examples in my integration tests in a real tenant which they all need to pass Test-DscConfiguration.

It would not be easy integrating my tests in M365DSC the way they're done, I've been discussing this with Nik and Yorick for a few months now to start somewhere but as you might have noticed the integration pipelines here in the project are not working due to problems with their tenant so while this is not solved someone on their side must do the tests themselves manually like I've been doing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants