-
Notifications
You must be signed in to change notification settings - Fork 503
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Exporting Intune device configurations to a new tenant creates duplicate device configuration entries each time it is run #5447
Comments
@rick-engle For any of the policies that are duplicated, do you have verbose logs so we can track what the resource is doing? Do you have a minimal reproducible configuration that always forces the issue? |
@FabienTschanz sure thing, let me run the whole thing again for the Intune components, make sure that I then see another row of duplicates in device configurations then share the verbose output here. |
@rick-engle Awesome, thanks a lot. |
Microsoft365DSC Intune components run logs for GitHub bug report.docx << The logs info I pasted here I guess had a problem as GitHub didn't let me save the comment so check out the attached Word doc instead >> |
@rick-engle Thank you very much for the detailed output. Unfortunately, I can't find anything indicating that the
When investigating, I stumbled upon two resources with an issue though. Don't know if you are aware of that, so I'm just putting it here for you:
|
Thank you @FabienTschanz. I did notice those errors. I did have quite a number of other duplicate policies and some duplicated far more than others as Windows Defender AV Baseline and iOS device restriction to block Game Center had an incredible number of duplicates. I did find this great script below which is helping me easily buly delete the duplicates. I can make sure I have 1 version of each policy left and then see what happens when I do the entire run again: Rick |
@rick-engle The logic is the following in all Intune resources:
That's generally the flow of the resources. Personally, I would start with zero policies from scratch and work my way towards the failure. It could be that there is an incorrect way we're fetching the resources from Graph. That's what we need to find out. Thank you very much for your testing, really appreciated. |
Part 2 - Microsoft365DSC Intune components run logs for GitHub bug report.docx After – still only 13 policies: The output is again too large so I attached a new Word document. |
Got it. So let's quickly recap the different policies and their respective type:
One policy exists in the new tenant but not in the old one: iOS device restriction to block Game Center - Device restrictions Can you tell me how these policies look like in the old tenant? All of them do not show up in the export, that's why they are not being created on your new tenant. A screenshot from one of those policies would be great, then I can rebuild them and test. Also, if you are familiar with your browser's dev tools, you can do the following:
That way, I can take a look at how the policy is structured internally with the request / response from Microsoft Graph. That's the fastest way how we can get to the bottom of these resources and why they are not being exported. Possibly it's a condition in the |
@rick-engle Also, I think I know why some resources are being created over and over again. When we switch from one tenant to another, the ID changes, so it does a lookup by display name. But we don't fetch all results from the Graph API, potentially leading to no result, even though it would actually exist. That's what my PR #5456 addresses, but I didn't mention that it is the solution for your issue. Just a part of the puzzle. |
Hi @FabienTschanz. Ok I collected all of the information you wanted for 2 of the policies that were being skipped. I'm attaching a Word doc with the text information and screenshots as well as the HAR network captures when opening up the 2 policies. Let me know if Imissed anything. You should have enough info to recreate those policies.Part 3 - Microsoft365DSC - Investigation into Intune policies that are being skipped.docx |
@rick-engle Awesome, thank you so much for the information. I'll report back with my findings once I'm done checking. |
Okidoki, so let's get started:
General suggestion: Instead of using the hardcoded list of all components to export, use it like the following: $resources = Get-M365DSCAllResources | Where-Object -FilterScript { $_-like "Intune*" }
Export-M365DSCConfiguration -Components $resources ... So if you check the XML (maybe it's also the export that's being done wrong) and fix that, you should be able to export and import all policies. And if one is missing but you got the resources dynamically (with my command), then it is in fact missing and needs a new resource. Does that help? Hope the export now works for you. |
Hi @FabienTschanz that sounds exciting, I'll try your new technique. I've had a number of scenarios where various policies are not being migrated and I wonder if I've missed other resources. I've been using https://export.microsoft365dsc.com/ since the beginning to create the component list that I use for Export-M365DSCConfiguration -Components so maybe its not the best approach. Should I be using something similar to your technique for the other categories of components too to find all of the Active Directory, Security & Compliance components, etc.? Thanks! |
@rick-engle You can try it for the other workloads as well, but there might be even something simpler: |
Hi @FabienTschanz. ok I did a new run and sure enough it exported all of the missing policies PS C:\Users\rick\Documents\Azure AD\scripts\Microsoft365DSC_More_AAD> $PathToCompiledMOF = "C:\Users\rick\Azure AD\scripts\Microsoft365DSC_More_AAD\M365TenantConfig_M365x648977_Backup" VERBOSE: Operation 'Invoke CimMethod' complete. |
This resource IntuneDeviceManagementEnrollmentAndroidGooglePlay was merged without having all the properties defined in its schema also defined in the Test method of the module, they're also missing from Get and Set and are all commented out. @ykuijs @NikCharlebois And this is why I'm very concerned, and frustrated, of seeing all these new resources popping up which are being merged without actually being thoroughly tested and then trivial problems like these are randomly discovered because of something else. |
@ricmestre That's what I was just about to write, stumbled upon it. I guess this was never executed, otherwise it would have been detected right away. |
@ricmestre , @FabienTschanz , should I then just edit my .ps1 file and remove this block for now? |
@rick-engle Yes please, let's see how much far you can get into it, you might find other issues beware, here be dragons |
@rick-engle Until we have a solution for that, yes. Not quite sure how we should proceed. @ricmestre Do you want me to simply remove the properties from the schema.mof? That would be the easiest solution, but might trigger a breaking change... Although the properties would have been ignored anyway. |
@FabienTschanz My opinion is that this resource should not be even available until it works |
@FabienTschanz this seems like it could be a side effect of using $resources = Get-M365DSCAllResources | Where-Object -FilterScript { $_-like "Intune*" } as it may populate components that are not yet ready? Can that FilterScript syntax be modified so that it has -like "Intune*" but also has an exception for "IntuneDeviceManagementEnrollmentAndroidGooglePlay"? That way I can just run the export again filtering out that bad component and generate all of the successive files without a need to manually edit any of the output files? |
@rick-engle The issue would appear also if you were using $resources = Get-M365DSCAllResources | Where-Object -FilterScript { $_-like "Intune*" }
$excludeResources = @("IntuneDeviceManagementEnrollmentAndroidGooglePlay")
$finalResources = $resources | Where-Object { $_ -notin $excludeResources } |
Ok @FabienTschanz good progress! I used this code before seeing your suggestion: $resources = Get-M365DSCAllResources | Where-Object -FilterScript { $_ -like "Intune*" -and $_ -notlike "IntuneDeviceManagementEnrollmentAndroidGooglePlay" } And then after the migration steps. I went from 20 policies in the original tenant to 18 in the new. That actually makes 18 + 3 = 21 policies but there was 1 policy for iOS that was in the new but not in the original tenant which makes sense. I attached the verbose output with errors highlighted. Part 4 - Verbose output with errors from Start-DscConfiguration command.docx |
@rick-engle Yes, the display name is the key property to make policies unique per tenant. That's the only way we can guarantee that they can also be created in another tenant, where the id of the policy would always be different. Am I right that all resources you now used are in your previous .ps1 script which you shared? Some resources (e.g. the |
@FabienTschanz , ok thanks. I see at least one workaround by renaming my policies with duplicative names. I wish the verbose output provided self-service hints as to what we might do or look at fixing in the source policies. Intune didn't complain when saving those original policies so it's interesting that the export uncovers issues not seen by the Intune console. |
You should still see the errors even without -Verbose. They are still errors 😄 The Intune portal itself does a bunch of validation and prevents saving invalid input, something Microsoft365DSC doesn't (or better said: in selected places). |
The android restrictions are split in 2 and they MUST have the same name, then the Windows restriction has another separate one, that's why the error message appears on Block personal devices from enrolling-3 with the ending "-3", just change the Windows restriction policy name to fix it. |
Hi @ricmestre that took me a bit to realize that the -number usage in the verbose update was not part of the actual name of policies but a hint that those policies were duplicates and the numbers were added as a distinguisher. |
I guess combing through the verbose output, BadRequests are errors based on a faulty value/configuration on the Intune side while "XXX threw one or more non-terminating errors while running XXX" are errors caught trying to process the job. The first can be corrected by fixing the source policy in Intune (if you can figure out what) and the 2nd may not be fixed due to a bug? |
@rick-engle BadRequests simply means that a provided value was not correct, either one from the configuration or through the code which connects to Microsoft Graph or any other service. XXX threw one or more non-terminating errors is the subsequent error message from the DSC engine itself that an error occurred. This can be because of a BadRequest, but it's not limited to such errors, |
@rick-engle I had a look at your configuration:
|
@rick-engle @FabienTschanz The issue with IntuneMobileAppsWindowsOfficeSuiteApp is easy, I also stumbled upon this issue myself, see #5253, this is because the policy Office 365 Desktop is old and was created at a time when property officeSuiteAppDefaultFileFormat was still not mandatory, now that it is you just need to define it as officeOpenXMLFormat, recommended, or officeOpenDocumentFormat |
Can't believe it, just because @rick-engle did an export and reimport of his settings, I created two PRs to address a number of issues... We absolutely need a better pipeline than what currently is implemented. Many of those things would be caught if some real examples would be run. |
@FabienTschanz That's why I currently have created more than 600 individual examples which I use for testing my parsers, which indirectly also test ReverseDSC, DSCParser and Get-M365DSCExportContentForResource from M365DSCUtil, and I use those same examples in my integration tests in a real tenant which they all need to pass Test-DscConfiguration. It would not be easy integrating my tests in M365DSC the way they're done, I've been discussing this with Nik and Yorick for a few months now to start somewhere but as you might have noticed the integration pipelines here in the project are not working due to problems with their tenant so while this is not solved someone on their side must do the tests themselves manually like I've been doing. |
Description of the issue
As I was using Microsoft365DSC to migrate settings from my old to my new tenant, I noticed that fortunately the majority of settings imported were not duplicated with each run however Intune Device Configurations are being duplicated:
This should be a screenshot that illustrates the problem in Intune.
The command being used to export my Intune components is:
Export-M365DSCConfiguration -Components @("IntuneAccountProtectionLocalAdministratorPasswordSolutionPolicy", "IntuneAccountProtectionLocalUserGroupMembershipPolicy", "IntuneAccountProtectionPolicy", "IntuneAntivirusPolicyWindows10SettingCatalog", "IntuneAppConfigurationDevicePolicy", "IntuneAppConfigurationPolicy", "IntuneApplicationControlPolicyWindows10", "IntuneAppProtectionPolicyAndroid", "IntuneAppProtectionPolicyiOS", "IntuneASRRulesPolicyWindows10", "IntuneAttackSurfaceReductionRulesPolicyWindows10ConfigManager", "IntuneDeviceAndAppManagementAssignmentFilter", "IntuneDeviceCategory", "IntuneDeviceCleanupRule", "IntuneDeviceCompliancePolicyAndroid", "IntuneDeviceCompliancePolicyAndroidDeviceOwner", "IntuneDeviceCompliancePolicyAndroidWorkProfile", "IntuneDeviceCompliancePolicyiOs", "IntuneDeviceCompliancePolicyMacOS", "IntuneDeviceCompliancePolicyWindows10", "IntuneDeviceConfigurationAdministrativeTemplatePolicyWindows10", "IntuneDeviceConfigurationCustomPolicyWindows10", "IntuneDeviceConfigurationDefenderForEndpointOnboardingPolicyWindows10", "IntuneDeviceConfigurationDeliveryOptimizationPolicyWindows10", "IntuneDeviceConfigurationDomainJoinPolicyWindows10", "IntuneDeviceConfigurationEmailProfilePolicyWindows10", "IntuneDeviceConfigurationEndpointProtectionPolicyWindows10", "IntuneDeviceConfigurationFirmwareInterfacePolicyWindows10", "IntuneDeviceConfigurationHealthMonitoringConfigurationPolicyWindows10", "IntuneDeviceConfigurationIdentityProtectionPolicyWindows10", "IntuneDeviceConfigurationImportedPfxCertificatePolicyWindows10", "IntuneDeviceConfigurationKioskPolicyWindows10", "IntuneDeviceConfigurationNetworkBoundaryPolicyWindows10", "IntuneDeviceConfigurationPkcsCertificatePolicyWindows10", "IntuneDeviceConfigurationPlatformScriptMacOS", "IntuneDeviceConfigurationPlatformScriptWindows", "IntuneDeviceConfigurationPolicyAndroidDeviceAdministrator", "IntuneDeviceConfigurationPolicyAndroidDeviceOwner", "IntuneDeviceConfigurationPolicyAndroidOpenSourceProject", "IntuneDeviceConfigurationPolicyAndroidWorkProfile", "IntuneDeviceConfigurationPolicyiOS", "IntuneDeviceConfigurationPolicyMacOS", "IntuneDeviceConfigurationPolicyWindows10", "IntuneDeviceConfigurationSCEPCertificatePolicyWindows10", "IntuneDeviceConfigurationSecureAssessmentPolicyWindows10", "IntuneDeviceConfigurationSharedMultiDevicePolicyWindows10", "IntuneDeviceConfigurationTrustedCertificatePolicyWindows10", "IntuneDeviceConfigurationVpnPolicyWindows10", "IntuneDeviceConfigurationWindowsTeamPolicyWindows10", "IntuneDeviceConfigurationWiredNetworkPolicyWindows10", "IntuneDeviceEnrollmentLimitRestriction", "IntuneDeviceEnrollmentPlatformRestriction", "IntuneDeviceEnrollmentStatusPageWindows10", "IntuneDeviceRemediation", "IntuneDiskEncryptionMacOS", "IntuneEndpointDetectionAndResponsePolicyWindows10", "IntuneExploitProtectionPolicyWindows10SettingCatalog", "IntunePolicySets", "IntuneRoleAssignment", "IntuneRoleDefinition", "IntuneSettingCatalogASRRulesPolicyWindows10", "IntuneSettingCatalogCustomPolicyWindows10", "IntuneWifiConfigurationPolicyAndroidDeviceAdministrator", "IntuneWifiConfigurationPolicyAndroidEnterpriseDeviceOwner", "IntuneWifiConfigurationPolicyAndroidEnterpriseWorkProfile", "IntuneWifiConfigurationPolicyAndroidForWork", "IntuneWifiConfigurationPolicyAndroidOpenSourceProject", "IntuneWifiConfigurationPolicyIOS", "IntuneWifiConfigurationPolicyMacOS", "IntuneWifiConfigurationPolicyWindows10", "IntuneWindowsAutopilotDeploymentProfileAzureADHybridJoined", "IntuneWindowsAutopilotDeploymentProfileAzureADJoined", "IntuneWindowsInformationProtectionPolicyWindows10MdmEnrolled", "IntuneWindowsUpdateForBusinessDriverUpdateProfileWindows10", "IntuneWindowsUpdateForBusinessFeatureUpdateProfileWindows10", "IntuneWindowsUpdateForBusinessRingUpdateProfileWindows10") -ApplicationId $clientId -TenantId $tenantIdDomainName -ApplicationSecret $clientSecretValue -Path $SavePath -FileName $SaveFileName
Microsoft 365 DSC Version
1.24.1120.1
Which workloads are affected
Intune
The DSC configuration
Verbose logs showing the problem
No error is presented in the logs.
Environment Information + PowerShell Version
The text was updated successfully, but these errors were encountered: