A recent update to dotnet/msbuild completely broke the CI/CD of my Blazor Server project (Workflows, Docker)

[SparkyTD]

I want to talk about a weird issue that has been taking place in one of my projects for the past couple of days.

I'm currently working on a Blazor Server project that has GitHub Workflows for both building/testing the project, and also building/publishing a docker image to a private repository. I have a workflow job that triggers on every commit to master, and it tries to build the project against the dotnet 6.0.x SDK.

As of about 5 days ago, I noticed that the build workflow started to fail with a completely nonsensical compile error that I couldn't reproduce on my workstation. It basically complained about an unresolved reference to a generic Blazor Component that certain pages were using. This was a weird error, because I was able to compile the exact same codebase on my workstation without any issues, and I also verified that there were no reference issues regarding that component in question.

Since none of the recent commits had anything to do with this error, I started examining the build logs of both the older successful runs, and comparing them with the newer failed ones. What I noticed is that the last successful build was using version 17.5.1+f6fdcf537 of MSBuild, but a couple days ago it was sneakily updated to 17.6.1+8ffc3fe3d, which coincides with the first failed build. A couple days later the issue had propagated over to Microsoft's Docker registries, and the image build tasks also started to fail with the exact same compile errors. It's important to note that I haven't changed the version of dotnet in at least 5 months, so this change was introduced by whoever maintains the GitHub Runs and the Docker Images of the Dotnet SDK.

The only emergency solution that I was able to implement was to update my workflows from dotnet 6.0.x to dotnet 8.0-preview, which now uses version 17.7.0-preview-23251-02+59879b095 of MSBuild, and compiles my project with no errors. Although this seems to work for now, I am not exactly comfortable using an unreleased preview version of the SDK, and I'm worried that this could introduce even more issues similar to this one in the short-term future.

---

Has anyone else experienced similar issues with Blazor Server projects in the past few days? Is there some sort of known bug/regression in MSBuild 17.6.1 that might be causing errors like this? Could the project maintainers give more information regarding recent updates to the mcr.microsoft.com/dotnet/aspnet:6.0 Docker registry and the actions/setup-dotnet@v3 action?

[baronfel]

Can you provide binlogs of the builds that error in this new way? 17.6 versions of MSBuild map to the .NET SDK version 7.0.300, which released in May. I would expect that if you were somehow using 7.0 images, or making use of managed build agents without explicitly managing your SDK versions then you would see this kind of version floating.

[SparkyTD]

I've reproduced the error in Docker by using mcr.microsoft.com/dotnet/sdk:7.0 and mcr.microsoft.com/dotnet/aspnet:7.0 as the base.

msbuild.binlog.zip

[SparkyTD]

Also, it appears that I've made a mistake in my post. The Dockerfile was referencing 7.0 base images when the errors started happening, and not 6.0. The github workflow was referencing 6.0.x. Interestingly, the issue started to appear in both at around the same time, with a delay of only 2-3 days. I am unable to provide binlogs from the first failed workflow run, but after inspecting the logs it seem like it was using 6.0.408 with MSBuild 17.6.1+8ffc3fe3d, and produced the same errors as the Docker image on 7.0.

I'm using GitHub's hosted runners, so I'm not managing the SDK. I wonder why GitHub used an MSBuild version belonging to 7.0, if the specified SDK version was 6.0.

[GangWang01]

@SparkyTD .NET 8.0 is GA now. Do you still see the issue? If yes, can you provide the binlog for our investigation.

[SparkyTD]

Hello, I am no longer experiencing the issue.