Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate symbols for all Go toolchain binaries #1210

Merged
merged 4 commits into from
Apr 29, 2024

Conversation

qmuntal
Copy link
Member

@qmuntal qmuntal commented Apr 26, 2024

Note

This PR uses a PDB generation tool, gopdb, which is still experimental and not available publicly.

This PR adds support for publishing symbols in the form of PDBs. It is enabled by default for the rolling-internal pipeline.

This is a summary of the changes:

  • Download and build gopdb from the internal AzDO mirror.
  • Update the build command to run the gopdb tool for all the Go toolchain binaries.
  • Store all PDBs as pipeline artifacts
  • Update the publish stage to also publish the PDB files using PublishSymbols@2. Symbols are published into the private dnceng symbol server, which is the only server that our pipeline has access to.
  • To achieve the previous task, I had to move the publish-stage.yml execution to builders-to-stages.yml, as it now needs to know which builders were run in order to download the symbol artifacts they published.

See these changes in action in https://dev.azure.com/dnceng/internal/_build/results?buildId=2439083&view=results.

@qmuntal qmuntal requested review from dagood and gdams April 26, 2024 09:31
@qmuntal qmuntal requested a review from a team as a code owner April 26, 2024 09:31
eng/pipeline/stages/builders-to-stages.yml Outdated Show resolved Hide resolved
eng/_core/cmd/build/build.go Outdated Show resolved Hide resolved
eng/pipeline/stages/builders-to-stages.yml Outdated Show resolved Hide resolved
eng/pipeline/stages/publish-stage.yml Outdated Show resolved Hide resolved
eng/pipeline/stages/publish-stage.yml Outdated Show resolved Hide resolved
@@ -124,16 +138,21 @@ stages:
- pwsh: Write-Host "##vso[task.setvariable variable=GOARM]6"
displayName: Set GOARM for cross-compile

- ${{ if eq(variables.createPDB, true) }}:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting... wasn't aware job variables would be accessible here. (I experienced pipeline variables not being accessible inside a stage template in go-images, but it seems now that that's because that stage's variables are the ones that are in scope at that point, excluding the outer variables.)

https://learn.microsoft.com/en-us/azure/devops/pipelines/process/expressions?view=azure-devops#variables seems very vague about it, but another thing it mentions is that they're evaluated in order... so maybe some of the value reuse we're currently doing through layers of templates could be done this way too.

No suggestion to change anything now, just some context about why this isn't showing up anywhere else. 😄

eng/pipeline/steps/install-gopdb.yml Outdated Show resolved Hide resolved
eng/pipeline/steps/install-gopdb.yml Outdated Show resolved Hide resolved
@qmuntal qmuntal requested a review from dagood April 29, 2024 08:17
@dagood dagood merged commit ee78796 into microsoft/main Apr 29, 2024
23 checks passed
@dagood dagood deleted the dev/qmuntal/azdo2 branch April 29, 2024 18:39
dagood added a commit that referenced this pull request Jun 7, 2024
Generate symbols for all Go toolchain binaries

(cherry picked from commit
ee78796 with
conflict resolution due to not using distpack)
dagood added a commit that referenced this pull request Jun 7, 2024
Generate symbols for all Go toolchain binaries

(cherry picked from commit ee78796)
dagood added a commit that referenced this pull request Jun 7, 2024
Generate symbols for all Go toolchain binaries

(cherry picked from commit
ee78796 with
conflict resolution due to not using distpack)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants