Update docs to v4.0.0

PostBuildCleanup/en-US/changeNotes.md

@@ -2,8 +2,12 @@
 
 # Post Build Cleanup - Change Notes
 
+#### 4.0.0
+- Update task to Node 16 (the task now supports Node 10, Node 16, and Node 20).
+- Small fix to make the task work better with YAML pipelines (see [YAML Guidance](./yamlGuidance.md) for guidance on cleanup and YAML pipelines).
+
 #### 3.1.1
-- Bump version of the azure-pipelines-task-lib to get rid of NodeJS warning `Warning: Use Cipheriv for counter mode of aes-256-ctr`
+- Bump version of the azure-pipelines-task-lib to get rid of NodeJS warning `Warning: Use Cipheriv for counter mode of aes-256-ctr`.
 
 #### 3.1.0
 - Fix issue with disabling NodeJS certificate check.

PostBuildCleanup/en-US/cleanupRepository.yml

@@ -0,0 +1,127 @@
+# This steps template is meant to replace the Post Build Cleanup task in YAML pipelines.
+
+parameters:
+  # This parameter takes the path to your cloned repository (e.g., '$(Build.SourcesDirectory)' or '$(Build.SourcesDirectory)/{repository name}').
+  # If you have only one repository, leave the default (i.e., don't specify the parameter). If you have multiple repositories, use the value
+  # '$(Build.SourcesDirectory)/{repository name}' (replace the placeholder in curly brackets). If you specified the path parameter in your
+  # checkout steps, use the same value for repositoryPath.
+  - name: repositoryPath
+    displayName: 'The path into which the repository was cloned'
+    type: string
+    default: '$(Build.SourcesDirectory)'
+
+  # This parameter takes the clean option similar to the old classic UI. Valid values are:
+  #   - sources: cleans only the repository path
+  #   - sourcesAndOutput: cleans the repository path and the $(Build.BinariesDirectory)
+  #   - sourcesDirectory: deletes all contents from the repository path
+  #   - all: empties all standard directories in the pipelin workspace
+  - name: cleanOption
+    displayName: 'The clean option to use, similar to the old classic UI.'
+    type: string
+    default: sources
+    values:
+      - sources
+      - sourcesAndOutput
+      - sourcesDirectory
+      - all
+
+  # This parameter must be set to true if checkout.submodules for your repository is set to true or recursive. Otherwise it should be false.
+  - name: checkoutSubmodules
+    displayName: 'True if submodules have been checked out, otherwise false'
+    type: boolean
+    default: false
+
+steps:
+  - pwsh: |
+      function Cleanup-GitFolder($path) {
+        Write-Host "Cleaning up git folder $path"
+        if (Test-Path -Path $path -PathType Container) {
+          $fsEntries = Get-ChildItem -Path $path -Directory
+
+          foreach ($fsEntry in $fsEntries) {
+              Remove-Item -Path "$($fsEntry.FullName)\*" -Recurse -Force
+          }
+        }
+      }
+
+      function Delete-AllContent($path, $keepDirs, $exclusions) {
+        Write-Host "Deleting all content from $path"
+        $fsEntries = Get-ChildItem -Path $path | Where-Object { $exclusions -notcontains $_.Name }
+
+        foreach ($fsEntry in $fsEntries) {
+          if ($fsEntry.PSIsContainer) {
+            if ($fsEntry.Name -eq '.git') {
+              Cleanup-GitFolder $fsEntry.FullName
+            } else {
+              $gitFolder = Join-Path -Path $fsEntry.FullName -ChildPath '.git'
+              if (Test-Path -Path $gitFolder -PathType Container) {
+                Cleanup-GitFolder $gitFolder
+                Delete-AllContent -path $fsEntry.FullName -keepDirs $false -exclusions '.git'
+              } else {
+                if ($keepDirs) {
+                  Remove-Item -Path "$($fsEntry.FullName)\*" -Recurse -Force
+                } else {
+                  Remove-Item -Path $fsEntry.FullName -Recurse -Force
+                }
+              }
+            }
+          } else {
+            Remove-Item -Path $fsEntry.FullName -Force
+          }
+        }
+      }
+
+      function Cleanup-GitRepo() {
+        $repoPath = "${{ parameters.repositoryPath }}"
+        Write-Host "Cleaning up git repository $repoPath"
+        $checkoutSubmodules = ("${{ parameters.checkoutSubmodules }}".ToLower() -eq "true")
+        $origDir = Get-Location
+
+        Set-Location $repoPath
+        & git clean -fdx
+        if ($LASTEXITCODE -eq 0) {
+          & git reset --hard HEAD
+        }
+        if ($checkoutSubmodules) {
+          if ($LASTEXITCODE -eq 0) {
+            & git submodule foreach git clean -fdx
+          }
+          if ($LASTEXITCODE -eq 0) {
+            & git submodule foreach git reset --hard HEAD
+          }
+        }
+        Set-Location $origDir
+
+        if ($LASTEXITCODE -ne 0) {
+          Delete-AllContent -path $repoPath
+        }
+      }
+
+      switch ("${{ parameters.cleanOption }}") {
+        "sources" {
+          Cleanup-GitRepo
+          Break
+        }
+        "sourcesAndOutput" {
+          Cleanup-GitRepo
+          Delete-AllContent -path "$(Build.BinariesDirectory)"
+          Break
+        }
+        "sourcesDirectory" {
+          Delete-AllContent -path "${{ parameters.repositoryPath }}"
+          Break
+        }
+        "all" {
+          $exclusions = @(
+            (Get-Item "$(Build.BinariesDirectory)").BaseName,
+            (Get-Item "$(Build.SourcesDirectory)").BaseName,
+            (Get-Item "$(Build.StagingDirectory)").BaseName,
+            (Get-Item "$(Common.TestResultsDirectory)").BaseName
+          )
+          Delete-AllContent -path "$(Pipeline.Workspace)" -keepDirs $false -exclusions $exclusions
+          Delete-AllContent -path "$(Pipeline.Workspace)" -keepDirs $true
+          Break
+        }
+      }
+    displayName: 'Clean path ${{ parameters.repositoryPath }} with option ${{ parameters.cleanOption }}'
+    workingDirectory: $(Pipeline.Workspace)

PostBuildCleanup/en-US/overview.md

@@ -11,6 +11,7 @@ The *Post Build Cleanup* task deletes unwanted files from your build agent, **af
 You can find the changes notes for this task [here](https://github.com/MicrosoftPremier/VstsExtensions/blob/master/PostBuildCleanup/en-US/changeNotes.md).
 
 ### Known Issues
+- Task is not properly working with YAML pipelines and we will not provide a version that is fully compatible with YAML. Please see [YAML Guidance](./yamlGuidance.md) for more information.
 - We are phasing out Team Foundation Server 2015. If you're still using Team Foundation Server 2015, please stay on version 2.x of the extension which can be downloaded [here](https://github.com/MicrosoftPremier/VstsExtensions/blob/master/PostBuildCleanup/archive/MSPremier.PostBuildCleanup-2.1.2.vsix). For newer versions of Team Foundation Server or Azure DevOps Services please use version 3.x and higher.
 
 ### Support

PostBuildCleanup/en-US/yamlGuidance.md

@@ -0,0 +1,39 @@
+[Back to Overview](./overview.md)
+
+# Guidance for Post Build Cleanup and YAML Pipelines
+For some time now we have been receiving feedback about issues with *Post Build Cleanup* and YAML pipelines, and we have investigated how to fix these. However, the way pipeline tasks work prevents us from really fixing the problems. Thus, there will never be a version of *Post Build Cleanup* that fully supports YAML, and here is why (if you are not interested in the backgroun, go down to [Recommendations for YAML pipelines](#recommendations-for-yaml-pipelines)):
+
+## Why the task is not working
+*Post Build Cleanup* relies on two pieces of information:
+- **The predefined variable *Build.Repository.Clean***  
+  In classic pipelines, this variables is set to `true` when the *Clean* setting of the *Get sources* step is set to `true`.
+- **The value of the build definition's `repository.properties.cleanOptions` setting**  
+  The task reads the build definition using the Azure DevOps API and evaluates this property. The value depends on the setting *Clean options* of the *Get sources* step.
+
+Since YAML supports working with multiple repositories (unlike classic), these options have been separated into the `clean` property of the `checkout` step, which cleans one specific repository folder on the agent, and the `clean` property of the `workspace` element of a job, which cleans the binaries directory (i.e., `$(Build.BinariesDirectory)`, e.g., `/agent/work/1/b`) when set to `outputs`, the sources directory (i.e., `$(Build.SourcesDirectory)`, e.g., `/agent/work/1/s`) when set to `resources`, or the pipeline workspace (i.e., `$(Pipeline.Workspace)`, e.g., `/agent/work/1`) when set to `all`.
+
+![Clean Settings in Classic Pipelines](../assets/CleanOptions.png)
+
+When running a YAML pipeline, Azure DevOps only sets the values mentioned above as follows:
+
+- Variable *Build.Repository.Clean* is only set if the `clean` parameter for the *self* repository is set to `true`. The `clean` parameter of all other repositories does not impact the variabale and is not visible to a pipeline task.
+- The build definition returned by the API only contains the *self* repository. Any additional repository that is used by the pipeline is not listed in the definition JSON. The property `repository.properties.cleanOptions` is not affected at all by the `workspace.clean` setting of the job. It is only set if you specify *Clean* and *Clean options* through the UI for a YAML pipeline (i.e., open *Triggers* from the pipeline editor context menu, then got to the *YAML* tab, select *Get sources* and specify the settings they way you did for classic pipelines).
+
+## What happens when the task runs in a YAML pipeline
+Due to the aforementioned behavior of the variables and API responses, there can be multiple different behaviors of the *Post Build Cleanup* task:
+
+- **Only one repository with default clone path**  
+  When you only use one repository in your YAML pipeline and do not specify any special path to clone the repository to, *Post Build Cleanup* may work. With the latest update, we ensured that the task cleans the `$(Build.SourcesDirectory)` (e.g., `/agent/work/1/s`) when you set the `clean` property of the `checkout` step to `true`. In addition, you can specify other clean targets (e.g., the source and binaries folder) by going through the pipeline UI (see above).
+- **Only one repository with custom clone path**  
+  If you set the `path` property of the `checkout` task to something else then `$(Build.SourcesDirectory)`, *Post Build Cleanup* will fail. The task does not know about custom clone paths and cannot figure out where your repository was cloned. As a result, if you also set the `clean` property of the `checkout` step to `true`, the task will delete the `$(Build.SourcesDirectory)` (which will likely be empty anyway) but not the repository path.
+- **Multiple repositories**  
+  When using multiple repositories in you pipeline, *Post Build Cleanup* **might** work. As soon as you clone multiple repositories, the default behavior of Azure Pipelines is to clone them into `$(Build.SourcesDirectory)/{repository name}`. Since *Post Build Cleanup* does not know about multiple repositories, it tries to clean the `$(Build.SourcesDirectory)`, realizes that this is not a Git repository path, and deletes all contents of the folder. This automatically breaks incremental builds and leads to errors in the post-checkout jobs. You would have to clone the *self* repo to `$(Build.SourcesDirectory)` and put additional repositories outside of that folder in order to make *Post Build Cleanup* work partially.
+
+## Recommendations for YAML pipelines
+Our recommendation depends on the number of repositories and your personal liking:
+
+### Only one repository in the pipeline
+If you only need to clone one repository (the *self* repository) and you are willing to configure the cleanup behavior outside of your YAML file in the pipeline UI, you can continue using the *Post Build Cleanup* task in the newest version. To configure the cleanup behavior, edit your YAML pipeline in the browser, open the pipeline editor context menu (the three dots in the upper right corner), and select *Triggers*. Then click on the *YAML* tab, select the *Get sources* step on the left, set *Clean* to *true* and specify the required *Clean options*.
+
+### All other cases
+In all other cases where you need multiple repositories and/or only want to configure the pipeline through the YAML file itself, you must use a custom script to do the cleanup steps. We have put together a [steps template](./cleanupRepository.yml) that you can add at the end of your pipeline for every repository you want to clean. It executes the same logic that *Post Build Cleanup* executes.