-
-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Task: Split J2N.Tests.xUnit into separate projects per collection to improve performance #120
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ss of whether they target NUnit or Xunit
…sting collections) between test projects
…ectory-level configuration across projects that apply to a given test framework.
…ectories and reorganized warning suppressions so they apply globally, to NUnit, or Xunit tests.
…TestUtilities.Xunit, J2N.TestFramework, and J2N.TestCommon so the build script will skip them when running tests
…es.Xunit to the tests/Xunit folder and J2N.TestUtilities to the tests folder from the src/ folder
…ince it applies to all projects in this directory.
…ReferenceOverride.targets
…mmon.References.targets to both NUnit and XUnit folders to simplify adding new test projects
…we don't need to check for all of the target frameworks that are not being currently tested
… than build artifact
…t rather than build artifact" This reverts commit 4491cd9.
… is enabled in Azure DevOps
… so we don't have to run so many templates to upload the results.
…l take the longest and we can maximize our parallel processing
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #109
This refactors the test projects to put NUnit and XUnit into their own directory with their own dependencies. The single
J2N.Tests.xUnit
project was split into multiple projects:Test Projects
J2N.Collections.Concurrent.LurchTable.Tests
J2N.Collections.Generic.Dictionary.Tests
J2N.Collections.Generic.Extensions.Tests
J2N.Collections.Generic.HashSet.Tests
J2N.Collections.Generic.LinkedDictionary.Tests
J2N.Collections.Generic.LinkedHashSet.Tests
J2N.Collections.Generic.List.Tests
J2N.Collections.Generic.SortedDictionary.Tests
J2N.Collections.Generic.SortedSet.Tests
J2N.Collections.ObjectModel.Tests
J2N.Runtime.Tests
Test Infrastructure Projects
J2N.TestCommon
- Base classes for collection tests that are shared among collection Test projectsJ2N.TestUtilities
- Shared infrastructure between XUnit and NUnit, such as platform detectionJ2N.TestUtilities.Xunit
- Extensions to Xunit for use in the Microsoft-derived testsThe infrastructure for NUnit (
J2N.TestFramework
) was also moved fromsrc
totests/NUnit
along sideJ2N.Tests
, but neither of them have been split (we may consider this because there are 40,000 tests).Optimization
There were some optimizations done to make the Azure DevOps pipeline faster.
run-tests-on-os.yml
template was changed to only allow a single target framework. This allows us to pass the target framework topublish-test-results-for-test-projects.yml
so we don't spend time cycling through all of the target frameworks (thepublish-test-results-for-target-frameworks.yml
template was removed).System.Debug
environment variable is true.Publishing
This also changes the publishing to put each project into its own directory rather than publishing the entire solution to a single directory, which is no longer supported.
Results
Unfortunately, this didn't make as big of an impact as was hoped, shaving about 5 minutes off of a build/test run.
This could be significantly improved by uploading the test results through the REST API from a background job instead of waiting for all of the results to be uploaded using the PublishTestResults@2 task. I looked into this, but it looks like it will require us to:
Maybe something this complex should be put into a dotnet tool or a Powershell module, both of which could be reused in all of our projects. Last I checked Microsoft used to maintain such a tool for TFS, but the project has been abandoned.