Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tools: add WPT updater for specific subsystems #54460

Merged
merged 5 commits into from
Nov 26, 2024

Conversation

mertcanaltin
Copy link
Member

@mertcanaltin mertcanaltin commented Aug 20, 2024

a WPT updater that specifically targets the url subsystem for more controlled and focused updates.

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/actions

@nodejs-github-bot nodejs-github-bot added the meta Issues and PRs related to the general management of the project. label Aug 20, 2024
.github/workflows/update-wpt.yml Outdated Show resolved Hide resolved
.github/workflows/update-wpt.yml Outdated Show resolved Hide resolved
.github/workflows/update-wpt.yml Outdated Show resolved Hide resolved
.github/workflows/update-wpt.yml Outdated Show resolved Hide resolved
.github/workflows/update-wpt.yml Outdated Show resolved Hide resolved
.github/workflows/update-wpt.yml Outdated Show resolved Hide resolved
.github/workflows/update-wpt.yml Outdated Show resolved Hide resolved
.github/workflows/update-wpt.yml Show resolved Hide resolved
@mertcanaltin
Copy link
Member Author

thanks for your suggestions

Copy link
Member

@avivkeller avivkeller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here are my opinions

.github/workflows/update-wpt.yml Outdated Show resolved Hide resolved
.github/workflows/update-wpt.yml Outdated Show resolved Hide resolved
.github/workflows/update-wpt.yml Outdated Show resolved Hide resolved
.github/workflows/update-wpt.yml Outdated Show resolved Hide resolved
@mertcanaltin mertcanaltin force-pushed the dev-wpt-test-updater branch 2 times, most recently from 616d720 to 9b1cda3 Compare August 20, 2024 12:55
.github/workflows/update-wpt.yml Outdated Show resolved Hide resolved
.github/workflows/update-wpt.yml Outdated Show resolved Hide resolved
.github/workflows/update-wpt.yml Outdated Show resolved Hide resolved
.github/workflows/update-wpt.yml Outdated Show resolved Hide resolved
@mertcanaltin mertcanaltin changed the title ci: add WPT updater tools: add WPT updater Aug 20, 2024
Copy link
Member

@avivkeller avivkeller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also feel like this could be moved to a dep_updater file.

.github/workflows/update-wpt.yml Outdated Show resolved Hide resolved
.github/workflows/update-wpt.yml Outdated Show resolved Hide resolved
.github/workflows/update-wpt.yml Outdated Show resolved Hide resolved
.github/workflows/update-wpt.yml Outdated Show resolved Hide resolved
.github/workflows/update-wpt.yml Outdated Show resolved Hide resolved
.github/workflows/update-wpt.yml Outdated Show resolved Hide resolved
@mertcanaltin mertcanaltin force-pushed the dev-wpt-test-updater branch 2 times, most recently from 380a165 to 625ec8b Compare August 21, 2024 08:34
@avivkeller
Copy link
Member

By the way, I'm trying to update the WPT files in #54468.

IMO this could be done via

git node wpt <...>

Because the README.md file also needs to be updated.

@mertcanaltin
Copy link
Member Author

By the way, I'm trying to update the WPT files in #54468.

IMO this could be done via

git node wpt <...>

Because the README.md file also needs to be updated.

Do you have a suggestion, would that be enough for this?

- Name: Update README.md file
        Run: |
            # Update the README.md file with the new WPT version
            # Assuming you have a script or command to do this
            ./scripts/update-readme.sh ‘${{{ env.NEW_VERSION }}’

      - Name: Commit to Changes
        Run: |
            git config --global user.name ‘Node.js GitHub Bot’
            git config --global user.email ‘[email protected]’
            git add test/fixtures/wpt README.md
            git commit -m ‘test: Update WPT to ${{{ env.NEW_VERSION }} and update README.md’

@avivkeller
Copy link
Member

In my opinion, the entire WPT update process could be moved to a dep-updater script, using git node for each section. But, there are a lot of cases where updating these tests cause problems (just look at the errors in my PR), so maybe keeping it manual is good?

I'm not sure.

panva
panva previously requested changes Aug 26, 2024
Copy link
Member

@panva panva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general I don't think updating WPTs automatically is a good idea. I can't find the issue now but I believe we've already had that discussion in the past, that's not to say we can't have it again... So:

  • The way to update WPTs is through the git node wpt ... command from node-core-utils, not checking out the WPT repo and copying over files
  • WPTs have dependencies in resources, interfaces, or generally a new folder that is not part of the checkout and requires manual intervention
  • Pulling in new WPTs often makes new WPTs fail, that also requires manual intervention
  • There are tentative WPTs that are not tied to a finished spec change, that will fail too
  • If there was an automated process then WPTs should be updated by their subsystem, not all at once, but that again raises an issue when that subsystem WPT change requires to pull in a file from a different directory that also needs to be updated
  • Some WPT subsystems currently cannot be updated (e.g. WebCryptoAPI) because we don't have a way to conditionally mark tests as failing depending on the system they run on, in WebCryptoAPI's case - test: update wpt test for WebCryptoAPI #54127, test,crypto: update WebCryptoAPI WPT #52222 (comment), RHEL's linked OpenSSL makes some vectors pass that don't pass anywhere else.

In my opinion a sensible way to update WPTs would be to pull a WPT subsystem from upstream (using git node wpt ...), run it, if it passes - open a PR, if it fails - have a keepalive issue where pending WPT updates are tracked.

FWIW this could also be hooked into the daily wpt job that runs the most up to date WPT checkout for the purpose of uploading the current state to wpt.io.

Remember that WPTs may very well fail for a long time when they are a) tentative or b) require a semver-major change to pass.

@panva
Copy link
Member

panva commented Aug 26, 2024

cc @nodejs/web-standards

@legendecas
Copy link
Member

yeah, we had discussion at #50567.

@mertcanaltin
Copy link
Member Author

I'm closing this place, thank you very much for the comments

@panva panva added review wanted PRs that need reviews. and removed review wanted PRs that need reviews. labels Nov 1, 2024
@anonrig anonrig added commit-queue Add this label to land a pull request using GitHub Actions. and removed review wanted PRs that need reviews. labels Nov 26, 2024
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Nov 26, 2024
@nodejs-github-bot nodejs-github-bot merged commit 4f62ab5 into nodejs:main Nov 26, 2024
16 checks passed
@nodejs-github-bot
Copy link
Collaborator

Landed in 4f62ab5

@panva
Copy link
Member

panva commented Nov 26, 2024

🎉

https://github.com/nodejs/node/actions/runs/12031931489
#55999

More subsystems can be added like so

aduh95 added a commit that referenced this pull request Nov 27, 2024
targos pushed a commit that referenced this pull request Dec 2, 2024
github-actions bot pushed a commit to aduh95/node that referenced this pull request Dec 4, 2024
aduh95 pushed a commit to aduh95/node that referenced this pull request Dec 4, 2024
aduh95 pushed a commit to aduh95/node that referenced this pull request Dec 4, 2024
aduh95 pushed a commit to aduh95/node that referenced this pull request Dec 4, 2024
aduh95 pushed a commit to aduh95/node that referenced this pull request Dec 4, 2024
aduh95 pushed a commit to aduh95/node that referenced this pull request Dec 4, 2024
aduh95 pushed a commit that referenced this pull request Dec 13, 2024
aduh95 pushed a commit that referenced this pull request Dec 13, 2024
aduh95 pushed a commit that referenced this pull request Dec 13, 2024
aduh95 pushed a commit that referenced this pull request Dec 18, 2024
ruyadorno pushed a commit that referenced this pull request Jan 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. meta Issues and PRs related to the general management of the project.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants