Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: flaky test ERC721 NFTs testdapp interaction #25854

Merged
merged 1 commit into from
Jul 16, 2024
Merged

Conversation

seaona
Copy link
Contributor

@seaona seaona commented Jul 16, 2024

Description

This PR fixes the flakiness of this specs due to how the window handling is managed in the spec file. When the test fails, we are in an invalidated context making the test flaky, when the current window is not correctly selected.
StaleElementReferenceError: stale element reference: stale element not found in the current frame

ci failure example: https://app.circleci.com/pipelines/github/MetaMask/metamask-extension/91382/workflows/70a1e3eb-3ff0-4e77-866a-fa2c4ed1d6ab/jobs/3407807/tests

Open in GitHub Codespaces

Related issues

Fixes: #25855

Manual testing steps

  1. Check ci

Screenshots/Recordings

Before

After

Pre-merge author checklist

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

@seaona seaona self-assigned this Jul 16, 2024
Copy link
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@seaona seaona added team-extension-platform flaky tests area-qa Relating to QA work (Quality Assurance) labels Jul 16, 2024
Copy link

@metamaskbot metamaskbot added the team-confirmations Push issues to confirmations team label Jul 16, 2024
@seaona seaona added area-qa Relating to QA work (Quality Assurance) and removed area-qa Relating to QA work (Quality Assurance) team-confirmations Push issues to confirmations team labels Jul 16, 2024
@seaona seaona marked this pull request as ready for review July 16, 2024 11:44
@seaona seaona requested a review from a team as a code owner July 16, 2024 11:44
await driver.waitForSelector({
css: '.confirm-page-container-summary__action__name',
text: 'Deposit',
});
await driver.clickElement({ text: 'Confirm', tag: 'button' });
await driver.waitUntilXWindowHandles(2);
await driver.switchToWindow(extension);
Copy link
Contributor Author

@seaona seaona Jul 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changing this for switchToWindowWithTitle which has more advance logic with retrials to mitigate flakiness, so we can remove all the window management adhoc steps from the spec file:

const windowHandles = await driver.getAllWindowHandles();
const [extension] = windowHandles;

@@ -38,19 +38,16 @@ describe('ERC721 NFTs testdapp interaction', function () {

// Notification
await driver.waitUntilXWindowHandles(3);
const windowHandles = await driver.getAllWindowHandles();
const [extension] = windowHandles;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having the extension window as a variable and keep referencing it steps later can cause the context to be invalidated. This is not recommended pattern and instead we should get the windows at the moment we want to switch to each of them

@metamaskbot
Copy link
Collaborator

Builds ready [a0fa95f]
Page Load Metrics (189 ± 235 ms)
PlatformPageMetricMin (ms)Max (ms)Average (ms)StandardDeviation (ms)MarginOfError (ms)
ChromeHomefirstPaint623001104924
domContentLoaded96526168
load402313189488235
domInteractive96526168
Bundle size diffs
  • background: 0 Bytes (0.00%)
  • ui: 0 Bytes (0.00%)
  • common: 0 Bytes (0.00%)

Copy link

codecov bot commented Jul 16, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 69.77%. Comparing base (2d416bf) to head (a0fa95f).

Additional details and impacted files
@@           Coverage Diff            @@
##           develop   #25854   +/-   ##
========================================
  Coverage    69.77%   69.77%           
========================================
  Files         1398     1398           
  Lines        49165    49165           
  Branches     13574    13574           
========================================
  Hits         34304    34304           
  Misses       14861    14861           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@chloeYue chloeYue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM ! Thanks for the quick fix !

@hjetpoluru hjetpoluru self-requested a review July 16, 2024 14:13
Copy link
Contributor

@hjetpoluru hjetpoluru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hjetpoluru hjetpoluru merged commit 52ee6ea into develop Jul 16, 2024
105 checks passed
@hjetpoluru hjetpoluru deleted the fix-flaky-erct21 branch July 16, 2024 14:15
@github-actions github-actions bot locked and limited conversation to collaborators Jul 16, 2024
@metamaskbot metamaskbot added the release-12.3.0 Issue or pull request that will be included in release 12.3.0 label Jul 16, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-qa Relating to QA work (Quality Assurance) flaky tests release-12.3.0 Issue or pull request that will be included in release 12.3.0 team-extension-platform
Projects
None yet
Development

Successfully merging this pull request may close these issues.

flaky test: ERC721 NFTs testdapp interaction
4 participants