Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ci): Reduce e2e attempts from 3 to 2 in CircleCI #24531

Merged
merged 1 commit into from
May 31, 2024

Conversation

legobeat
Copy link
Contributor

@legobeat legobeat commented May 14, 2024

Description

This reduces:

  • E2E CI retries from 2 to 1 (ie attempts 2 times instead of 3 times)
  • The average cost and time to run individual CI jobs
  • Feedback loops for e2e errors
  • Success rate (and therefore hopefully introduction rate) of flaky tests
  • Success rate (and therefore hopefully introduction rate) of inconsistent application behavior which relies on specific CI conditions to pass E2E tests

Non-E2E CI jobs have their existing retry count unchanged.

Open in GitHub Codespaces

Related issues

Manual testing steps

Screenshots/Recordings

Before

After

Pre-merge author checklist

  • I’ve followed MetaMask Coding Standards.
  • I've completed the PR template to the best of my ability
  • I’ve included tests if applicable
  • I’ve documented my code using JSDoc format if applicable
  • I’ve applied the right labels on the PR (see labeling guidelines). Not required for external contributors.

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

Copy link
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@legobeat legobeat added area-CI team-security e2e-test End to end test for the MetaMask extension labels May 14, 2024
@legobeat legobeat marked this pull request as ready for review May 14, 2024 22:08
@legobeat legobeat requested review from kumavis and a team as code owners May 14, 2024 22:08
@legobeat legobeat requested review from tmashuang and hjetpoluru May 14, 2024 22:09
hjetpoluru
hjetpoluru previously approved these changes May 16, 2024
Copy link
Contributor

@hjetpoluru hjetpoluru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@legobeat legobeat force-pushed the e2e-decrease-retries branch from f36f251 to f955d27 Compare May 16, 2024 22:08
@legobeat legobeat requested review from a team and removed request for brad-decker May 16, 2024 22:08
@legobeat legobeat force-pushed the e2e-decrease-retries branch from fcab3dd to 353b7e0 Compare May 23, 2024 11:22
seaona
seaona previously approved these changes May 23, 2024
@legobeat legobeat requested a review from hjetpoluru May 23, 2024 11:25
@legobeat legobeat force-pushed the e2e-decrease-retries branch 9 times, most recently from fa9d9c1 to fb946ff Compare May 23, 2024 19:13
@legobeat legobeat requested a review from rekmarks May 31, 2024 05:36
@legobeat legobeat force-pushed the e2e-decrease-retries branch from d659566 to 17502bd Compare May 31, 2024 05:47
@legobeat legobeat requested a review from digiwand May 31, 2024 06:00
@legobeat legobeat force-pushed the e2e-decrease-retries branch 12 times, most recently from 8508a6d to 7951645 Compare May 31, 2024 10:16
@legobeat legobeat force-pushed the e2e-decrease-retries branch 7 times, most recently from c02098b to 81a6632 Compare May 31, 2024 14:56
@legobeat legobeat force-pushed the e2e-decrease-retries branch from 81a6632 to 354b9de Compare May 31, 2024 17:33
@legobeat legobeat merged commit 3983762 into MetaMask:develop May 31, 2024
77 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators May 31, 2024
@benjisclowder benjisclowder added the release-12.0.0 Issue or pull request that will be included in release 12.0.0 label Jun 6, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-CI e2e-test End to end test for the MetaMask extension release-12.0.0 Issue or pull request that will be included in release 12.0.0 team-security
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

9 participants