Skip to content

Commit

Permalink
test: improve logs for e2e errors (#28479)
Browse files Browse the repository at this point in the history
<!--
Please submit this PR as a draft initially.
Do not mark it as "Ready for review" until the template has been
completely filled out, and PR status checks have passed at least once.
-->

## **Description**
This PR adds an improvement on our logs when the errors do not have the
expected form of a.value, leading to displaying empty errors and not
failing the test. Those are happening for RPC and some snap errors
types, which currently are displayed as empty (see below screenshots):
- The RPC errors doesn't have a `value` property but a `description`, so
we were seeing empty errors in the logs
- In the snaps errors, the a.value property is not directly present,
instead, the relevant information is nested within the
preview.properties array
- Other error structures, which doesn't fall under the 3 error
categories, will be captured in a fallback, which will stringify the
complete error


With this change we are now able to see better error logs in our e2e.

[![Open in GitHub
Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/MetaMask/metamask-extension/pull/28479?quickstart=1)

## **Related issues**

Fixes: MetaMask/MetaMask-planning#3648

## **Manual testing steps**

1. Run a test which triggers an RPC error like: `yarn test:e2e:single
test/e2e/tests/request-queuing/ui.spec.js --browser=chrome`
2. Check console errors before and after this change

## **Screenshots/Recordings**

<!-- If applicable, add screenshots and/or recordings to visualize the
before and after of your change. -->

### **Before**
See empty RPC error logs:

![Screenshot from 2024-11-15
08-47-39](https://github.com/user-attachments/assets/40f4a2dd-00f2-4bb3-b8da-740cd24254ec)

See empty snap error logs (the 1st one type is logged but the 2nd one is
empty):

![Screenshot from 2024-11-15
10-57-48](https://github.com/user-attachments/assets/019c1088-0816-4de3-a33a-9ff0c4266a9a)

### **After**
See complete RPC error logs

![Screenshot from 2024-11-15
09-43-45](https://github.com/user-attachments/assets/80b6ff10-e615-4261-8b13-30674e7a51bf)

See complete snaps error logs

![Screenshot from 2024-11-15
10-58-04](https://github.com/user-attachments/assets/629ec3da-ee19-4cda-ba82-fb73a01a8d03)




## **Pre-merge author checklist**

- [x] I've followed [MetaMask Contributor
Docs](https://github.com/MetaMask/contributor-docs) and [MetaMask
Extension Coding
Standards](https://github.com/MetaMask/metamask-extension/blob/develop/.github/guidelines/CODING_GUIDELINES.md).
- [x] I've completed the PR template to the best of my ability
- [x] I’ve included tests if applicable
- [x] I’ve documented my code using [JSDoc](https://jsdoc.app/) format
if applicable
- [x] I’ve applied the right labels on the PR (see [labeling
guidelines](https://github.com/MetaMask/metamask-extension/blob/develop/.github/guidelines/LABELING_GUIDELINES.md)).
Not required for external contributors.

## **Pre-merge reviewer checklist**

- [ ] I've manually tested the PR (e.g. pull and build branch, run the
app, test code being changed).
- [ ] I confirm that this PR addresses all acceptance criteria described
in the ticket it closes and includes the necessary testing evidence such
as recordings and or screenshots.
  • Loading branch information
seaona authored Nov 19, 2024
1 parent fd78a56 commit 3c51786
Show file tree
Hide file tree
Showing 3 changed files with 19 additions and 1 deletion.
1 change: 1 addition & 0 deletions test/e2e/snaps/test-snap-metrics.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -900,6 +900,7 @@ describe('Test Snap Metrics', function () {
testSpecificMock: mockSegment,
ignoredConsoleErrors: [
'MetaMask - RPC Error: Failed to fetch snap "npm:@metamask/bip32-example-snap": Failed to fetch tarball for package "@metamask/bip32-example-snap"..',
'Failed to fetch snap "npm:@metamask/bip32-example-…ball for package "@metamask/bip32-example-snap"..',
],
},
async ({ driver, mockedEndpoint: mockedEndpoints }) => {
Expand Down
1 change: 1 addition & 0 deletions test/e2e/tests/account/lockdown.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ describe('lockdown', function (this: Mocha.Suite) {
{
fixtures: new FixtureBuilder().build(),
ganacheOptions,
ignoredConsoleErrors: ['Error: Could not establish connection.'],
title: this.test?.fullTitle(),
},
async ({ driver }: { driver: Driver }) => {
Expand Down
18 changes: 17 additions & 1 deletion test/e2e/webdriver/driver.js
Original file line number Diff line number Diff line change
Expand Up @@ -1311,7 +1311,23 @@ class Driver {

#getErrorFromEvent(event) {
// Extract the values from the array
const values = event.args.map((a) => a.value);
const values = event.args.map((a) => {
// Handle snaps error type
if (a && a.preview && Array.isArray(a.preview.properties)) {
return a.preview.properties
.filter((prop) => prop.value !== 'Object')
.map((prop) => prop.value)
.join(', ');
} else if (a.description) {
// Handle RPC error type
return a.description;
} else if (a.value) {
// Handle generic error types
return a.value;
}
// Fallback for other error structures
return JSON.stringify(a, null, 2);
});

if (values[0]?.includes('%s')) {
// The values are in the "printf" form of [message, ...substitutions]
Expand Down

0 comments on commit 3c51786

Please sign in to comment.