Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add UI testing workflow for contributor PRs #8577

Merged
merged 1 commit into from
May 27, 2021

Conversation

AaronMT
Copy link
Contributor

@AaronMT AaronMT commented May 25, 2021

This adds a GitHub Action workflow for running UI smoke tests against contributor forks on Pull Request. It runs against MacOS-latest (here) on Xcode 12.4 (this is configurable). The SmokeXCUITests testplanh is used on an iPhone 11 (also configurable).

Of note there is an issue with a false state check (tbartelmess/analyze-xcoderesults-action#35) on the XCResult parsing step with a Marketplace GitHub Action I am using.

@AaronMT AaronMT requested a review from isabelrios May 25, 2021 13:22
@AaronMT AaronMT self-assigned this May 25, 2021
@AaronMT
Copy link
Contributor Author

AaronMT commented May 25, 2021

@isabelrios Will probably need a force merge because it's trying to run with on: [pull_request] but it doesn't have access to GitHub token yet I guess.

@isabelrios
Copy link
Contributor

@isabelrios Will probably need a force merge because it's trying to run with on: [pull_request] but it doesn't have access to GitHub token yet I guess.

Would that be the cause of this error that can be seen by going to the red check, or that's a different thing?:

pythonLocation: /Users/runner/hostedtoolcache/Python/3.9.5/x64
Analyzing TestResults.xcresult ...
Error: Resource not accessible by integration

@AaronMT
Copy link
Contributor Author

AaronMT commented May 26, 2021

@isabelrios Will probably need a force merge because it's trying to run with on: [pull_request] but it doesn't have access to GitHub token yet I guess.

Would that be the cause of this error that can be seen by going to the red check, or that's a different thing?:

pythonLocation: /Users/runner/hostedtoolcache/Python/3.9.5/x64
Analyzing TestResults.xcresult ...
Error: Resource not accessible by integration

Yeah I believe so because it can't upload the artifact so it can't parse anything.

@isabelrios
Copy link
Contributor

I'm happy to land this and see how it works. My only concern is the red status report failure if the token is not the solution.. let's have a plan to act in case that's not the cause...

@isabelrios isabelrios closed this May 27, 2021
@isabelrios isabelrios reopened this May 27, 2021
Copy link
Contributor

@isabelrios isabelrios left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's try this. Although the result is red, it does not prevent from landing so we can try this out, but I would let the team know so that they don't panic if they see all PRs red temporarly :)

@AaronMT AaronMT merged commit dddb3a7 into mozilla-mobile:main May 27, 2021
@AaronMT AaronMT deleted the build-contributor-pr branch May 27, 2021 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants