Add tags and exclude-tags support #2286
Xcode test results
Test Runner
Summary
Total | Passed | Failed | Skipped | Expected Failure | ⏱️ Time |
---|---|---|---|---|---|
31 | 22 | 8 | 1 | 0 | 11.30s |
Test Summary
RunnerUITests
- Device: iPad (10th generation), 18.0 (22A5316j)
- SDK: Simulator - iOS 17.5, 17.5
Test | Total | ||||
---|---|---|---|---|---|
RunnerUITests | 31 | 22 | 8 | 1 | 0 |
Failures
RunnerUITests/RunnerUITests/define_test check dart-define-from-file
File | /Users/runner/work/patrol/patrol/dev/e2e_app/ios/RunnerUITests/RunnerUITests.m:5 |
Issue Type | Assertion Failure |
Message | ((passed) is true) failed - ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
The following TestFailure was thrown running a test:
Expected: exactly one matching candidate
Actual: PatrolFinder:
Which: means none were found but one was expected
When the exception was thrown, this was the stack: This was caught by the test expectation on the following line: |
RunnerUITests/RunnerUITests/external_link_test Open external url
File | /Users/runner/work/patrol/patrol/dev/e2e_app/ios/RunnerUITests/RunnerUITests.m:5 |
Issue Type | Assertion Failure |
Message | ((passed) is true) failed - ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
The following PatrolActionException was thrown running a test:
Patrol action failed: NativeAutomatorClientException:
waitUntilVisible() failed with Invalid response: 400 The
operation couldn�t be completed. (patrol.PatrolError error 0.)
When the exception was thrown, this was the stack: The test description was: |
RunnerUITests/RunnerUITests/open_app_test same open maps test that should be skipped
File | /Users/runner/work/patrol/patrol/dev/e2e_app/ios/RunnerUITests/RunnerUITests.m:5 |
Issue Type | Assertion Failure |
Message | Failed to get launch progress for : Simulator device failed to launch pl.leancode.patrol.e2eApp. The system shell (SpringBoard:9883) probably crashed. (Underlying Error: The request to terminate "pl.leancode.patrol.e2eApp" failed. The system shell (SpringBoard:9883) probably crashed.) |
RunnerUITests/RunnerUITests/permissions.notifications_test taps on notification
File | /Users/runner/work/patrol/patrol/dev/e2e_app/ios/RunnerUITests/RunnerUITests.m:5 |
Issue Type | Assertion Failure |
Message | ((passed) is true) failed - ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
The following PatrolActionException was thrown running a test:
Patrol action failed: NativeAutomatorClientException:
tapOnNotificationBySelector() failed with Invalid response: 400
The operation couldn�t be completed. (patrol.PatrolError error
0.)
When the exception was thrown, this was the stack: The test description was: |
RunnerUITests/RunnerUITests/permissions.notifications_test taps on notification native2
File | /Users/runner/work/patrol/patrol/dev/e2e_app/ios/RunnerUITests/RunnerUITests.m:5 |
Issue Type | Assertion Failure |
Message | ((passed) is true) failed - ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
The following PatrolActionException was thrown running a test:
Patrol action failed: NativeAutomatorClientException:
tapOnNotificationBySelector() failed with Invalid response: 400
The operation couldn�t be completed. (patrol.PatrolError error
0.)
When the exception was thrown, this was the stack: The test description was: |
RunnerUITests/RunnerUITests/permissions.permissions_location_test accepts location permission
File | /Users/runner/work/patrol/patrol/dev/e2e_app/ios/RunnerUITests/RunnerUITests.m:5 |
Issue Type | Assertion Failure |
Message | Failed to get matching snapshots: Lost connection to the application (pid 13020). (Underlying Error: Couldn’t communicate with a helper application. Try your operation again. If that fails, quit and relaunch the application and try again. The connection to service created from an endpoint was invalidated: failed to check-in, peer may have been unloaded: mach_error=10000003.) |
File | /Users/runner/work/patrol/patrol/dev/e2e_app/ios/RunnerUITests/RunnerUITests.m:5 |
Issue Type | Assertion Failure |
Message | Failed to get matching snapshots: Lost connection to the application (pid 13020). (Underlying Error: Couldn’t communicate with a helper application. Try your operation again. If that fails, quit and relaunch the application and try again. The connection to service created from an endpoint was invalidated: failed to check-in, peer may have been unloaded: mach_error=10000003.) |
File | /Users/runner/work/patrol/patrol/dev/e2e_app/ios/RunnerUITests/RunnerUITests.m:5 |
Issue Type | Assertion Failure |
Message | Failed to get matching snapshots: Lost connection to the application (pid 13020). (Underlying Error: Couldn’t communicate with a helper application. Try your operation again. If that fails, quit and relaunch the application and try again. The connection to service created from an endpoint was invalidated: failed to check-in, peer may have been unloaded: mach_error=10000003.) |
File | /Users/runner/work/patrol/patrol/dev/e2e_app/ios/RunnerUITests/RunnerUITests.m:5 |
Issue Type | Assertion Failure |
Message | Failed to get matching snapshots: Lost connection to the application (pid 13020). (Underlying Error: Couldn’t communicate with a helper application. Try your operation again. If that fails, quit and relaunch the application and try again. The connection to service created from an endpoint was invalidated: failed to check-in, peer may have been unloaded: mach_error=10000003.) |
File | /Users/runner/work/patrol/patrol/dev/e2e_app/ios/RunnerUITests/RunnerUITests.m:5 |
Issue Type | Assertion Failure |
Message | Failed to get matching snapshots: Lost connection to the application (pid 13020). (Underlying Error: Couldn’t communicate with a helper application. Try your operation again. If that fails, quit and relaunch the application and try again. The connection to service created from an endpoint was invalidated: failed to check-in, peer may have been unloaded: mach_error=10000003.) |
RunnerUITests/RunnerUITests/permissions.permissions_location_test accepts location permission native2
File | /Users/runner/work/patrol/patrol/dev/e2e_app/ios/RunnerUITests/RunnerUITests.m:5 |
Issue Type | Assertion Failure |
Message | Failed to get launch progress for : Simulator device failed to launch pl.leancode.patrol.e2eApp. The system shell (SpringBoard:13020) probably crashed. (Underlying Error: The request to terminate "pl.leancode.patrol.e2eApp" failed. The system shell (SpringBoard:13020) probably crashed.) |
RunnerUITests/RunnerUITests/permissions.permissions_many_test grants various permissions
File | /Users/runner/work/patrol/patrol/dev/e2e_app/ios/RunnerUITests/RunnerUITests.m:5 |
Issue Type | Assertion Failure |
Message | ((passed) is true) failed - ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
The following TestFailure was thrown running a test:
Expected: 'Granted'
Actual: 'Not granted'
Which: is different.
Expected: Granted
Actual: Not grante ...
^
Differ at offset 0
When the exception was thrown, this was the stack: This was caught by the test expectation on the following line: |
Details
Test Details
RunnerUITests
RunnerUITests
⏱️ | ||||
---|---|---|---|---|
22 (71%) | 8 (26%) | 1 (3%) | 0 (0%) | 11.30s |
deep_link_test Open url in the app Activities:
| |
define_test check dart-define-from-file Activities:
| |
example_test counter state is the same after going to Home and switching apps Activities:
| |
example_test short test with tag Activities:
| |
example_test short test with two tags Activities:
| |
external_link_test Open external url Activities:
| |
internal.group_test at the beginning Activities:
| |
internal.group_test top level group in file alpha first Activities:
| |
internal.group_test top level group in file alpha second Activities:
| |
internal.group_test top level group in file bravo first Activities:
| |
internal.group_test top level group in file bravo second Activities:
| |
internal.group_test top level group in file in the middle Activities:
| |
internal.long_name_test aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Activities:
| |
internal.long_name_test alphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaalphaa Activities:
| |
internal.long_name_test kocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykkocykk Activities:
| |
macos.macos_app_test taps around Activities:
| |
macos.macos_app_test taps around test to skip Activities:
| |
open_app_test open browser Activities:
| |
open_app_test open maps Activities:
| |
open_app_test same open maps test that should be skipped Activities:
| |
open_quick_settings_test opens quick settings Activities:
| |
permissions.deny_many_permissions_twice_test denies various permissions Activities:
| |
permissions.notifications_test taps on notification Activities:
| |
permissions.notifications_test taps on notification native2 Activities:
| |
permissions.permissions_location_test accepts location permission Activities:
| |
permissions.permissions_location_test accepts location permission native2 Activities:
| |
permissions.permissions_many_test grants various permissions Activities:
| |
service_dark_mode_test disables and enables dark mode twice Activities:
| |
swipe_test scrolls the Settings app Activities:
| |
tap_at_test taps at the lower middle of the screen in the Settings app Activities:
| |
volume_test change volume Activities:
|
Annotations
github-actions / Patrol tests on iPad (10th generation)
Assertion Failure
((passed) is true) failed - ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
The following TestFailure was thrown running a test:
Expected: exactly one matching candidate
Actual: PatrolFinder:<Found 0 widgets with text "FIRST_KEY:
First key from defines_1.json": []>
Which: means none were found but one was expected
When the exception was thrown, this was the stack:
#4 main.<anonymous closure> (file:///Users/runner/work/patrol/patrol/dev/e2e_app/integration_test/define_test.dart:12:5)
<asynchronous suspension>
#5 patrolTest.<anonymous closure> (package:patrol/src/common.dart:135:7)
<asynchronous suspension>
#6 testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:183:15)
<asynchronous suspension>
#7 TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:1017:5)
<asynchronous suspension>
#8 TestWidgetsFlutterBinding._createTestCompletionHandler.<anonymous closure> (package:flutter_test/src/binding.dart:809:12)
<asynchronous suspension>
This was caught by the test expectation on the following line:
file:///Users/runner/work/patrol/patrol/dev/e2e_app/integration_test/define_test.dart line 12
The test description was:
check dart-define-from-file
═════════════════════════════════════════════════════════════════
github-actions / Patrol tests on iPad (10th generation)
Assertion Failure
((passed) is true) failed - ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
The following PatrolActionException was thrown running a test:
Patrol action failed: NativeAutomatorClientException:
waitUntilVisible() failed with Invalid response: 400 The
operation couldnât be completed. (patrol.PatrolError error 0.)
When the exception was thrown, this was the stack:
#0 NativeAutomator._wrapRequest (package:patrol/src/native/native_automator.dart:226:7)
<asynchronous suspension>
#1 NativeAutomator.waitUntilVisible (package:patrol/src/native/native_automator.dart:768:5)
<asynchronous suspension>
#2 main.<anonymous closure> (file:///Users/runner/work/patrol/patrol/dev/e2e_app/integration_test/external_link_test.dart:27:5)
<asynchronous suspension>
#3 patrolTest.<anonymous closure> (package:patrol/src/common.dart:135:7)
<asynchronous suspension>
#4 testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:183:15)
<asynchronous suspension>
#5 TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:1017:5)
<asynchronous suspension>
#6 TestWidgetsFlutterBinding._createTestCompletionHandler.<anonymous closure> (package:flutter_test/src/binding.dart:809:12)
<asynchronous suspension>
The test description was:
Open external url
═════════════════════════════════════════════════════════════════
github-actions / Patrol tests on iPad (10th generation)
Assertion Failure
Failed to get launch progress for <XCUIApplicationImpl: 0x600000cc7930 pl.leancode.patrol.e2eApp at /Users/runner/work/patrol/patrol/dev/e2e_app/build/ios_integ/Build/Products/Debug-iphonesimulator/Runner.app>: Simulator device failed to launch pl.leancode.patrol.e2eApp. The system shell (SpringBoard:9883) probably crashed. (Underlying Error: The request to terminate "pl.leancode.patrol.e2eApp" failed. The system shell (SpringBoard:9883) probably crashed.)
github-actions / Patrol tests on iPad (10th generation)
Assertion Failure
((passed) is true) failed - ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
The following PatrolActionException was thrown running a test:
Patrol action failed: NativeAutomatorClientException:
tapOnNotificationBySelector() failed with Invalid response: 400
The operation couldnât be completed. (patrol.PatrolError error
0.)
When the exception was thrown, this was the stack:
#0 NativeAutomator._wrapRequest (package:patrol/src/native/native_automator.dart:226:7)
<asynchronous suspension>
#1 NativeAutomator.tapOnNotificationBySelector (package:patrol/src/native/native_automator.dart:452:5)
<asynchronous suspension>
#2 main.<anonymous closure> (file:///Users/runner/work/patrol/patrol/dev/e2e_app/integration_test/permissions/notifications_test.dart:36:7)
<asynchronous suspension>
#3 patrolTest.<anonymous closure> (package:patrol/src/common.dart:135:7)
<asynchronous suspension>
#4 testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:183:15)
<asynchronous suspension>
#5 TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:1017:5)
<asynchronous suspension>
#6 TestWidgetsFlutterBinding._createTestCompletionHandler.<anonymous closure> (package:flutter_test/src/binding.dart:809:12)
<asynchronous suspension>
The test description was:
taps on notification
═════════════════════════════════════════════════════════════════
github-actions / Patrol tests on iPad (10th generation)
Assertion Failure
((passed) is true) failed - ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
The following PatrolActionException was thrown running a test:
Patrol action failed: NativeAutomatorClientException:
tapOnNotificationBySelector() failed with Invalid response: 400
The operation couldnât be completed. (patrol.PatrolError error
0.)
When the exception was thrown, this was the stack:
#0 NativeAutomator2._wrapRequest (package:patrol/src/native/native_automator2.dart:109:7)
<asynchronous suspension>
#1 NativeAutomator2.tapOnNotificationBySelector (package:patrol/src/native/native_automator2.dart:335:5)
<asynchronous suspension>
#2 main.<anonymous closure> (file:///Users/runner/work/patrol/patrol/dev/e2e_app/integration_test/permissions/notifications_test.dart:81:7)
<asynchronous suspension>
#3 patrolTest.<anonymous closure> (package:patrol/src/common.dart:135:7)
<asynchronous suspension>
#4 testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:183:15)
<asynchronous suspension>
#5 TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:1017:5)
<asynchronous suspension>
#6 TestWidgetsFlutterBinding._createTestCompletionHandler.<anonymous closure> (package:flutter_test/src/binding.dart:809:12)
<asynchronous suspension>
The test description was:
taps on notification native2
═════════════════════════════════════════════════════════════════
github-actions / Patrol tests on iPad (10th generation)
Assertion Failure
Failed to get matching snapshots: Lost connection to the application (pid 13020). (Underlying Error: Couldn’t communicate with a helper application. Try your operation again. If that fails, quit and relaunch the application and try again. The connection to service created from an endpoint was invalidated: failed to check-in, peer may have been unloaded: mach_error=10000003.)
github-actions / Patrol tests on iPad (10th generation)
Assertion Failure
Failed to get matching snapshots: Lost connection to the application (pid 13020). (Underlying Error: Couldn’t communicate with a helper application. Try your operation again. If that fails, quit and relaunch the application and try again. The connection to service created from an endpoint was invalidated: failed to check-in, peer may have been unloaded: mach_error=10000003.)
github-actions / Patrol tests on iPad (10th generation)
Assertion Failure
Failed to get matching snapshots: Lost connection to the application (pid 13020). (Underlying Error: Couldn’t communicate with a helper application. Try your operation again. If that fails, quit and relaunch the application and try again. The connection to service created from an endpoint was invalidated: failed to check-in, peer may have been unloaded: mach_error=10000003.)
github-actions / Patrol tests on iPad (10th generation)
Assertion Failure
Failed to get matching snapshots: Lost connection to the application (pid 13020). (Underlying Error: Couldn’t communicate with a helper application. Try your operation again. If that fails, quit and relaunch the application and try again. The connection to service created from an endpoint was invalidated: failed to check-in, peer may have been unloaded: mach_error=10000003.)
github-actions / Patrol tests on iPad (10th generation)
Assertion Failure
Failed to get matching snapshots: Lost connection to the application (pid 13020). (Underlying Error: Couldn’t communicate with a helper application. Try your operation again. If that fails, quit and relaunch the application and try again. The connection to service created from an endpoint was invalidated: failed to check-in, peer may have been unloaded: mach_error=10000003.)
github-actions / Patrol tests on iPad (10th generation)
Assertion Failure
Failed to get launch progress for <XCUIApplicationImpl: 0x600000cc7930 pl.leancode.patrol.e2eApp at /Users/runner/work/patrol/patrol/dev/e2e_app/build/ios_integ/Build/Products/Debug-iphonesimulator/Runner.app>: Simulator device failed to launch pl.leancode.patrol.e2eApp. The system shell (SpringBoard:13020) probably crashed. (Underlying Error: The request to terminate "pl.leancode.patrol.e2eApp" failed. The system shell (SpringBoard:13020) probably crashed.)
github-actions / Patrol tests on iPad (10th generation)
Assertion Failure
((passed) is true) failed - ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
The following TestFailure was thrown running a test:
Expected: 'Granted'
Actual: 'Not granted'
Which: is different.
Expected: Granted
Actual: Not grante ...
^
Differ at offset 0
When the exception was thrown, this was the stack:
#4 _requestAndGrantCameraPermission (file:///Users/runner/work/patrol/patrol/dev/e2e_app/integration_test/permissions/permissions_many_test.dart:29:3)
<asynchronous suspension>
#5 main.<anonymous closure> (file:///Users/runner/work/patrol/patrol/dev/e2e_app/integration_test/permissions/permissions_many_test.dart:13:5)
<asynchronous suspension>
#6 patrolTest.<anonymous closure> (package:patrol/src/common.dart:135:7)
<asynchronous suspension>
#7 testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:183:15)
<asynchronous suspension>
#8 TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:1017:5)
<asynchronous suspension>
#9 TestWidgetsFlutterBinding._createTestCompletionHandler.<anonymous closure> (package:flutter_test/src/binding.dart:809:12)
<asynchronous suspension>
This was caught by the test expectation on the following line:
file:///Users/runner/work/patrol/patrol/dev/e2e_app/integration_test/permissions/permissions_many_test.dart line 29
The test description was:
grants various permissions
═════════════════════════════════════════════════════════════════