Profiling the app with patrol

[ahmdaeyz]

Steps to reproduce

Try to run the tests in profile mode on a physical iOS device.

Actual results

The patrol_cli exits with an error that It must be a release to run on a physical device. I want to get the performance related data on the binding and be able to use patrol at the same time.

Logs

Logs

<!-- Replace this line with your logs. Do not remove the backticks! -->

Patrol version

Both latest v1 and latest v2

Patrol Doctor output

Patrol Doctor output

Program adb found in /opt/homebrew/bin/adb
Env var $ANDROID_HOME set to /Users/ahmdaeyz/Library/Android/sdk
Program xcodebuild found in /usr/bin/xcodebuild
Program ideviceinstaller found in /opt/homebrew/bin/ideviceinstaller
Program ios-deploy found in /opt/homebrew/bin/ios-deploy

Flutter Doctor output

Flutter Doctor output

[!] Flutter (Channel stable, 3.10.5, on macOS 13.4.1 22F82 darwin-arm64, locale en-EG)
    • Flutter version 3.10.5 on channel stable at /Users/ahmdaeyz/development/flutter
    ! Warning: `dart` on your path resolves to /opt/homebrew/Cellar/dart/3.0.4/libexec/bin/dart, which is not inside your current Flutter SDK checkout at /Users/ahmdaeyz/development/flutter. Consider adding /Users/ahmdaeyz/development/flutter/bin to the front of your path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 796c8ef792 (4 weeks ago), 2023-06-13 15:51:02 -0700
    • Engine revision 45f6e00911
    • Dart version 3.0.5
    • DevTools version 2.23.1
    • If those were intentional, you can disregard the above warnings; however it is recommended to use "git" directly to perform update checks and upgrades.

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
    • Android SDK at /Users/ahmdaeyz/Library/Android/sdk
    • Platform android-33, build-tools 33.0.0
    • ANDROID_HOME = /Users/ahmdaeyz/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.13+0-b1751.21-8125866)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.3.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14E300c
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.3)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.13+0-b1751.21-8125866)

[✓] VS Code (version 1.71.0)
    • VS Code at /Users/ahmdaeyz/Downloads/Visual Studio Code.app/Contents
    • Flutter extension version 3.64.0

[✓] VS Code (version 1.75.1)
    • VS Code at /Users/ahmdaeyz/Downloads/sympols/Visual Studio Code.app/Contents
    • Flutter extension version 3.64.0

[✓] Connected device (3 available)
    • Ahmed Aboelyazeed’s iPhone (mobile) • 00008030-000D7458348A802E • ios            • iOS 16.3.1 20D67
    • macOS (desktop)                     • macos                     • darwin-arm64   • macOS 13.4.1 22F82 darwin-arm64
    • Chrome (web)                        • chrome                    • web-javascript • Google Chrome 114.0.5735.198

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.

[bartekpacia]

Hi, this is not possible. To run tests on a physical iOS device, the Flutter app must be built in release mode.

Learn why here.

[ahmdaeyz]

@bartekpacia Noted. How do you advice we do profile the app then? I need to automate traversing the app and get a performance report after say 10 or 20 runs. Your input is highly appreciated.

[bartekpacia]

I haven't done much profiling of Flutter apps, so can't really answer that, sorry.
I guess the official docs are a good starting point. Also, why do you need Patrol when profiling? Is it because there're some native UI elements?

[ahmdaeyz]

This is kind of buried in the docs: profiling utilizing integration testing
I want to reuse the code I wrote for integration tests using patrol and to be able to use the native automator. It's is iOS which is making this hard tho.

[bartekpacia]

Did you have any success with profiling on Android?

[ahmdaeyz]

I'm more concerned with iOS since most of our users are iOS users. But I will investigate and get back to you.

[bartekpacia]

Hi @ahmdaeyz, it appears that running Patrol tests on a physical iOS device in profile mode is possible.

I'll release the hotfix soon, so you'll be able to do:

patrol test --profile

[ahmdaeyz]

Hello @bartekpacia
I've tried running android in profile but the assembleProfile task isn't actually ran. The debug task is the one that is ran. I'm using patrol v1 (because I have a very weird behavior with v2).

I've noticed that the report data doesn't get passed to the flutter driver too. I tried printing the reportData at the end of the test but using the summariser like the docs I sent earlier (at the end).

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar problem, please file a new issue. Make sure to follow the template and provide all the information necessary to reproduce the issue.