Consider migrating from ios-deploy to go-ios

[bartekpacia]

To run tests on a physical iOS device, patrol_cli uses ios-deploy (the user has to brew install ios-deploy. They can also run patrol doctor to see the status).

I'd call ios-deploy a fragile dependency:

• it's non-debuggable because it's basically a single Objective-C file containing 3000+ lines of code with no tests.
• is pretty much in maintenance mody (this probably stems from the fact that no one knows what's happening inside of it)
• it's somewhat flaky: see also Waiting for ios-deploy to launch the app is taking unusually long time (30s) #949

Meanwhile go-ios:

• is more actively maintained
• written in quite clean Go
• outputs all logs in somewhat standardized JSON
• has its own Discord server.
• has much more features than ios-deploy (for example, ios lang or ios ps)
• we could use it instead of xcode test-without-building, which is kinda annoying (see xcodebuild doesn't exit immediately after test finish #850)

I tested go-ios and it already does what we currently do with ios-deploy.

Install the app under test:

ios install --path build/ios_integ/Build/Products/Debug-iphoneos/Runner.app

Install the UI test instrumentation:

ios install --path build/ios_integ/Build/Products/Debug-iphoneos/RunnerUITests-Runner.app

Run the app under test:

ios debug build/ios_integ/Build/Products/Debug-iphoneos/Runner.app

and then run the UI test instrumentation:

ios runtest \
  --bundle-id pl.leancode.patrol.Example \
  --test-runner-bundle-id pl.leancode.patrol.Example.RunnerUITests.xctrunner \
  --xctest-config RunnerUITests.xctest

Works just fine (tested with notifications_test.dart).

[bartekpacia]

Additionally, I'd like patrol_cli to have a clearer lifecycle:

• patrol build builds the app binaries (app under test and instrumentation app)
• patrol install installs the app binaries to simulator/device, but doesn't run them. Requires the app binaries to be built beforehand.
• patrol run runs the installed instrumentation app. Requires the apps to be installed beforehand.

Of course, we'll keep the patrol test command, which does all the 3 things in a single invocation.

---

Notes:

• patrol install on iOS can use:
  • simulator: xcrun simctl install booted <Runner.app path>
  • device: ios install --path=<ipaOrAppFolder> (using go-ios)

The same changes should also be applied for Android, to keep behavior consistent.

[bartekpacia]

patrol_cli doesn't depend on ios-deploy anymore (see #1303), but this issue still stands, especially the comment above about splitting the lifecycle.

[bartekpacia]

Note for the future: ios runtest fails to run tests and I don't know why. Meanwhile patrol test --release works fine (it uses xcodebuild test-without-building under the hood).

Logs

$ ios runtest \
  --bundle-id pl.leancode.patrol.Example \
  --test-runner-bundle-id pl.leancode.patrol.Example.RunnerUITests.xctrunner \
  --xctest-config build/ios_integ/Build/Products/Runner_iphoneos16.4-arm64.xctestrun
{"level":"info","msg":"no udid specified using first device in list","time":"2023-05-23T17:08:41+02:00","udid":"00008020-001C499A3420802E"}
{"bundleID":"pl.leancode.patrol.Example.RunnerUITests.xctrunner","channel_id":"com.apple.instruments.server.services.processcontrol","level":"info","msg":"Launching process","time":"2023-05-23T17:08:42+02:00"}
{"channel_id":"com.apple.instruments.server.services.processcontrol","level":"info","msg":"Process started successfully","pid":714,"time":"2023-05-23T17:08:42+02:00"}
{"fields.msg":"2023-05-23 17:08:43.071615+0200 RunnerUITests-Runner[714:125747] Running tests...\n","fields.time":229514261701,"level":"info","msg":"outputReceived:fromProcess:atTime:","pid":714,"time":"2023-05-23T17:08:42+02:00"}
{"level":"info","msg":"device called local method","sel":"_XCT_didFailToBootstrapWithError:","time":"2023-05-23T17:08:42+02:00"}
{"fields.msg":"2023-05-23 17:08:43.085256+0200 RunnerUITests-Runner[714:125747] Failed to create a bundle instance representing '/private/var/containers/Bundle/Application/6755DAAF-9F80-4F86-9DF0-412FCC173852/RunnerUITests-Runner.app.ipa.app/PlugIns/build/ios_integ/Build/Products/Runner_iphoneos16.4-arm64.xctestrun'. Check that the bundle exists on disk.\n","fields.time":229514589851,"level":"info","msg":"outputReceived:fromProcess:atTime:","pid":714,"time":"2023-05-23T17:08:42+02:00"}
{"level":"error","msg":"DeviceConnection failed writing 402 bytes, only 0 sent","time":"2023-05-23T17:08:43+02:00"}
{"level":"error","msg":"Failed sending: write unix -\u003e/var/run/usbmuxd: write: broken pipe","time":"2023-05-23T17:08:43+02:00"}
{"channel_id":"dtxproxy:XCTestDriverInterface:XCTestManager_IDEInterface","error":"write unix -\u003e/var/run/usbmuxd: write: broken pipe","level":"info","methodselector":"_IDE_startExecutingTestPlanWithProtocolVersion:","msg":"failed starting invoking method","time":"2023-05-23T17:08:43+02:00"}
{"level":"error","msg":"write unix -\u003e/var/run/usbmuxd: write: broken pipe","time":"2023-05-23T17:08:43+02:00"}
{"error":"write unix -\u003e/var/run/usbmuxd: write: broken pipe","level":"info","msg":"Failed running Xcuitest","time":"2023-05-23T17:08:43+02:00"}