Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Can't run more than 4 emulators in MacPro on 2.0.0-SNAPSHOT for test sharding #505

Closed
KyleLearnedThis opened this issue Dec 7, 2017 · 9 comments
Closed

Can't run more than 4 emulators in MacPro on 2.0.0-SNAPSHOT for test sharding #505

KyleLearnedThis opened this issue Dec 7, 2017 · 9 comments

Comments

@KyleLearnedThis
Copy link

KyleLearnedThis commented Dec 7, 2017 •
edited
Loading

Hi,
It seems that I can't run more than 4 emulators/phones without tests randomly failing due to InstallException on Spoon Runner 2.0.0-SNAPSHOT for test running in sharding mode.

  • Hardwares: OSX 10.12.6 MacPro
  • Emulators: Stock Google Pixel 2
  • APKs: (build with 4.3.1 Gradle, debug mode for both test and app)
  • Android Studio 3.0.1

The commit I am on:
commit 14d7b38a2569dbfcf2d0db3db102e097b3099f92 (HEAD -> master, origin/master, origin/HEAD) Author: Bartłomiej Hołota <[email protected]> Date: Tue Nov 28 17:44:03 2017 +0100 Fix instrumentation args parsing (#495)

The number 4 seems to be magic, I can run any combinations of phones or emulators, as long its up to 4. And I monitored CPU usage, I am not pushing MacPro to stress at all. Also tried the same thing with my MacBook Laptop, this happened as well.

2017-12-06 15:19:17 [SR.runTests] Executing instrumentation suite on 6 device(s). 03:19:17 E/Device: Error during Sync: Broken pipe 03:19:17 E/Device: Error during Sync: Broken pipe 2017-12-06 15:19:17 [SDR.run] InstallException while install other apk on device [emulator-5564] 2017-12-06 15:19:17 [SDR.run] InstallException while install other apk on device [emulator-5562] com.android.ddmlib.InstallException: Broken pipe at com.android.ddmlib.Device.installPackage(Device.java:888) at com.squareup.spoon.SpoonDeviceRunner.run(SpoonDeviceRunner.java:167) at com.squareup.spoon.SpoonRunner$1.run(SpoonRunner.java:211) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcherImpl.write0(Native Method) at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) at sun.nio.ch.IOUtil.write(IOUtil.java:65) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471) at com.android.ddmlib.AdbHelper.write(AdbHelper.java:815) at com.android.ddmlib.AdbHelper.write(AdbHelper.java:794) at com.android.ddmlib.AdbHelper.setDevice(AdbHelper.java:858) at com.android.ddmlib.SyncService.openSync(SyncService.java:195) at com.android.ddmlib.Device.getSyncService(Device.java:521) at com.android.ddmlib.Device.syncPackageToDevice(Device.java:923) at com.android.ddmlib.Device.installPackage(Device.java:884) ... 5 more com.android.ddmlib.InstallException: Broken pipe at com.android.ddmlib.Device.installPackage(Device.java:888) at com.squareup.spoon.SpoonDeviceRunner.run(SpoonDeviceRunner.java:167) at com.squareup.spoon.SpoonRunner$1.run(SpoonRunner.java:211) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcherImpl.write0(Native Method) at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) at sun.nio.ch.IOUtil.write(IOUtil.java:65) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471) at com.android.ddmlib.AdbHelper.write(AdbHelper.java:815) at com.android.ddmlib.AdbHelper.write(AdbHelper.java:794) at com.android.ddmlib.AdbHelper.setDevice(AdbHelper.java:858) at com.android.ddmlib.SyncService.openSync(SyncService.java:195) at com.android.ddmlib.Device.getSyncService(Device.java:521) at com.android.ddmlib.Device.syncPackageToDevice(Device.java:923) at com.android.ddmlib.Device.installPackage(Device.java:884) ... 5 more 2017-12-06 15:19:44 [SDR.handleFiles] Found class name dirs: [] 2017-12-06 15:19:57 [SDR.handleFiles] Found class name dirs: [] 2017-12-06 15:19:59 [SDR.handleFiles] Found class name dirs: [] 2017-12-06 15:20:12 [SDR.handleFiles] Found class name dirs: []

The reason I used 2.0.0-SNAPSHOT because I need the workaround for #461. Because for the APKs build with the latest gradle wrapper 4.3.1 comes with latest Android Studio 3.0.1, I can't them run with spoon-runner-1.7.1-jar-with-dependencies.jar

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 122 at com.squareup.spoon.axmlparser.StringBlock.getShort(StringBlock.java:241) at com.squareup.spoon.axmlparser.StringBlock.getRaw(StringBlock.java:99) at com.squareup.spoon.axmlparser.StringBlock.read(StringBlock.java:66) at com.squareup.spoon.axmlparser.AXMLParser.doStart(AXMLParser.java:200) at com.squareup.spoon.axmlparser.AXMLParser.<init>(AXMLParser.java:48) at com.squareup.spoon.SpoonInstrumentationInfo.parseFromFile(SpoonInstrumentationInfo.java:57) at com.squareup.spoon.SpoonRunner.run(SpoonRunner.java:117) at com.squareup.spoon.SpoonRunner.main(SpoonRunner.java:695) mac-coolserver-80:Spoon jenkins$

@JakeWharton
Copy link
Collaborator

JakeWharton commented Dec 7, 2017 via email

@edenman edenman closed this as completed Dec 7, 2017
@KyleLearnedThis
Copy link
Author

KyleLearnedThis commented Dec 7, 2017 •
edited
Loading

But why does this worked on 1.7.1 with APKs build on 3.X.X Gradle wrapper on the same environment with 6+ emulators? @JakeWharton @edenman

@KyleLearnedThis
Copy link
Author

KyleLearnedThis commented Dec 8, 2017 •
edited
Loading

@JakeWharton @jaredsburrows @dweebo @edenman can someone reopen this? this is prevent me from using Spoon.

I really tried hard to use Spoon for test sharding but maybe It's not meant to be for me.

  • Can't run latest apk build in gradle 4.3.1 with issue java.lang.ArrayIndexOutOfBoundsException when using gradle plugin 3.0.0.x #461 How do i get branches for 1.7.1 to apply fixes locally for? (I saw the 1.7.1-Parent branch, but it's not complete code, is that delta of the code) (Maybe build 1.7.1 Spoon with 4.3.1?)
  • Use unreleased Spoon 2.0.0 (Tried, can't be run more than 4 emulators/phone but 1.7.1 with older APKS worked on the same test environment with 6+ emulators okay)
  • Downgrade Gradle to 3.X.X to build our latest APKs (bad idea, tried, won't build)
  • Spoon Gradle plugin is broken for me. BUG some commrade suggested apply patches myself).
  • Spoon command line changed in 2.0.0 (why is --adb-timeout won't take string like 10, or 10s on 2.0.0?) (or spoon-runner: unrecognized option '--single-instrumentation-call')
  • Also tried to split tests into two shards so it will be using less than 4 devices at 1 process

java -jar spoon-runner-2.0.0-SNAPSHOT-all.jar
/Users/jenkins/labs/Spoon/new/Hello-prod-androidTest.apk
/Users/jenkins/labs/Spoon/new/Hello-prod-debug.apk
--serial emulator-5554
--serial emulator-5556
--serial emulator-5558
--shard
--class-name com.hello.whateverTest
--sdk /Users/$USER/Library/Android/sdk
-e numShards=2 shardIndex=0

java -jar spoon-runner-2.0.0-SNAPSHOT-all.jar
/Users/jenkins/labs/Spoon/new/Hello-prod-androidTest.apk
/Users/jenkins/labs/Spoon/new/Hello-prod-debug.apk
--serial emulator-5560
--serial emulator-5562
--serial emulator-5564
--shard
--class-name com.hello.whateverTest
--sdk /Users/$USER/Library/Android/sdk
-e numShards=2 shardIndex=1

And this turns out that entire suite was run three times total.

Alternatives:

  • "Multi-configuration project" Jenkins option for spoon, maybe limit each test uses selected 4 devices.
  • Firebase
  • GoJuno Composer
  • ADB test sharding.

@jaredsburrows
Copy link
Contributor

@KyleLearnedThis Spoon 2.0.0 has not been released and there is documentation lacking for it.

I too cannot use it because it seems to break sharding. I am not a collaborator or owner, I cannot open this issue again.

@mattbutlar
Copy link

mattbutlar commented Mar 28, 2018 •
edited
Loading

@KyleLearnedThis @JakeWharton @jaredsburrows @edenman I just found this issue as we were dealing with this as well and I found what it was and wanted to update you guys in case you wanted to keep using spoon 2.0.

This broken pipe issue with 4 or more devices is because of changes to the ddmlib in version 26.0.1.

I am digging into exactly what change broke it and why but for now, changing the version in the build.gradle of the snapshot does get it working again with more than 4 devices. If I come up with a different/better solution after more digging, I will let you know.

@jaredsburrows
Copy link
Contributor

@mattbutlar Honestly, it might be time to move on to a new tool or use the adb/gradlew commands directly.

@mattbutlar
Copy link

I would agree if we didn't already have multiple test suites setup using it and no one with time to currently explore other options.

@jaredsburrows
Copy link
Contributor

@mattbutlar Its good for sharding with screenshots reports. You could try using https://github.com/gojuno/composer.

@KyleLearnedThis
Copy link
Author

KyleLearnedThis commented Jun 8, 2018 •
edited
Loading

Yeah I was able to use GoJuno Composer for test sharding and very happy with it in past months.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@JakeWharton @edenman @jaredsburrows @mattbutlar @KyleLearnedThis