Runtime Performance Ideas

Runtime performance is important for our users. We care about the startup performance and also the performance in general.

Comparisons

All tests below are running on a Pixel 5. The average of 10 runs using the Activity Displayed time.

Animations were also disabled with:

> adb shell settings put global window_animation_scale 0
> adb shell settings put global transition_animation_scale 0
> adb shell settings put global animator_duration_scale 0

Versions tested:

• Xamarin.Android 12.1.99.56 and Visual Studio 2022 17.1 Preview
• Xamarin.Forms 5.0.0.2196 and Visual Studio 2022 17.1 Preview
• Android 31 Preview 10/11 for .NET 6 and Visual Studio 2022 17.1 Preview
• .NET MAUI Preview 10/11 and Visual Studio 2022 17.1 Preview

Projects tested:

• Xamarin.Android, Xamarin.Android Single View Application template
• Xamarin.Forms, Xamarin.Forms Flyout template
• dotnet new android
• dotnet new maui

Mode:

• JIT = plain Release build
• AOT = AotAssemblies=true or RunAOTCompilation=true (.NET 6) and AndroidEnableProfiledAot=true

I did not measure Full AOT, because it normally provides slower startup and larger app sizes.

Application	Mode	Framework	Time(ms)
XamarinAndroidApp	JIT	Xamarin	334.1
XamarinAndroidApp	AOT	Xamarin	306.5
dotnet new android	JIT	MAUI P10	265.4
dotnet new android	AOT	MAUI P10	210.5
dotnet new android	JIT	MAUI P11	269.4
dotnet new android	AOT	MAUI P11	197.4
XamarinFormsApp	JIT	Xamarin	1369.5
XamarinFormsApp	AOT	Xamarin	817.7
dotnet new maui	JIT	MAUI P10	1078.0
dotnet new maui	AOT	MAUI P10	683.9
dotnet new maui	JIT	MAUI P11	1072.6
dotnet new maui	AOT	MAUI P11	677.4

Recent improvements

• xamarin-android#6482 Profiled AOT can Full AOT the main assembly

• xamarin-android#6311 Embedded assemblies store

• xamarin-android#6210 Speed up P/Invoke override lookups

• xamarin-android#6188 Pre-allocate buffers for embedded assembly names

• xamarin-android#6171 Profiled AOT support for .NET 6

• dotnet/maui#2606 use a custom Resource.designer.cs

• dotnet/maui#2496 add AOT profile for .NET MAUI

Measurements

Type of measurements we use to analyze the runtime performance

• Regular test measurements with time and size plots

• Profiler - we use manual measurements with profiler to analyze the performance of the managed code. Usually the calls, alloc and sample reports.

  • NDK native profiler can be used to profile Mono + Xamarin.Android native runtimes

• JIT times measurements We can now measure the JIT time per method, using the debug.mono.log=timing property and on device methods.txt output.

  • Example of JIT times on Pixel XL 2, when running Xamarin.Forms test

Findings

• Not preloading all app assemblies during runtime init (Java_mono_android_Runtime_init in monodroid-glue.cc) buys us 100ms.

• Application started via an Intent is ~2.5x slower than one started "normally" by clicking the app icon in the launcher (sometimes even slower than that - e.g. logger initialization in managed code takes ~4ms with normal startup, 28ms with the "Intent" one). Measured on Pixel 3 XL. Nobody knows why it happens, yet.

• Xamarin.Forms apparently attempt to reflection-load all the assemblies in the app, this causes our savings in the native init code to disappear once we hit the managed land.

• It is important to consider the time to JIT as it takes about 50% of the app startup time

Ideas

• New JNI marshal methods to speedup startup by avoiding System.Reflection.Emit use. That would hopefully provide faster marshal methods registration and reduce the JIT time and apk size

• Run some of the regular tests with profiling. Process and use the data in the time plots

• Measure performance regularly on devices as well - today we only use Android emulator to run the tests by the build bots. It might be also worth to run performance measurements on dedicated machine and collect data on mobile devices and emulators. That might be more stable compared to the build bots.

• p/invoke optimization Mono runtime uses dlsym to look up native functions in DSOs whenever the p/invoke is used. The same thing happens on Android, especially for all the __Internal externals. However, this is completely unnecessary since by the time Mono runtime is initialized by us, we already know addresses of all the exported functions. The idea is to inform Mono about the addresses of those functions so that it can skip the lookup. This requires changes to the Mono runtime.

• Xamarin.Android runtime uses JNI's FindClass method quite a lot to look up Java classes during startup. This might not be necessary if we store references to those classes in the Java mono.android.Runtime class which calls into our native initialization sequence. This way we can skip the "reflection" part and let ART do the job for us before our native code runs.

• Interpreter? Rumor has it that mono's new interpreter is "reasonably fast," in that it can execute some methods in less time than it would take to normally JIT + execute that same method. This makes it potentially interesting during process startup, when many methods are executed only once, e.g. JNIEnv.Initialize(), the AndroidRuntime constructor, and some others. This would have a "cost" in larger .apk sizes (to include the interpreter), but the tradeoff may very well be worth it.

• We generate two type map files, for managed to java and reverse lookups. They are stored in the apk and subsequently loaded into memory during runtime startup (and kept in memory). We can do better than that - we can generate an native assembler file with the data, compile with as (which we'll have to ship, but it's small and standalone) and relink the XA runtime when building the APK (Android SDK ships with the native linker). The data would be placed in a read-only section, loaded by the system loader for us and immediately available whenever it is required - without incurring any runtime overhead whatsoever. We can also reuse the generated assembly for other things (environment variables, flags - for instance whether the app uses embedded DSOs, LLVM, AOT etc). MSBuild side caching would make sure that we don't relink the runtime unnecessary. The gains can be quite worth the effort of developing this. Assembly generation can easily be implemented by creating a custom Stream implementation which can then be used with TypeNameMapGenerator without changing the latter at all.

Planned improvements

• Fix Bcl test measurements and add them to the plots again

• Generate and use JNI marshal methods for constructors used in the ConstructorBuilder to avoid last System.Reflection.Emit use