Skip to content

Build Performance Ideas

Jump to bottom
Dean Ellis edited this page Jun 26, 2019 · 92 revisions

Xamarin.Android build times are a key pain point for developers.

Parts of the build Xamarin.Android does not have control over:

  • aapt to process Android resources
  • javac to compile Java code to *.class files
  • dx (or soon d8) to convert compiled Java code to Android dex format

But there are still quite a few places we can improve, so we should do that!

Where are we today?

For an in-depth comparison between Visual Studio 2017 15.8.4 and what will ship in 15.9 or 16.0, see results here.

What can developers do today?

The SmartHotel360 app was originally using Xamarin.Forms 2.5.x and Xamarin.Build.Download 0.4.7.

We have MSBuild improvements in both of these packages. After the changes here, I saw drastic improvements to incremental build times:

Build Before Logs (Before) After Logs (After)
First build (fresh) 01:24.93 binlog 01:11.05 binlog
First package 00:28.81 binlog 00:10.47 binlog
First install 00:34.09 binlog 00:15.64 binlog
Second build (no changes) 00:22.20 binlog 00:03.41 binlog
Second package 00:28.70 binlog 00:03.42 binlog
Second install 00:34.27 binlog 00:03.42 binlog
Third build (change XAML) 00:34.45 binlog 00:11.05 binlog
Third package 00:33.12 binlog 00:07.91 binlog
Third install 00:40.62 binlog 00:08.29 binlog

Changes that made this possible:

Update your NuGet packages!

In-Depth Measurements

We are working on a CI setup. More info to come when that is available.

The idea being:

  • On dedicated hardware benchmark builds on a few apps.
  • We generate some nice-looking graphs in PowerBI.
  • We can have a benchmark of where things stand across different Visual Studio versions.
  • Eventually this could run for CI, PR builds, etc.

@pjcollins is working on this. @jonathanpeppers to assist with getting additional data from MSBuild when we get there.

Until then, we will continuing using our Jenkins Plots and do custom measurements locally.

Examples of "slow" MSBuild Tasks in Xamarin.Android

Some of the well-known targets that take up time are:

These are purely under our control, and we should improve them!

Improvements Released

Visual Studio 2019 (15.9):

  • PR 1957: Design-time builds were causing full builds to "always build" and not building incrementally.
  • PR 1938: ResolveSdks caches the output of java -version and javac -version in memory, speeding up builds with multiple Xamarin.Android projects.
  • PR 2088: Fix incremental builds for Xamarin.Forms projects.
  • PR 2093: Improve LINQ usage in ConvertResourcesCases
  • PR 2130: Move inline C# MSBuild task to a compiled assembly
  • PR 2131: Remove unnecessary MSBuild target, simplify inputs to _CompileToDalvik
  • PR 2132: The _BuildLibraryImportsCache target was always running
  • PR 2140 Leave classes.zip uncompressed, to speed up javac and dx
  • PR 2105: Java.Interop is no longer a PCL.

Release Notes for Xamarin.Android 9.1

Visual Studio 2019 RC (16.0):

  • PR 2128: _CopyIntermediateAssemblies improvements
  • PR 2129: Split up the work in ConvertResourcesCases, so some can be skipped
  • PR 2150: More cleanup in ConvertResourcesCases
  • PR 2148: Improve Mono.Cecil usage in BuildApk task
  • PR 2162: Consolidate StripEmbeddedLibraries task with the linker, helps release builds
  • PR 2174: Merge the CheckTargetFrameworks task into ResolveAssemblies
  • PR 2223: Optimize MSBuild $(AssemblySearchPaths)
  • PR 2309: Remove unused Inputs and Outputs from MSBuild targets
  • PR 2019: D8/R8 integration
  • PR 2328: ConvertResourcesCases task, RegexOptions.Compiled and removed more LINQ usage
  • PR 2348: Whitelist support libraries, so ConvertResourcesCases will not run against them
  • PR 2367: Fix an issue on first build when enabling $(AndroidUseAapt2)
  • PR 2535: Remove all usage of temp files in GenerateJavaStubs
  • PR 2540: Use less temp files in ResolveLibraryProjectImports

Release Notes for Xamarin.Android 9.2

16.1:

  • PR 2590: Linker improvements in Debug mode.
  • PR 2612: Use System.Reflection.Metadata in the <ResolveAssemblies/> MSBuild task
  • PR 2624: Use System.Reflection.Metadata in the <GetAdditionalResourcesFromAssemblies/> MSBuild task.
  • PR 2626: Install can skip Build inside of IDEs.
  • PR 2643: <GenerateJavaStubs/> should only load/evaluate TargetFrameworkIdentifier=MonoAndroid assemblies.

Release Notes for Xamarin.Android 9.3

16.2:

  • PR 2896: We don't need to invoke aapt to create R.java for libraries.
  • PR 2930: Perf improvements for <GetImportedLibraries/>
  • PR 2934: Filter for TargetFrameworkIdentifer=MonoAndroid for two MSBuild targets
  • ~~PR 2945:~ _LinkAssembliesNoShrink MSBuild target can skip non-MonoAndroid assemblies in Debug mode.~~
  • PR 2952: Removed lots of unnecessary log messages.
  • PR 2975: Cache aapt2 version call across builds

Ideas for Future Releases

System-Wide Aapt2 Caches

We should investigate using a system-wide Aapt2 cache on NuGet packages. This would speed up <Aapt /> build times across projects.

See the Github issue for details.

There is a general perf issue, posted here.

GenerateJavaStubs

We should optimize <GenerateJavaStubs/> so that it takes assemblies into consideration, so that it skips processing of assemblies which have not changed.

We should likewise split up <GenerateJavaStubs/> because it just does "too much". Portions of it could be skipped when small C#-code changes are made to a single assembly.

<GenerateJavaStubs/> also heavily relies on Mono.Cecil and opens each assembly that was previously opened by other steps in the build, such as the linker. We would like to avoid this and/or use System.Reflection.Metadata instead if possible.

A plan for GenerateJavaStubs

This plan is a decent amount of work, but should have benefits with initial build times, incremental build times, and even help APK sizes. Parts of it could be accomplished in different phases--it would not have to be a single, enormous PR.

  1. Any data needed by <GenerateJavaStubs/> we generate from the linker, which already has every assembly opened. The thought is we would use some XML format, generated for each assembly in a separate file.
  2. We could move RemoveRegisterAttribute into the linker. Have it run against all assemblies. This is somewhat corollary to the goals here, but this would now be possible.
  3. <GenerateJavaStubs/> will be split up: one portion generates AndroidManifest.xml, one typemaps, and one the java stubs.
  4. The different steps should build incrementally: a small .NET assembly change shouldn't "rebuild the world". For example, we would not regenerate AndroidManifest.xml if a netstandard assembly changed, and we would only generate java stubs for MonoAndroid assemblies that have changed.
  5. Java.Interop could generate Java stubs without using Mono.Cecil at all. It should be able to reuse the XML files (and object model) generated by the linker.

Direct .dex generation

Various bits of our build system produce .java files which are then compiled with javac, then "re-compiled" into .dex files. (Such parts include <GenerateJavaStubs/> and <GeneratePackageManagerJava/>, among others.)

It should be possible to directly emit .dex files within some contexts. (Not easy, mind, but possible.) This would avoid the overhead of invoking javac and dx.

@(AndroidJavaSource) will still require the javac and dx invocations, and the @(AndroidJavaLibrary)/@(EmbeddedJar)/@(EmbeddedReferenceJar) build actions will still require dx invocations.

NOTE: this is likely not a viable option. The Android designer needs .class files, it does not operate against Android-specific .dex files.

General Ideas

  • Are there any MSBuild tasks that can run in the background? So other tasks can run in parallel while the work is done?
  • One to mention is GetPrimaryCpuAbi, which is in the proprietary source of Xamarin.Android.

Embrace Reference Assemblies

MSBuild and Roslyn support generating reference assemblies by setting the $(ProduceReferenceAssembly) MSBuild property to True.

Why do we care? What's this mean?

Assume you have a solution with two projects. Project Referenced.csproj has no further references. Project Referencer.csproj has a @(ProjectReference) to Referenced.csproj.

The developer makes a change to something within Referenced.csproj.

Question: Does Referencer.csproj need to be rebuilt?

In the "original" MSBuild world -- the world that Xamarin.Android still lives in -- the answer is yes, Referencer.csproj must always be rebuilt, because the change to Referenced.csproj may contain an API breaking change which would prevent Referencer.csproj from building.

In the new $(ProduceReferenceAssembly)=True world order, the answer is instead maybe: Referencer.csproj only needs to be built if the reference assembly produced as part of the Referenced.csproj build is updated, which in turn only happens when the public API changes. Meaning if a change doesn't alter the public API -- adding comments, fixing a method implementation, adding private/internal members, etc. -- then Referencer.csproj need not be rebuilt at all.

In more concrete terms, assume you have a Xamarin.Forms solution containing a Xamarin.Forms PCL project and a referencing Android App project. Currently, whenever the PCL project is changed, the App project must always be rebuilt. In a $(ProduceReferenceAssembly)=True order, the App project would need to be rebuilt less often.

So let's just export $(ProduceReferenceAssembly)=True! What's holding us back?

The problem is that our current build model is that a .csproj has only one output: the assembly. The assembly contains everything useful: native libraries (embedded .zip resource), Android Resources (embedded .zip resource), environment files, etc. This means that a reference assembly would be unusable: everything requires that the assembly be a full assembly, not some stubbed out reference assembly. Updates to Android Resources would thus be ignored, etc.

Thus, to support $(ProduceReferenceAssembly)=True, we need to change our build system's view of .csproj files. They can no longer "just" produce a .dll. Instead, they need to produce lots of things: native libraries, Android resources, etc.

Then we can update our build system to use reference assemblies for compilation, and go "around" the reference assemblies to pull in referenced assets, as needed.

Update for Reference Assemblies

@jonathanpeppers found out that the following scenario wasn't working as expected:

  1. Create a Xamarin.Forms project + NetStandard
  2. Add <ProduceReferenceAssembly>True</ProduceReferenceAssembly> to the NetStandard library
  3. Build
  4. Modify XAML, Build Again

It turns out that the reference assembly contains EmbeddedResource inside it! So modifying XAML causes the Xamarin.Android head to rebuild! This prevents the feature from helping us at all...

Issue is being fixed in the next Dev16 release: https://github.com/dotnet/roslyn/issues/31197

Running Tasks in Parallel

One of the idea we have had is to kick off some long running tasks in the background while the build is running.

Examples might be

  • Fast Deployment of items to device
  • Patching of Resources.
  • Parallelise Building of the Base apk and compilation of Java code.

The problem is MSBuild does not really support building Tasks or Targets in Parallel only projects. So we need a way to kick off a background task and wait for it to complete later in the build process. The idea is to use GetRegisteredTaskObject to register a global TaskManager which can be used to register background tasks with. We can then use the AsyncTask in conjunction with Task.WhenAll to wait later in the build for those tasks to complete.

public class TaskManager : IDisposable {

	SynchronizedCollection<TPL.Task> tasks = new SynchronizedCollection<TPL.Task> ();
        CancellationTokenSource tcs = new CancellationTokenSource ();

	public void RegisterTask (TPL.Task task)
	{
                lock (tasks.SyncRoot)
		    tasks.Add (task);
	}

	public TPL.Task [] Tasks {
		get {
			return tasks.ToArray ();
		}
	}
   
        public void Dispose ()
        {
            tcs.Cancel ();
        }

        public CancellationToken Token { get { return tcs.Token; } }

	public int Count => tasks.Count; 
}

We can then use code like this within a MSBuild Task to run a background task and instantly return back to MSBuild.

var manager = (TaskManager)BuildEngine4.GetRegisteredTaskObject ("TaskManager", RegisteredTaskObjectLifetime.Build);
if (manager == null) {
	manager = new TaskManager ();
	BuildEngine4.RegisterTaskObject ("TaskManager", manager, RegisteredTaskObjectLifetime.Build, allowEarlyCollection: false);
}
var task =  TPL.Task.Run (async () => {
	await TPL.Task.Delay (10000);	
});
manager.RegisterTask (task);

The in the Execute method of an AsyncTask derived task we can do something like

manager = (TaskManager)BuildEngine4.GetRegisteredTaskObject ("TaskManager", RegisteredTaskObjectLifetime.Build);
TPL.Task task;
if (manager == null || manager.Count == 0) {
	task = TPL.Task.CompletedTask;
} else {
       task = TPL.Task.WhenAll (manager.Tasks);
}
task.ContinueWith(Complete);
base.Execute ();

This can either be in a specific task, say WhenAll or bolted into an existing task like InstallPackagedAssemblies.

One this we need to figure out is how to deal with Logging. Since its a background task we cannot access the normal MSBuild Log.LogXXXX methods. So we will need some way to collect all the logging from the task and then emit the messages, warnings and errors when the task completes.

One of the other problems we thought of was "What if the user cancels the build when a Task is not running". Or if the build is NOT in the WaitAll task.. The solution there is to implement IDisposable on the TaskManager. Because we are registering it as part of RegisteredTaskObjectLifetime.Build, it should be disposed of if the user cancels or when the build completes. If the build completes then we should have got through the Wait Task already so all the registered tasks will be complete. On Cancellation, the CancellationTokenSource on the TaskManager should be Canceled.

Allow users to fast dev custom debug files

Currently the fast dev .__override__ directory can only be used by our build system. Should we look at expanding it to allow users to fast deploy custom files to the fast deployment directory. Files such as test sqlite databases, game textures and models or Json files. This could be handled via the @(Content) item group or via a new @(FastDevUserFiles) item group.

We could then perhaps provide support for loading such files by overriding the AppContext.BaseDirectory to point at the .__override__ directory so users can use a normal File.Open.

Silly Ideas

Should we move the linker to a separate process?

Can we support a "daemon" mode?

On Windows, dotnet build, etc. long-lived processes are used to improve build performance. MSBuild.exe and VBCSCompiler.exe (Roslyn) stay running, so their startup time is avoided during incremental builds. Can we use this idea in Xamarin.Android?

Aapt2 Daemon Mode

aapt2 has a new "daemon mode" feature in the latest 2.19 release.

In order to use this for aapt2 we would need to:

  • Download the latest bits for aapt2 from Maven, and ship them (their license should be OK). The Android SDK ships outdated versions of aapt2, and customers can use different versions of the Android SDK anyway...
  • Add daemon-mode support for <Aapt2Compile/> and <Aapt2Link/> MSBuild tasks

Java Daemon

Can we keep a Java daemon running all the time?

We call java.exe to run various jar files:

  • apksigner.jar
  • desugar_deploy.jar
  • dx.jar (or r8.jar)
  • proguard.jar

Can we use a daemon to reduce JVM startup time? gradle does this. Can we use an existing tool to do this?

@jonathanpeppers did an experiment using Facebook's nailgun:

https://github.com/facebook/nailgun

It is a Java daemon used by Facebook's Buck build system.

A comparison of running d8.jar through a running nailgun server:

  • 1st (cold): ~12 seconds
  • 2nd: ~7 seconds
  • 3rd: ~5 seconds

Beyond the 3rd run, it was stable at ~5 seconds.

Unfortunately nailgun opens a TCP port on localhost to function. We may need to avoid doing this and implement something ourselves instead.

General

Contributing

Tips & tricks

Specifications

Release Notes

Clone this wiki locally