Can we remove the CustomMakeBuilder:CopySource from custom_make workflow?

[komapa]

Description:

I already oped a ticket with aws-cli-sam here aws/aws-sam-cli#3350 about one of the problems/limitations of the CustomMakeBuilder:CopySource step. I was able to work around it for the time being but I just saw another problem with copying source code to a scratch directory: go build cache becomes completely unusable. The issue is that every time the golang source code is in a different directory so the GOCACHE becomes useless as it caches based on the path of the source files.

You can see how for a relatively complicated Lambda, building with sam build is much slower because each time it needs to recompile not just the source code but also ALL the imports.

So, I went in and edited workflow.py and changed self.actions = [CopySourceAction(source_dir, scratch_dir, excludes=self.EXCLUDED_FILES), make_action] to self.actions = [make_action] and then sam build worked just as before but this time it did not copy any files and obviously was able to reuse the GOCACHE.

So my ask is, why do we have the copy source file into a scratch directory step? And if it makes sense for some use cases, can we have it configurable somehow so it can be turned off when not desired? Thank you!

Observed result:

These are timings of running the build of the same lambda 3 different ways:

1. go build -ldflags="-s -w" -o target/lambda 0.65s user 1.33s system 297% cpu 0.669 total
2. make build-Lambda 0.67s user 1.40s system 288% cpu 0.719 total
3. sam build Lambda 40.38s user 11.85s system 513% cpu 10.178 total

Expected result:

I would expect sam build to take the same time as go build

sam build Lambda 1.47s user 1.61s system 153% cpu 2.008 total

[komapa]

Any update folks?

[mndeveci]

Thanks for your feedback @kirooshu

We've got couple of similar requests which suggest to use existing folder when building each function. As of now, we are using a temporary directory which will copy all source files in the beginning and use that folder for building, and copy built artifacts back to designated build folder (.aws-sam/build/{FunctionId})

I will take this to our PMs to schedule the work.

[m-ronchi]

any news? it's been a year...

[torresxb1]

Closing so that we can track in the above issue (Feature request: Build in source). There's multiple open issues relating to similar problems, so we want to have a central place to track.