sam local build does not copy .so files

[jrowen]

Description

The build process doesn't copy over .so files when building a Python Lambda function.

Steps to reproduce

Install the ptvsd library:

pip install --upgrade -t ./package ptvsd

Build a Lambda function that uses ptvsd

sam build --template ./aws/template.yaml --use-container

The template file should include a reference to the package folder noted earlier (like CodeUri: ../package).

Observed result

When following the ptvsd debugging steps, the following error is thrown from the sam local invoke process:

Expected: /var/task/ptvsd/_vendored/pydevd/pydevd_attach_to_process/attach_linux_amd64.so to exist.

Looking into the .aws-sam/build/Function/ptvsd/pydevd/pydevd_attach_to_process folder, I can confirm the .so files were not copied over. Looking in package/ptvsd/_vendored/pydevd/pydevd_attach_to_process, I can confirm that the .so files are present.

Expected result

Debugging process runs without error.

[sanathkr]

This is indeed a bug. We intentionally ignore certain files, .so files are one of them - https://github.com/awslabs/aws-lambda-builders/blob/develop/aws_lambda_builders/workflows/python_pip/workflow.py#L27

The intention behind excluding .so files is to prevent people from inadvertently including .so files built for one platform (say Mac) and hoping it will run on Lambda.

Since ptvsd produces .so files it needs to be compiled for the Lambda environment. I would recommend adding ptvsd to your requirements.txt during development and removing it before you push to production.

If this works, I can update the documentation to reflect the "sam build" usecase as well. Of course, it will be easier if sam build had support a requirements-dev.txt that gets built only for development.

[jrowen]

I tried adding ptvsd to my requirements.txt file, but it didn't seem to resolve the issue.

My current workaround is to run the following after the build step, which eliminates the error

cp ./package/ptvsd/_vendored/pydevd/pydevd_attach_to_process/*.so .aws-sam/build/Function/ptvsd/_vendored/pydevd/pydevd_attach_to_process

[kbrowns]

Are there any plans to support some kind of .samignore file type of mechanism? This really needs to be customizable by the user IMO. I'm seeing that sam build is including my .vagrant directory and other things I don't want it including and is bloating my deployment packages (which as you know have a size limitation. Many tools (e.g. Vagrant) need to have files in the root directory

[tomlockwood]

I've precompiled pymssql using freetds and want to use this with my lambda. It'd be great for me if I could do this with SAM somehow.

[rtrive]

This feature will be useful for certain libraries that require .so files

[torresxb1]

We have a few issues asking for a similar thing in ignore: or a .samignore file. Closing so that we can track here: aws/aws-lambda-builders#185

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.