Boto3 Import Error after Lambda Runtime Environment Change

[Burakt94]

Describe the bug

Currently, I am working with Lambda functions using Python 3.9 with custom layers including botocore==1.24.21

Recently, new runtime environment is introduced and boto3 version changed from 1.26.90 to 1.34.42 (because update runtime version is in Auto mode as default.)

After this change, I can no longer import boto3 in functions. It seems that there is a conflict between botocore==1.24.21 and boto3==1.34.42

This is the error when I try to import boto3

{
"errorMessage": "Unable to import module 'lambda_function': cannot import name 'DEPRECATED_SERVICE_NAMES' from 'botocore.docs' (/opt/python/botocore/docs/init.py)",
"errorType": "Runtime.ImportModuleError",
"requestId": "requestId",
"stackTrace": []
}

Expected Behavior

I would expect backward compatibility with an older botocore module when importing boto3. Because all lambda functions have default behaviour as automatic to update runtime environment, this leads to unexpected crashes for working system.

Current Behavior

This is also full lambda output.

Response
{
"errorMessage": "Unable to import module 'lambda_function': cannot import name 'DEPRECATED_SERVICE_NAMES' from 'botocore.docs' (/opt/python/botocore/docs/init.py)",
"errorType": "Runtime.ImportModuleError",
"stackTrace": []
}

Function Logs
START RequestId: Version: $LATEST
LAMBDA_WARNING: Unhandled exception. The most likely cause is an issue in the function code. However, in rare cases, a Lambda runtime update can cause unexpected function behavior. For functions using managed runtimes, runtime updates can be triggered by a function change, or can be applied automatically. To determine if the runtime has been updated, check the runtime version in the INIT_START log entry. If this error correlates with a change in the runtime version, you may be able to mitigate this error by temporarily rolling back to the previous runtime version. For more information, see https://docs.aws.amazon.com/lambda/latest/dg/runtimes-update.html
[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': cannot import name 'DEPRECATED_SERVICE_NAMES' from 'botocore.docs' (/opt/python/botocore/docs/init.py)
Traceback (most recent call last):END

Reproduction Steps

Try simple following code with environment boto3==1.34.42 botocore==1.24.21

import boto3

Possible Solution

No response

Additional Information/Context

No response

SDK version used

boto3==1.34.42 botocore==1.24.21

Environment details (OS name and version, etc.)

Lambda Python 3.9

[AyhanSetirekli]

We had the same issue today with Python 3.8 updating from v43 to v44

[nateprewitt]

Hi @Burakt94, @AyhanSetirekli,

The issue you're seeing is due to including a partial installation of boto3 in your Lambda Package. boto3 has a tight coupling to the version of Botocore it requires, usually released within the same minor version band. botocore 1.24.21 is roughly 2 years old at this point and won't be compatible with recent versions of boto3.

To avoid these sorts of incompatibilities, you'll need to either include all boto3 dependencies or remove them from your package to allow Lambda to manage them. Full documentation on Working with .zip file archives for Python Lambda functions and Best practices for working with AWS Lambda functions have guidance on what should (or shouldn't) be included in your Lambda runtime. If you would like to bundle boto3, it should include boto3, botocore, and s3transfer to ensure forward compatibility with upgrades.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.