DynamoDB resource ignoring the configuration parameter 'max_attempts' for retries

[pinarkaymaz6]

Describe the bug

For our DynamoDB implementation in a Lambda, I pass different values to the configuration parameter 'max_attempts' to override the default retry max attempts. However, it does not seem to take effect. I still see the default retry number in the CloudWatch logs of the Lambda.

Here is the code snippet for the dynamodb with retry config:

import os
import boto3
from botocore import config

RETRY_CONFIG = config.Config(
   retries = {
      'max_attempts': int(os.getenv('RETRY_MAX_ATTEMPTS', default=3)),
      'mode': 'standard'
   }
)

db = boto3.resource('dynamodb', config=RETRY_CONFIG)
table = db.Table(table_name)

With the default value, I see the following in the logs stating the default retry value 3:
[DEBUG] 2023-11-13T12:43:14.106Z Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None)

When I set the environmental variable RETRY_MAX_ATTEMPTS=7, I still see the same log as the default one.
[DEBUG] 2023-11-13T12:43:14.106Z Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None)

Expected Behavior

Converted retries value in the logs should be the new value.

Example: Retry total should be 7 when max_attempts is set to 7:
Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None)

Current Behavior

Even if I pass a new value to 'max_attempts' to override retry configuration, the retry count is still the same in the logs:

When I set the environmental variable RETRY_MAX_ATTEMPTS=7, I still see the same log as the default one.
[DEBUG] 2023-11-13T12:43:14.106Z Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None)

Reproduction Steps

import os
import boto3
from botocore import config

RETRY_CONFIG = config.Config(
   retries = {
      'max_attempts': int(os.getenv('RETRY_MAX_ATTEMPTS', default=3)),
      'mode': 'standard'
   }
)

db = boto3.resource('dynamodb', config=RETRY_CONFIG)
table = db.Table(table_name)

• Use different RETRY_MAX_ATTEMPTS values
• Check the logs to see retry count

Possible Solution

No response

Additional Information/Context

No response

SDK version used

Python 3.8.1, boto3 1.28.78, botocore 1.31.78

Environment details (OS name and version, etc.)

AWS Lambda Python runtime environment, Linux, Python 3.8

[RyanFitzSimmonsAK]

Hi @pinarkaymaz6, thanks for reaching out. I had some trouble reproducing this issue; Lambda was able to read the environment variable I had set and created a RETRY_CONFIG with the correct value. Could you confirm that your script is successfully reading the environment variable and that the RETRY_CONFIG is correct? Thanks!

[pinarkaymaz6]

Hi @RyanFitzSimmonsAK , thanks for your answer!
I can confirm that my script is able to read the environmental variables and the RETRY_CONFIG itself is correctly created. What I'm having trouble is to confirm that dynamodb actually uses the new max_attempts variable that is being set. I enabled DEBUG logs for all botocore and boto3 loggers, but i'm not able to see any logs that mentions max_attemps or retry quota value.

I did the implementation according to the boto3 documentation here. However I am having trouble testing this. I was hoping to see some values in the logs. However it does not seem to be the case. How did you confirm that dynamodb uses the new config, for example, if you set max_attempts to 7?

[RyanFitzSimmonsAK]

The config is added to the client during client creation under meta. total_max_attempts should be your configured attempts value, plus one. Just for the sake of simplifying the testing, can you specify a value for max_attempts rather than tying it to an environment variable? Also, did you get this (Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None)) logging with just the default boto3.set_stream_logger('')?

[pinarkaymaz6]

After some digging, I realised that the log Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None) comes from requests.Session and not from botocore.

Even though my original question figuring out how to confirm retry quota in dynamodb queries still stands, I think this issue is not a Bug. So I'll close this one and open a new one with General Question.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[pinarkaymaz6]

Hey @RyanFitzSimmonsAK , here is my question: #3934