guardduty.list_findings API call states invalid value

[gyoza]

Describe the bug

Using guardduty.list_findings with DetectorId="string" produces an error stating 'detectorId has an invalid value'

Expected Behavior

The api call to accept my str.

Current Behavior

            DetectorId="xxxxx", # replaced actual detector id from console with xxxxx
            FindingCriteria={},
            PaginationConfig={"PageSize": 100},
        )

Produces this

botocore.errorfactory.BadRequestException: An error occurred (BadRequestException) when calling the ListFindings operation: The request is rejected because the parameter detectorId has an invalid value.

During handling of the above exception, another exception occurred:

Reproduction Steps

        client = session.client("guardduty")
        paginator = client.get_paginator("list_findings")
        print(self.detectorid)
        page_iterator = paginator.paginate(
            DetectorId="xxxxxxx",
            FindingCriteria={},
            PaginationConfig={"PageSize": 100},
        )

Possible Solution

None avail:(

Additional Information/Context

I am 100% sending a string and it is the correct detector ID as I am using it in terraform so.. I think i found an actual bug.

SDK version used

boto3==1.28.61 botocore==1.31.61

Environment details (OS name and version, etc.)

  System Version: macOS 14.0 (23A344)       Kernel Version: Darwin 23.0.0

[RyanFitzSimmonsAK]

Hi @gyoza, thanks for reaching out. I wasn't able to reproduce this behavior using the sample code you provided. Could you provide debug logs of this behavior? You can get debug logs by adding boto3.set_stream_logger('') to the top of your script, and redacting any sensitive information. Thanks!

[gyoza]

2023-10-06 12:49:35,665 botocore.credentials [DEBUG] Looking for credentials via: env
2023-10-06 12:49:35,665 botocore.credentials [INFO] Found credentials in environment variables.
2023-10-06 12:49:35,665 botocore.loaders [DEBUG] Loading JSON file: /opt/homebrew/lib/python3.9/site-packages/botocore/data/endpoints.json
2023-10-06 12:49:35,671 botocore.loaders [DEBUG] Loading JSON file: /opt/homebrew/lib/python3.9/site-packages/botocore/data/sdk-default-configuration.json
2023-10-06 12:49:35,671 botocore.hooks [DEBUG] Event choose-service-name: calling handler <function handle_service_name_alias at 0x1047e0700>
2023-10-06 12:49:35,678 botocore.loaders [DEBUG] Loading JSON file: /opt/homebrew/lib/python3.9/site-packages/botocore/data/guardduty/2017-11-28/service-2.json
2023-10-06 12:49:35,686 botocore.loaders [DEBUG] Loading JSON file: /opt/homebrew/lib/python3.9/site-packages/botocore/data/guardduty/2017-11-28/endpoint-rule-set-1.json.gz
2023-10-06 12:49:35,686 botocore.loaders [DEBUG] Loading JSON file: /opt/homebrew/lib/python3.9/site-packages/botocore/data/partitions.json
2023-10-06 12:49:35,687 botocore.hooks [DEBUG] Event creating-client-class.guardduty: calling handler <function add_generate_presigned_url at 0x10472b820>
2023-10-06 12:49:35,687 botocore.configprovider [DEBUG] Looking for endpoint for guardduty via: environment_service
2023-10-06 12:49:35,687 botocore.configprovider [DEBUG] Looking for endpoint for guardduty via: environment_global
2023-10-06 12:49:35,687 botocore.configprovider [DEBUG] Looking for endpoint for guardduty via: config_service
2023-10-06 12:49:35,687 botocore.configprovider [DEBUG] Looking for endpoint for guardduty via: config_global
2023-10-06 12:49:35,687 botocore.configprovider [DEBUG] No configured endpoint found.
2023-10-06 12:49:35,727 botocore.endpoint [DEBUG] Setting guardduty timeout as (60, 60)
2023-10-06 12:49:35,728 botocore.loaders [DEBUG] Loading JSON file: /opt/homebrew/lib/python3.9/site-packages/botocore/data/_retry.json
2023-10-06 12:49:35,728 botocore.client [DEBUG] Registering retry handlers for service: guardduty
2023-10-06 12:49:35,734 botocore.loaders [DEBUG] Loading JSON file: /opt/homebrew/lib/python3.9/site-packages/botocore/data/guardduty/2017-11-28/paginators-1.json
mygooddetectorid
2023-10-06 12:49:35,734 botocore.hooks [DEBUG] Event before-parameter-build.guardduty.ListFindings: calling handler <function generate_idempotent_uuid at 0x1047ff0d0>
2023-10-06 12:49:35,734 botocore.regions [DEBUG] Calling endpoint provider with parameters: {'Region': 'us-west-2', 'UseDualStack': False, 'UseFIPS': False}
2023-10-06 12:49:35,734 botocore.regions [DEBUG] Endpoint provider result: https://guardduty.us-west-2.amazonaws.com
2023-10-06 12:49:35,734 botocore.hooks [DEBUG] Event before-call.guardduty.ListFindings: calling handler <function add_recursion_detection_header at 0x1047f9d30>
2023-10-06 12:49:35,734 botocore.hooks [DEBUG] Event before-call.guardduty.ListFindings: calling handler <function inject_api_version_header_if_needed at 0x104800940>
2023-10-06 12:49:35,734 botocore.endpoint [DEBUG] Making request for OperationModel(name=ListFindings) with params: {'url_path': '/detector/mygooddetectorid/findings', 
'query_string': {}, 'method': 'POST', 'headers': {'Content-Type': 'application/json', 
'User-Agent': 'Boto3/1.28.61 md/Botocore#1.31.61 ua/2.0 os/macos#23.0.0 md/arch#arm64 lang/python#3.9.18 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.31.61'}, 
'body': b'{"findingCriteria": {}, "maxResults": 100}', 'url': 'https://guardduty.us-west-2.amazonaws.com/detector/mygooddetectorid/findings', 'context': {'client_region': 'us-west-2', 'client_config': <botocore.config.Config object at 0x10516d520>, 'has_streaming_input': False, 'auth_type': None}}
2023-10-06 12:49:35,734 botocore.hooks [DEBUG] Event request-created.guardduty.ListFindings: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x10516d370>>
2023-10-06 12:49:35,734 botocore.hooks [DEBUG] Event choose-signer.guardduty.ListFindings: calling handler <function set_operation_specific_signer at 0x1047f9f70>
2023-10-06 12:49:35,734 botocore.auth [DEBUG] Calculating signature using v4 auth.
2023-10-06 12:49:35,734 botocore.auth [DEBUG] CanonicalRequest:
POST
/detector/mygooddetectorid/findings

content-type:application/json
host:guardduty.us-west-2.amazonaws.com

2023-10-06 12:49:35,735 botocore.auth [DEBUG] StringToSign:

2023-10-06 12:49:35,735 botocore.auth [DEBUG] Signature:
2023-10-06 12:49:35,735 botocore.hooks [DEBUG] Event request-created.guardduty.ListFindings: calling handler <function add_retry_headers at 0x104801040>
2023-10-06 12:49:35,735 botocore.endpoint [DEBUG] Sending http request: <AWSPreparedRequest stream_output=False, method=POST, url=https://guardduty.us-west-2.amazonaws.com/detector/mygooddetectorid/findings, 
eaders={'Content-Type': b'application/json', '
User-Agent': b'Boto3/1.28.61 md/Botocore#1.31.61 ua/2.0 os/macos#23.0.0 md/arch#arm64 lang/python#3.9.18 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.31.61', 
'X-Amz-Date': b'20231006T184935Z', 'Authorization': b'AWS4-HMAC-SHA256 Credential=d, SignedHeaders=content-type;host;x-amz-date;x-amz-security-token,Signature=ddddd', 'amz-sdk-invocation-id': b'cc6cc800-ac0a-4a8d-a857-df2b31bd00bf', 'amz-sdk-request': b'attempt=1', 'Content-Length': '42'}>
2023-10-06 12:49:35,735 botocore.httpsession [DEBUG] Certificate path: /Users/aaaaaaaaaaaaaa/Library/Python/3.9/lib/python/site-packages/certifi/cacert.pem
2023-10-06 12:49:35,735 urllib3.connectionpool [DEBUG] Starting new HTTPS connection (1): guardduty.us-west-2.amazonaws.com:443 
2023-10-06 12:49:35,937 urllib3.connectionpool [DEBUG] https://guardduty.us-west-2.amazonaws.com:443 "POST /detector/mygooddetectorid/findings HTTP/1.1" 400 125
2023-10-06 12:49:35,937 botocore.parsers [DEBUG] Response headers: {'Date': 'Fri, 06 Oct 2023 18:49:36 GMT', 'Content-Type': 'application/json', 'Content-Length': '125', 
'Connection': 'keep-alive', 'x-amzn-RequestId': 'f31cdd9e-5804-45e5-be72-f30c711585d7', 'Access-Control-Allow-Origin': '*', '
Access-Control-Allow-Headers': 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token,X-Amz-Content-Sha256,X-Amz-User-Agent,*,Date,X-Amz-Target,x-amzn-platform-id,x-amzn-trace-id', 
'x-amzn-ErrorType': 'BadRequestException', 'x-amz-apigw-id': 'MZJ-CFgJvHcEWbQ=', 'Access-Control-Expose-Headers': 'x-amzn-ErrorType,x-amzn-requestid,x-amzn-errormessage,x-amzn-trace-id,x-amz-apigw-id,Date', '
X-Amzn-Trace-Id': 'Root=1-652056bf-2dfdb72c437f691b4c090eda;Sampled=0;lineage=a490b0ff:0', 'Access-Control-Max-Age': '86400'}
2023-10-06 12:49:35,937 botocore.parsers [DEBUG] Response body:
b'{"message":"The request is rejected because the parameter detectorId has an invalid value.","__type":"InvalidInputException"}'
2023-10-06 12:49:35,940 botocore.parsers [DEBUG] Response headers: {'Date': 'Fri, 06 Oct 2023 18:49:36 GMT', 'Content-Type': 'application/json', 'Content-Length': '125', 
'Connection': 'keep-alive', 'x-amzn-RequestId': 'f31cdd9e-5804-45e5-be72-f30c711585d7', 'Access-Control-Allow-Origin': '*', 
'Access-Control-Allow-Headers': 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token,X-Amz-Content-Sha256,X-Amz-User-Agent,*,Date,X-Amz-Target,x-amzn-platform-id,x-amzn-trace-id',
 'x-amzn-ErrorType': 'BadRequestException', 'x-amz-apigw-id': 'MZJ-CFgJvHcEWbQ=', 'Access-Control-Expose-Headers': 'x-amzn-ErrorType,x-amzn-requestid,x-amzn-errormessage,x-amzn-trace-id,x-amz-apigw-id,Date', 
 'X-Amzn-Trace-Id': 'Root=1-652056bf-2dfdb72c437f691b4c090eda;Sampled=0;lineage=a490b0ff:0', 'Access-Control-Max-Age': '86400'}
2023-10-06 12:49:35,940 botocore.parsers [DEBUG] Response body:
b'{"message":"The request is rejected because the parameter detectorId has an invalid value.","__type":"InvalidInputException"}'
2023-10-06 12:49:35,941 botocore.hooks [DEBUG] Event needs-retry.guardduty.ListFindings: calling handler <botocore.retryhandler.RetryHandler object at 0x105199910>
2023-10-06 12:49:35,941 botocore.retryhandler [DEBUG] No retry needed.
[2023-10-06 18:49:35 - guardduty.py] :  error: Oh No! Something went wrong paginating aws!
Traceback (most recent call last):
  File "/Users/aaaaaaaaaaaaaa/_repos/_prototyping/aws-guard-duty-jira/./guardduty.py", line 308, in paginate_collect
    collect = self.get_paginate(session=session)
  File "/Users/aaaaaaaaaaaaaa/_repos/_prototyping/aws-guard-duty-jira/./guardduty.py", line 353, in get_paginate
    for each in page_iterator:  # .search():
  File "/opt/homebrew/lib/python3.9/site-packages/botocore/paginate.py", line 269, in __iter__
    response = self._make_request(current_kwargs)
  File "/opt/homebrew/lib/python3.9/site-packages/botocore/paginate.py", line 357, in _make_request
    return self._method(**current_kwargs)
  File "/opt/homebrew/lib/python3.9/site-packages/botocore/client.py", line 535, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/opt/homebrew/lib/python3.9/site-packages/botocore/client.py", line 980, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.BadRequestException: An error occurred (BadRequestException) when calling the ListFindings operation: The request is rejected because the parameter detectorId has an invalid value.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/aaaaaaaaaaaaaa/_repos/_prototyping/aws-guard-duty-jira/./guardduty.py", line 382, in <module>
    results = AWSdo(manifest=manifest).execute_paginate()
  File "/Users/aaaaaaaaaaaaaa/_repos/_prototyping/aws-guard-duty-jira/./guardduty.py", line 368, in execute_paginate
    results = self.paginate_collect()
  File "/Users/aaaaaaaaaaaaaa/_repos/_prototyping/aws-guard-duty-jira/./guardduty.py", line 331, in paginate_collect
    raise super_except(
__main__.super_except: super_except('paginate_collect')
------------------------------------------------------------
ORIGINAL EXCEPTION:
  File "/Users/aaaaaaaaaaaaaa/_repos/_prototyping/aws-guard-duty-jira/./guardduty.py", line 308, in paginate_collect
    collect = self.get_paginate(session=session)

  File "/Users/aaaaaaaaaaaaaa/_repos/_prototyping/aws-guard-duty-jira/./guardduty.py", line 353, in get_paginate
    for each in page_iterator:  # .search():

  File "/opt/homebrew/lib/python3.9/site-packages/botocore/paginate.py", line 269, in __iter__
    response = self._make_request(current_kwargs)

  File "/opt/homebrew/lib/python3.9/site-packages/botocore/paginate.py", line 357, in _make_request
    return self._method(**current_kwargs)

  File "/opt/homebrew/lib/python3.9/site-packages/botocore/client.py", line 535, in _api_call
    return self._make_api_call(operation_name, kwargs)

  File "/opt/homebrew/lib/python3.9/site-packages/botocore/client.py", line 980, in _make_api_call
    raise error_class(parsed_response, operation_name)

BadRequestException('An error occurred (BadRequestException) when calling the ListFindings operation: The request is rejected because the parameter detectorId has an invalid value.')

[RyanFitzSimmonsAK]

Thanks for providing those. Could you tell me how you're making the call after you create the page iterator? It'd also be helpful to know the format of your detectorId i.e length, any hyphens, etc.

Also, it looks like PageSize is required to be 50 or less. That's not part of the problem, just thought I'd mention that.

[gyoza]

Thanks for providing those. Could you tell me how you're making the call after you create the page iterator? It'd also be helpful to know the format of your detectorId i.e length, any hyphens, etc.

Also, it looks like PageSize is required to be 50 or less. That's not part of the problem, just thought I'd mention that.

Yea sure.. Thanks for the pagesize tip :D

but the detectorid is something like 11a111aaaa1aa11aaa11111a111aa1 with a class 'str'

Also after creating the iterator the next thing is a simple for loop through the page_iterator object, even if i add print(page_iterator) right after it doesn't even get that far.

Just anecdotally, this page iterator works describe_instances, etc.

[RyanFitzSimmonsAK]

Is your detectorId actually only 30 characters? If so, that would explain this.

[gyoza]

I might have missed one char

…

On Fri, Oct 6, 2023, 5:25 PM Ryan F. ***@***.***> wrote: Is it actually only 30 characters? If so, that would explain this. — Reply to this email directly, view it on GitHub <#3890 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEJWBT75TUX6XMXBKONXZTX6CHPDAVCNFSM6AAAAAA5WCOTZKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJRGQ4TMOJSGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[gyoza]

Ya its 31 Characters still having issues.

[RyanFitzSimmonsAK]

Those are supposed to be 32 characters. Try deleting your detector and making a new one, which should (hopefully) generate one that is 32 characters long.

[gyoza]

Hey thank you very much! It does appear I had a bad value.
I must have been really not paying attention. Apologize for the noise.
Sorry for the delayed response was traveling.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.