3 tests fail with Python 3.12

[befeleme]

Describe the bug

When testing botocore 1.29.164 against Python 3.12b3, there are 3 test failures

Expected Behavior

Tests pass

Current Behavior

=================================== FAILURES ===================================
___________ TestAutoGeneratedClient.test_paginator_help_from_client ____________

self = <tests.unit.test_client.TestAutoGeneratedClient testMethod=test_paginator_help_from_client>

    def test_paginator_help_from_client(self):
        pagination_config = {
            'pagination': {
                'TestOperation': {
                    "input_token": "Marker",
                    "output_token": "Marker",
                    "more_results": "IsTruncated",
                    "limit_key": "MaxItems",
                    "result_key": "Users",
                }
            }
        }
        self.loader.load_service_model.side_effect = [
            self.service_description,
            self.endpoint_ruleset,
            pagination_config,
        ]
        creator = self.create_client_creator()
        service_client = creator.create_client('myservice', 'us-west-2')
        paginator = service_client.get_paginator('test_operation')
        with mock.patch('sys.stdout', io.StringIO()) as mock_stdout:
            help(paginator.paginate)
        contents = mock_stdout.getvalue()
        lines = [
            (
                '    Creates an iterator that will paginate through responses '
                'from :py:meth:`MyService.Client.test_operation`.'
            ),
            '    **Request Syntax**',
            '    ::',
            '      response_iterator = paginator.paginate(',
            "          Foo='string',",
            "          Bar='string',",
            '          PaginationConfig={',
            "              'MaxItems': 123,",
            "              'PageSize': 123,",
            "              'StartingToken': 'string'",
            '          }',
            '      )',
            '    :type Foo: string',
            '    :param Foo: **[REQUIRED]** Documents Foo',
            '    :type Bar: string',
            '    :param Bar: Documents Bar',
            '    :type PaginationConfig: dict',
            '    :param PaginationConfig: ',
            (
                '      A dictionary that provides parameters to control '
                'pagination.'
            ),
            '      - **MaxItems** *(integer) --*',
            (
                '        The total number of items to return. If the total '
                'number of items available is more than the value specified '
                'in max-items then a ``NextToken`` will be provided in the '
                'output that you can use to resume pagination.'
            ),
            '      - **PageSize** *(integer) --*',
            '        The size of each page.',
            '      - **StartingToken** *(string) --*',
            (
                '        A token to specify where to start paginating. This is '
                'the ``NextToken`` from a previous response.'
            ),
            '    :returns: None',
        ]
        for line in lines:
>           self.assertIn(line, contents)
E           AssertionError: '    :param PaginationConfig: ' not found in "Help on method paginate in module botocore.client:\n\npaginate(**kwargs) method of botocore.client.MyService.Paginator.TestOperation instance\n    Creates an iterator that will paginate through responses from :py:meth:`MyService.Client.test_operation`.\n\n    **Request Syntax**\n    ::\n\n      response_iterator = paginator.paginate(\n          Foo='string',\n          Bar='string',\n          PaginationConfig={\n              'MaxItems': 123,\n              'PageSize': 123,\n              'StartingToken': 'string'\n          }\n      )\n\n    :type Foo: string\n    :param Foo: **[REQUIRED]** Documents Foo\n\n\n    :type Bar: string\n    :param Bar: Documents Bar\n\n\n    :type PaginationConfig: dict\n    :param PaginationConfig:\n\n      A dictionary that provides parameters to control pagination.\n\n\n\n\n      - **MaxItems** *(integer) --*\n\n        The total number of items to return. If the total number of items available is more than the value specified in max-items then a ``NextToken`` will be provided in the output that you can use to resume pagination.\n\n\n\n\n      - **PageSize** *(integer) --*\n\n        The size of each page.\n\n\n\n\n      - **StartingToken** *(string) --*\n\n        A token to specify where to start paginating. This is the ``NextToken`` from a previous response.\n\n\n\n\n\n\n    :returns: None\n\n"

tests/unit/test_client.py:1254: AssertionError
_______________ TestClientErrors.test_paginator_help_from_client _______________

self = <tests.unit.test_client.TestClientErrors testMethod=test_paginator_help_from_client>

    def test_paginator_help_from_client(self):
        pagination_config = {
            'pagination': {
                'TestOperation': {
                    "input_token": "Marker",
                    "output_token": "Marker",
                    "more_results": "IsTruncated",
                    "limit_key": "MaxItems",
                    "result_key": "Users",
                }
            }
        }
        self.loader.load_service_model.side_effect = [
            self.service_description,
            self.endpoint_ruleset,
            pagination_config,
        ]
        creator = self.create_client_creator()
        service_client = creator.create_client('myservice', 'us-west-2')
        paginator = service_client.get_paginator('test_operation')
        with mock.patch('sys.stdout', io.StringIO()) as mock_stdout:
            help(paginator.paginate)
        contents = mock_stdout.getvalue()
        lines = [
            (
                '    Creates an iterator that will paginate through responses '
                'from :py:meth:`MyService.Client.test_operation`.'
            ),
            '    **Request Syntax**',
            '    ::',
            '      response_iterator = paginator.paginate(',
            "          Foo='string',",
            "          Bar='string',",
            '          PaginationConfig={',
            "              'MaxItems': 123,",
            "              'PageSize': 123,",
            "              'StartingToken': 'string'",
            '          }',
            '      )',
            '    :type Foo: string',
            '    :param Foo: **[REQUIRED]** Documents Foo',
            '    :type Bar: string',
            '    :param Bar: Documents Bar',
            '    :type PaginationConfig: dict',
            '    :param PaginationConfig: ',
            (
                '      A dictionary that provides parameters to control '
                'pagination.'
            ),
            '      - **MaxItems** *(integer) --*',
            (
                '        The total number of items to return. If the total '
                'number of items available is more than the value specified '
                'in max-items then a ``NextToken`` will be provided in the '
                'output that you can use to resume pagination.'
            ),
            '      - **PageSize** *(integer) --*',
            '        The size of each page.',
            '      - **StartingToken** *(string) --*',
            (
                '        A token to specify where to start paginating. This is '
                'the ``NextToken`` from a previous response.'
            ),
            '    :returns: None',
        ]
        for line in lines:
>           self.assertIn(line, contents)
E           AssertionError: '    :param PaginationConfig: ' not found in "Help on method paginate in module botocore.client:\n\npaginate(**kwargs) method of botocore.client.MyService.Paginator.TestOperation instance\n    Creates an iterator that will paginate through responses from :py:meth:`MyService.Client.test_operation`.\n\n    **Request Syntax**\n    ::\n\n      response_iterator = paginator.paginate(\n          Foo='string',\n          Bar='string',\n          PaginationConfig={\n              'MaxItems': 123,\n              'PageSize': 123,\n              'StartingToken': 'string'\n          }\n      )\n\n    :type Foo: string\n    :param Foo: **[REQUIRED]** Documents Foo\n\n\n    :type Bar: string\n    :param Bar: Documents Bar\n\n\n    :type PaginationConfig: dict\n    :param PaginationConfig:\n\n      A dictionary that provides parameters to control pagination.\n\n\n\n\n      - **MaxItems** *(integer) --*\n\n        The total number of items to return. If the total number of items available is more than the value specified in max-items then a ``NextToken`` will be provided in the output that you can use to resume pagination.\n\n\n\n\n      - **PageSize** *(integer) --*\n\n        The size of each page.\n\n\n\n\n      - **StartingToken** *(string) --*\n\n        A token to specify where to start paginating. This is the ``NextToken`` from a previous response.\n\n\n\n\n\n\n    :returns: None\n\n"

tests/unit/test_client.py:1254: AssertionError
_ TestSwitchToChunkedEncodingForNonSeekableObjects.test_switch_to_chunked_encodeing_for_stream_like_object _

self = <tests.unit.test_utils.TestSwitchToChunkedEncodingForNonSeekableObjects testMethod=test_switch_to_chunked_encodeing_for_stream_like_object>

    def test_switch_to_chunked_encodeing_for_stream_like_object(self):
        request = AWSRequest(
            method='POST',
            headers={},
>           data=io.BufferedIOBase(b"some initial binary data"),
            url='https://foo.amazonaws.com/bucket/key.txt',
        )
E       TypeError: BufferedIOBase() takes no arguments

tests/unit/test_utils.py:1127: TypeError

=========================== short test summary info ============================
FAILED tests/unit/test_client.py::TestAutoGeneratedClient::test_paginator_help_from_client
FAILED tests/unit/test_client.py::TestClientErrors::test_paginator_help_from_client
FAILED tests/unit/test_utils.py::TestSwitchToChunkedEncodingForNonSeekableObjects::test_switch_to_chunked_encodeing_for_stream_like_object

Reproduction Steps

This occurred in Fedora Linux RPM builds. I'd be happy to retest it for you.
I tried to add py312 to tox and test, but couldn't force it to work.

Possible Solution

There is a difference in behavior of io.BufferedIOBase between Python 3.11 and Python 3.12

$ python3.11              
Python 3.11.3 (main, May 24 2023, 00:00:00) [GCC 13.1.1 20230511 (Red Hat 13.1.1-2)] on linux
>>> import io
>>> io.BufferedIOBase(b"some")
<io.BufferedIOBase object at 0x7fa56a35b700> 

$  python3.12
Python 3.12.0b3 (main, Jun 21 2023, 00:00:00) [GCC 13.1.1 20230614 (Red Hat 13.1.1-4)] on linux
>>> import io
>>> io.BufferedIOBase(b"some")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: BufferedIOBase() takes no arguments

In both cases it's possible to initialize io.BufferedIOBase with no arguments, so possibly it could be dropped.

In case of :param PaginationConfig: , the space is for some reason truncated. Changing the expected output to not check for the whitespace can help.

Additional Information/Context

No response

SDK version used

botocore 1.29.164

Environment details (OS name and version, etc.)

Fedora Linux 39, Python 3.12b3

[tim-finnigan]

Hi @befeleme thanks for reporting this issue. Currently support goes up to Python 3.11 but the team has started testing 3.12 pre-releases (see: #2967). We can leave this issue open for tracking and share any related updates here.

[nateprewitt]

This was resolved in #2967. Closing it out now that 3.12 is launched.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.