Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for python 3.13 #847

Merged
merged 2 commits into from
Dec 17, 2024
Merged

Add support for python 3.13 #847

merged 2 commits into from
Dec 17, 2024

Conversation

ddelange
Copy link
Contributor

@ddelange ddelange commented Dec 15, 2024

Title

Motivation

Reproduce and fix #846 introduced in python 3.13 by python/cpython#105104

Tests

Added a test for azure reader exception during __init__

Work in progress

Checklist

Before you create the PR, please make sure you have:

  • Picked a concise, informative and complete title
  • Clearly explained the motivation behind the PR
  • Linked to any existing issues that your PR will be solving
  • Included tests for any new functionality
  • Checked that all unit tests pass

Workflow

Please avoid rebasing and force-pushing to the branch of the PR once a review is in progress.
Rebasing can make your commits look a bit cleaner, but it also makes life more difficult from the reviewer, because they are no longer able to distinguish between code that has already been reviewed, and unreviewed code.

@ddelange ddelange changed the title Add cp313 to CI Add support for python 3.13 Dec 15, 2024
@ddelange ddelange marked this pull request as ready for review December 15, 2024 15:06
@ddelange
Copy link
Contributor Author

hi @mpenkov 👋 could you cut a hotfix release for the debian team?

@ddelange
Copy link
Contributor Author

ddelange commented Dec 15, 2024

these only occur under python 3.13, and then pytest is calling repr in order to generate these tracebacks (the original error report #846):

smart_open/tests/test_azure.py::WriterTest::test_nonexisting_container
  /opt/hostedtoolcache/Python/3.13.1/x64/lib/python3.13/site-packages/_pytest/unraisableexception.py:85: PytestUnraisableExceptionWarning: Exception ignored in: Writer(container='thiscontainerdoesntexist', blob='mykey', min_part_size=67108864)
  
  azure.core.exceptions.ResourceNotFoundError: The specified container does not exist.
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "/home/runner/work/smart_open/smart_open/smart_open/azure.py", line 437, in close
      self._blob.commit_block_list(self._block_list, **self._blob_kwargs)
      ^^^^^^^^^^
  AttributeError: 'Writer' object has no attribute '_blob'


smart_open/tests/test_hdfs.py::test_context_manager[schema1]
  /opt/hostedtoolcache/Python/3.13.1/x64/lib/python3.13/site-packages/_pytest/unraisableexception.py:85: PytestUnraisableExceptionWarning: Exception ignored in: <smart_open.hdfs.CliRawInputBase object at 0x7f71101dce20>
  
  Traceback (most recent call last):
    File "/home/runner/work/smart_open/smart_open/smart_open/hdfs.py", line 87, in close
      self._sub.terminate()
      ^^^^^^^^^^^^^^^^^^^
  AttributeError: 'NoneType' object has no attribute 'terminate'
  


smart_open/tests/test_hdfs.py::test_write[schema0]
  /opt/hostedtoolcache/Python/3.13.1/x64/lib/python3.13/site-packages/_pytest/unraisableexception.py:85: PytestUnraisableExceptionWarning: Exception ignored in: <smart_open.hdfs.CliRawOutputBase object at 0x7f710d781960>
  
  Traceback (most recent call last):
    File "/home/runner/work/smart_open/smart_open/smart_open/hdfs.py", line 139, in close
      self.flush()
      ~~~~~~~~~~^^
    File "/home/runner/work/smart_open/smart_open/smart_open/hdfs.py", line 144, in flush
      self._sub.stdin.flush()
      ~~~~~~~~~~~~~~~~~~~~~^^
  ValueError: flush of closed file

cpython 3.13 _pyio.py started hard failing on calling __del__ when an exception occurs during __init__.

this is introduced by this cpython PR: python/cpython#105104

https://github.com/python/cpython/blob/v3.12.8/Lib/_pyio.py

    def __del__(self):
        """Destructor.  Calls close()."""
        try:
            closed = self.closed
        except AttributeError:
            # If getting closed fails, then the object is probably
            # in an unusable state, so ignore.
            return

        if closed:
            return

        if _IOBASE_EMITS_UNRAISABLE:
            self.close()
        else:
            # The try/except block is in case this is called at program
            # exit time, when it's possible that globals have already been
            # deleted, and then the close() call might fail.  Since
            # there's nothing we can do about such failures and they annoy
            # the end users, we suppress the traceback.
            try:
                self.close()
            except:
                pass

https://github.com/python/cpython/blob/v3.13.0/Lib/_pyio.py

    def __del__(self):
        """Destructor.  Calls close()."""
        try:
            closed = self.closed
        except AttributeError:
            # If getting closed fails, then the object is probably
            # in an unusable state, so ignore.
            return

        if closed:
            return

        # If close() fails, the caller logs the exception with
        # sys.unraisablehook. close() must be called at the end at __del__().
        self.close()

@ddelange ddelange force-pushed the cp313 branch 2 times, most recently from 2e9e317 to 2f87c1a Compare December 15, 2024 22:37
@ddelange ddelange force-pushed the cp313 branch 6 times, most recently from e33e216 to 914288e Compare December 16, 2024 19:11
@mpenkov mpenkov merged commit 72de673 into piskvorky:develop Dec 17, 2024
31 checks passed
@mpenkov
Copy link
Collaborator

mpenkov commented Dec 17, 2024

Looks good to me. Thank you @ddelange !

@mpenkov
Copy link
Collaborator

mpenkov commented Dec 17, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Test failure with Python 3.13
2 participants