RuntimeError: There is no current event loop in thread 'MainThread'.

[qci-amos]

My previously passing test is now failing with pytest-asyncio==0.22. I get the error:

self = <Coroutine test_my_test[0]>

    def runtest(self) -> None:
        if self.get_closest_marker("asyncio"):
            self.obj = wrap_in_sync(
                # https://github.com/pytest-dev/pytest-asyncio/issues/596
                self.obj,  # type: ignore[has-type]
            )
>       super().runtest()

../../../miniconda3/envs/py311/lib/python3.11/site-packages/pytest_asyncio/plugin.py:426:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../miniconda3/envs/py311/lib/python3.11/site-packages/pytest_asyncio/plugin.py:804: in inner
    _loop = asyncio.get_event_loop()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <asyncio.unix_events._UnixDefaultEventLoopPolicy object at 0x7f5a92b814d0>

    def get_event_loop(self):
        """Get the event loop for the current context.

        Returns an instance of EventLoop or raises an exception.
        """
        if (self._local._loop is None and
                not self._local._set_called and
                threading.current_thread() is threading.main_thread()):
            self.set_event_loop(self.new_event_loop())

        if self._local._loop is None:
>           raise RuntimeError('There is no current event loop in thread %r.'
                               % threading.current_thread().name)
E           RuntimeError: There is no current event loop in thread 'MainThread'.

../../../miniconda3/envs/py311/lib/python3.11/asyncio/events.py:677: RuntimeError

I assume it's a coincidence, but just yesterday I switched from

pytestmark = pytest.mark.asyncio

to:

[tool.pytest.ini_options]
asyncio_mode = "auto"

[2e0byo]

Do you have a custom event loop fixture? I had (apparently) random failures of this kind in our test suite with this upgrade, only where a custom fixture was being used. I didn't investigate further. (We have always used asyncio_mode = "auto" btw)

[Pliner]

We have the same issue as @qci-amos reported (out setup.cfg contains [tool.pytest.ini_options] asyncio_mode = "auto").

[seifertm]

@qci-amos Can you provide the sources of test_my_test?

If you provide a short code example the reproduces the error, I'm happy to look into it.

As a temporary workaround, you can pin your version of pytest-asyncio to pytest-asyncio!=0.22.0

[lindycoder]

Thank you for fixing by removing 0.22.0.

I would like to share something regarding this warning:

PytestDeprecationWarning: xxxxxxxx is asynchronous and explicitly requests the "event_loop" fixture. Asynchronous fixtures and test functions should use "asyncio.get_running_loop()" instead.

We have session scoped async fixtures so we have this central fixture

@pytest.fixture(scope="session")
def event_loop() -> Generator[AbstractEventLoop, None, None]:
    """Make the loop session scope to use session async fixtures."""
    policy = asyncio.get_event_loop_policy()
    loop = policy.new_event_loop()
    yield loop
    loop.close()

And every single AsyncGenerator fixtures we have actually require event_loop to make sure the teardown is done before the loop is closed.

Sharing this use-case hoping to help further development.

[2e0byo]

@lindycoder larger scopes are being discussed over here #657

[gabrielmbmb]

Hey guys, just wanted to mention that pytest-asyncio==0.22.0 hasn't been yanked in conda-forge https://anaconda.org/conda-forge/pytest-asyncio

[seifertm]

@lindycoder Thanks for sharing your use case. These things are highly valuable for the pytest-asyncio devs.

@gabrielmbmb I created an account on anaconda and looked for way to claim or flag the package, but didn't find any. Do you have any information what the procedure is for yanking packages on conda-forge? It seems there is none.

[seifertm]

@gabrielmbmb I filed a PR to mark pytest-asyncio v0.22.0 as broken on conda-forge:
conda-forge/admin-requests#857

[gabrielmbmb]

Thanks @seifertm!

[qci-amos]

Unfortunately, I've already forgotten exactly which test this was! However, since this test used a @pytest.fixture(scope="session"), it seems almost certain now that this issue is effectively a duplicate of #657

[seifertm]

Since this seems to be related to session-scoped loops in v0.22.0, I'll close the issue as a duplicate as you suggested.

If you're interested to test out the upcoming release, pytest-asyncio v0.23.0a0 is available on PyPI and adds an optional scope keyword argument to the asyncio mark to control the scope used for each test. I'd appreciate your feedback on the pre-release version.

[qci-amos]

It's not letting me reopen this @seifertm , but I find my error was indeed separate.

In conftest.py:

async def my_async_method():
    print("here!")


@pytest.fixture
def nested_async():
    return asyncio.run(my_async_method())

In test_repro.py:

async def test_repro(nested_async):
    print("in test")

gives me:

======================================================================== FAILURES =========================================================================
_______________________________________________________________________ test_repro ________________________________________________________________________
self = <Coroutine test_repro>

    def runtest(self) -> None:
        if self.get_closest_marker("asyncio"):
            self.obj = wrap_in_sync(
                # https://github.com/pytest-dev/pytest-asyncio/issues/596
                self.obj,  # type: ignore[has-type]
            )
>       super().runtest()

../../../miniconda3/envs/py311/lib/python3.11/site-packages/pytest_asyncio/plugin.py:426:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../miniconda3/envs/py311/lib/python3.11/site-packages/pytest_asyncio/plugin.py:847: in inner
    _loop = asyncio.get_event_loop()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <asyncio.unix_events._UnixDefaultEventLoopPolicy object at 0x7f3b3a640810>

    def get_event_loop(self):
        """Get the event loop for the current context.

        Returns an instance of EventLoop or raises an exception.
        """
        if (self._local._loop is None and
                not self._local._set_called and
                threading.current_thread() is threading.main_thread()):
            self.set_event_loop(self.new_event_loop())

        if self._local._loop is None:
>           raise RuntimeError('There is no current event loop in thread %r.'
                               % threading.current_thread().name)
E           RuntimeError: There is no current event loop in thread 'MainThread'.

../../../miniconda3/envs/py311/lib/python3.11/asyncio/events.py:677: RuntimeError
------------------------------------------------------------------ Captured stdout setup ------------------------------------------------------------------
here!

[seifertm]

It looks like #675 fixes this issue. I'll leave the PR open for a day or so and create a new alpha release.

[seifertm]

@qci-amos pytest-asyncio 0.23.0a1 should fix this issue.

[qci-amos]

Thanks, this version fixed the test I created the repro for, but the other test (which looks the same in this regard to my eye) is still failing. I'll see if I can make another repro today.

[qci-amos]

Ok, here's a new repro:

pip install pytest-asyncio==0.23.0a1

import pytest
import asyncio


async def my_async_method():
    print("here!")


@pytest.fixture
def nested_async():
    return asyncio.run(my_async_method())


@pytest.mark.parametrize("a1", [True, False])
@pytest.mark.parametrize("a2", [True, False])
async def test_repro(a1, a2, nested_async):
    print("in test", a1, a2)

======================================================================== short test summary info =========================================================================
FAILED test_repro.py::test_repro[True-False] - RuntimeError: There is no current event loop in thread 'MainThread'.
FAILED test_repro.py::test_repro[False-True] - RuntimeError: There is no current event loop in thread 'MainThread'.
FAILED test_repro.py::test_repro[False-False] - RuntimeError: There is no current event loop in thread 'MainThread'.
================================================================ 3 failed, 1 passed, 23 warnings in 0.12s ================================================================

[lindycoder]

@qci-amos sorry if this goes off topic.... but i'm curious, why use a sync method to call async code when your fixture can be async?

@pytest.fixture()
async def nested_async():
    return await my_async_method()

[2e0byo]

@lindycoder I do have some tests which do this with one more level of isolation (where a test calls a sync function, but under the hood that sync function ends up running async code). In my particular case the reason is highly involved and boils down to using an async postgresql context inside alembic, which is sync. (The involved thing is why we decided to this in the first place...). Another example would be testing something like asyncio.run() (e.g. testing an async repl).

Of course if the fixture / test is under one's control one can simply make it async, but if the sync -> async boundary is under the hood it's harder.

I don't know why @qci-amos needs it though, but it's definitely a valid (if weird) use-case.

[qci-amos]

This repo was my first exposure to asyncio so things I did were not necessarily because I had good reason to! It's been a while, but probably in this case either I was unaware that a fixture could be async or I didn't see much difference and just went with the first implementation that occurred to me. Btw, if it's bad practice to do this, then I'd like to hear an explanation!

I do remember that at the time I was frustrated with Python that I couldn't use nested loops (a challenge with jupyter in particular). I ended up compromising and making both an async api for my library and a sync one with nest_asyncio.

[2e0byo]

@qci-amos calling asyncio.run anywhere except the top-level entrypoint is generally frowned upon. The docs say

This function should be used as a main entry point for asyncio programs, and should ideally only be called once.

Calling run cleans up and stops the executor, which is probably why. Thus it's not valid to run any async code afterwards (without provisioning a new loop). This is doubtless a surprise to pytest-asyncio, since the executor is already stopped when it goes to clean up.

FWIW I regard asyncio.run() as 'surprising' code: personally I'd definitely use the async fixture here to avoid the side effect. But there are times when you can't.

(The postgresql case is more fun although I haven't looked into it: sync_engine returns something something greenlets something something.)

[qci-amos]

Ok, good point. I am familiar with that doc and that makes sense. That's a good enough reason to me to update my test!

I'll just reiterate however that once I started using nest_asyncio I entered a mindset of "something seems wrong with Python" and that doc felt "not applicable".

[seifertm]

pytest assumes that tests are synchronous functions and gives a warning when you try to write async def tests.
pytest-asyncio wraps async tests to in synchronous functions to satisfy pytest. In v.0.21.0, this synchronization wrapper had an explicit reference to the event loop in which the test should run. This reference is no longer present in v0.23.0a1 (see 36b2269) and the synchronization wrappers run in the loop returned by asyncio.get_event_loop().

I can see that the order in which fixtures are evaluated has changed from v0.21.0 to v0.23.0a1.

v0.21.0:

$ pytest --asyncio-mode=auto --setup-show
===== test session starts =====
platform linux -- Python 3.11.6, pytest-7.4.3, pluggy-1.3.0
rootdir: /tmp/tst
plugins: asyncio-0.21.0
asyncio: mode=Mode.AUTO
collected 4 items                                                                                                                                                                                                                                                            

test_a.py 
        SETUP    F event_loop
        SETUP    F a1[True]
        SETUP    F a2[True]
        SETUP    F nested_async
        test_a.py::test_repro[True-True] (fixtures used: a1, a2, event_loop, nested_async).
        TEARDOWN F nested_async
        TEARDOWN F a2[True]
        TEARDOWN F a1[True]
        TEARDOWN F event_loop
        SETUP    F event_loop
        SETUP    F a1[False]
        SETUP    F a2[True]
        SETUP    F nested_async
        test_a.py::test_repro[True-False] (fixtures used: a1, a2, event_loop, nested_async).
        TEARDOWN F nested_async
        TEARDOWN F a2[True]
        TEARDOWN F a1[False]
        TEARDOWN F event_loop
        SETUP    F event_loop
        SETUP    F a1[True]
        SETUP    F a2[False]
        SETUP    F nested_async
        test_a.py::test_repro[False-True] (fixtures used: a1, a2, event_loop, nested_async).
        TEARDOWN F nested_async
        TEARDOWN F a2[False]
        TEARDOWN F a1[True]
        TEARDOWN F event_loop
        SETUP    F event_loop
        SETUP    F a1[False]
        SETUP    F a2[False]
        SETUP    F nested_async
        test_a.py::test_repro[False-False] (fixtures used: a1, a2, event_loop, nested_async).
        TEARDOWN F nested_async
        TEARDOWN F a2[False]
        TEARDOWN F a1[False]
        TEARDOWN F event_loop

===== 4 passed in 0.01s ====

$ pytest --asyncio-mode=auto --setup-show
===== test session starts =====
platform linux -- Python 3.11.6, pytest-7.4.3, pluggy-1.3.0
rootdir: /tmp/tst
plugins: asyncio-0.23.0a1
asyncio: mode=Mode.AUTO
collected 4 items                                                                                                                                                                                                                                                            

test_a.py 
SETUP    S event_loop_policy
        SETUP    F a1[True]
        SETUP    F a2[True]
        SETUP    F nested_async
        SETUP    F event_loop
        test_a.py::test_repro[True-True] (fixtures used: a1, a2, event_loop, event_loop_policy, nested_async).
        TEARDOWN F event_loop
        TEARDOWN F nested_async
        TEARDOWN F a2[True]
        TEARDOWN F a1[True]
        SETUP    F event_loop
        SETUP    F a1[False]
        SETUP    F a2[True]
        SETUP    F nested_async
        test_a.py::test_repro[True-False] (fixtures used: a1, a2, event_loop, event_loop_policy, nested_async)F
        TEARDOWN F nested_async
        TEARDOWN F a2[True]
        TEARDOWN F a1[False]
        TEARDOWN F event_loop
        SETUP    F event_loop
        SETUP    F a1[True]
        SETUP    F a2[False]
        SETUP    F nested_async
        test_a.py::test_repro[False-True] (fixtures used: a1, a2, event_loop, event_loop_policy, nested_async)F
        TEARDOWN F nested_async
        TEARDOWN F a2[False]
        TEARDOWN F a1[True]
        TEARDOWN F event_loop
        SETUP    F event_loop
        SETUP    F a1[False]
        SETUP    F a2[False]
        SETUP    F nested_async
        test_a.py::test_repro[False-False] (fixtures used: a1, a2, event_loop, event_loop_policy, nested_async)F
        TEARDOWN F nested_async
        TEARDOWN F a2[False]
        TEARDOWN F a1[False]
        TEARDOWN F event_loop
TEARDOWN S event_loop_policy

===== FAILURES ====
_____test_repro[True-False] _____

self = <Coroutine test_repro[True-False]>

    def runtest(self) -> None:
        if self.get_closest_marker("asyncio"):
            self.obj = wrap_in_sync(
                # https://github.com/pytest-dev/pytest-asyncio/issues/596
                self.obj,  # type: ignore[has-type]
            )
>       super().runtest()

venv/lib/python3.11/site-packages/pytest_asyncio/plugin.py:427: 
_ _ _
venv/lib/python3.11/site-packages/pytest_asyncio/plugin.py:856: in inner
    _loop = asyncio.get_event_loop()
_ _ _

self = <asyncio.unix_events._UnixDefaultEventLoopPolicy object at 0x7fb546759850>

    def get_event_loop(self):
        """Get the event loop for the current context.
    
        Returns an instance of EventLoop or raises an exception.
        """
        if (self._local._loop is None and
                not self._local._set_called and
                threading.current_thread() is threading.main_thread()):
            self.set_event_loop(self.new_event_loop())
    
        if self._local._loop is None:
>           raise RuntimeError('There is no current event loop in thread %r.'
                               % threading.current_thread().name)
E           RuntimeError: There is no current event loop in thread 'MainThread'.

/usr/lib/python3.11/asyncio/events.py:677: RuntimeError
----- Captured stdout setup -----
here!
_____test_repro[False-True] _____

self = <Coroutine test_repro[False-True]>

    def runtest(self) -> None:
        if self.get_closest_marker("asyncio"):
            self.obj = wrap_in_sync(
                # https://github.com/pytest-dev/pytest-asyncio/issues/596
                self.obj,  # type: ignore[has-type]
            )
>       super().runtest()

venv/lib/python3.11/site-packages/pytest_asyncio/plugin.py:427: 
_ _ _
venv/lib/python3.11/site-packages/pytest_asyncio/plugin.py:856: in inner
    _loop = asyncio.get_event_loop()
_ _ _

self = <asyncio.unix_events._UnixDefaultEventLoopPolicy object at 0x7fb546759850>

    def get_event_loop(self):
        """Get the event loop for the current context.
    
        Returns an instance of EventLoop or raises an exception.
        """
        if (self._local._loop is None and
                not self._local._set_called and
                threading.current_thread() is threading.main_thread()):
            self.set_event_loop(self.new_event_loop())
    
        if self._local._loop is None:
>           raise RuntimeError('There is no current event loop in thread %r.'
                               % threading.current_thread().name)
E           RuntimeError: There is no current event loop in thread 'MainThread'.

/usr/lib/python3.11/asyncio/events.py:677: RuntimeError
----- Captured stdout setup -----
here!
_____test_repro[False-False] _____

self = <Coroutine test_repro[False-False]>

    def runtest(self) -> None:
        if self.get_closest_marker("asyncio"):
            self.obj = wrap_in_sync(
                # https://github.com/pytest-dev/pytest-asyncio/issues/596
                self.obj,  # type: ignore[has-type]
            )
>       super().runtest()

venv/lib/python3.11/site-packages/pytest_asyncio/plugin.py:427: 
_ _ _
venv/lib/python3.11/site-packages/pytest_asyncio/plugin.py:856: in inner
    _loop = asyncio.get_event_loop()
_ _ _

self = <asyncio.unix_events._UnixDefaultEventLoopPolicy object at 0x7fb546759850>

    def get_event_loop(self):
        """Get the event loop for the current context.
    
        Returns an instance of EventLoop or raises an exception.
        """
        if (self._local._loop is None and
                not self._local._set_called and
                threading.current_thread() is threading.main_thread()):
            self.set_event_loop(self.new_event_loop())
    
        if self._local._loop is None:
>           raise RuntimeError('There is no current event loop in thread %r.'
                               % threading.current_thread().name)
E           RuntimeError: There is no current event loop in thread 'MainThread'.

/usr/lib/python3.11/asyncio/events.py:677: RuntimeError
----- Captured stdout setup -----
here!
===== short test summary info =====
FAILED test_a.py::test_repro[True-False] - RuntimeError: There is no current event loop in thread 'MainThread'.
FAILED test_a.py::test_repro[False-True] - RuntimeError: There is no current event loop in thread 'MainThread'.
FAILED test_a.py::test_repro[False-False] - RuntimeError: There is no current event loop in thread 'MainThread'.
===== 3 failed, 1 passed in 0.08s =====```

The logs show that tests fail when the nested_async fixture is setup after event_loop. Note that nested_async calls asyncio.run(). As 2e0byo already mentioned, asyncio.run sets the current event loop to None when finishing. This causes the synchronization wrappers to fail, because the current event loop is None.

I don't know why the fixtures are evaluated in different order for different parametrizations, but it don't think it's relevant for the case.

Although I agree that it's preferable to use

@pytest_asyncio.fixture
async def nested_async():
    return await my_async_method()

over asyncio.run(), the test fails for a non-obvious reason and the error message doesn't really help.