Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Admin: Improve moto usage, simplifying and speeding up tests #793

Merged
merged 1 commit into from
Sep 21, 2023

Conversation

bblommers
Copy link
Contributor

Hi there! Moto maintainer here. I had a look at the tests for s3fs, and realized that there is a cleaner way to use the Moto library for your usecase.

The ThreadedMotoServer starts an instance of MotoServer in the background, achieving the same result as calling moto_server s3 in a subprocess. So the behaviour is the same, it just hides the nasty details like calling subprocess.Popen, and having to manually call requests.get to verify whether the server is ready.

The second change is to turn the s3_base fixture into a module-scoped fixture. The ThreadedMotoServer will now only be created once per module (file), meaning that the tests are executed much faster (~3mins vs ~5 mins).

Because we do expect a clean slate in between tests, I added reset_s3_fixture - calling the reset-API in Moto deletes all existing state before every test.

@martindurant
Copy link
Member

That's great, thanks for getting in touch and contributing, and general thanks for moto's existence :)

@martindurant martindurant merged commit 315030e into fsspec:main Sep 21, 2023
5 checks passed
@bblommers bblommers deleted the admin/improve-test-times branch September 21, 2023 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants