Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP: Improvements for networkless testing (#2128 revamped) #2174

Draft
wants to merge 7 commits into
base: dev
Choose a base branch
from
Draft

WIP: Improvements for networkless testing (#2128 revamped) #2174

wants to merge 7 commits into from

Commits on Nov 17, 2024

  1. Use vcrpy in more tests 

    Cache network responses to focus on testing our consumption of the API.
This makes the tests more robust to API rate-limiting and denoises them
from network problems.

- tests/console/test_entry_point.py::test_download_song
- tests/console/test_entry_point.py::test_preload_song
- tests/test_init.py::test_download
- tests/test_init.py::test_get_urls
- tests/test_matching.py::test_ytmusic_matching
- tests/utils/test_ffmpeg.py::test_convert
- tests/utils/test_m3u.py::test_create_m3u_content
- tests/utils/test_m3u.py::test_create_m3u_file
- tests/utils/test_metadata.py::test_embed_metadata
- tests/utils/test_search.py::test_parse_query
    @hseg
    hseg authored and gesh [email protected] committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    699f27d View commit details
    Browse the repository at this point in the history

  2. Add pytest-freezegun 

    Having trouble getting yt-dlp invocations to work nicely with vcrpy,
suspect this is due to it receiving responses that are time-limited

Affected tests:
- tests/console/test_entry_point.py::test_download_song
- tests/console/test_entry_point.py::test_preload_song
- tests/test_init.py::test_get_urls
- tests/test_init.py::test_download
- tests/utils/test_ffmpeg.py::test_convert
- tests/utils/test_metadata.py::test_embed_metadata
    @hseg
    hseg authored and gesh [email protected] committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    6ebb628 View commit details
    Browse the repository at this point in the history

  3. test_matching: Update expected results

    @hseg
    hseg authored and gesh [email protected] committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    c1863ba View commit details
    Browse the repository at this point in the history

  4. Manually download urls passed to ffmpeg 

    This enables them to be picked up by vcrpy
    @hseg
    hseg authored and gesh [email protected] committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    008af93 View commit details
    Browse the repository at this point in the history

  5. Mark unvcrable tests that still need network 

    This is necessary to allow testing with --block-network -- just pass -m
'not novcr' as well to disable these tests.

Tests affected:

- tests/utils/test_ffmpeg.py::test_download_ffmpeg

  However, I doubt whether this function is even necessary -- shouldn't
  we be providing binaries with ffmpeg bundled instead for those users
  who can't install it conveniently on their machines?
    @hseg
    hseg authored and gesh [email protected] committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    8c8abf8 View commit details
    Browse the repository at this point in the history

  6. Add pytest-vcr-delete-on-fail 

    To smooth out cassette regeneration workflow -- am constantly getting
HTTP 401/429 responses, which is masking the cause of test failures.
    @hseg
    hseg authored and gesh [email protected] committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    c929126 View commit details
    Browse the repository at this point in the history

  7. Regenerate cassettes 

    Some of these were forgotten -- marked, but never generated.
Some would raise errors, indicating cached broken responses.
Some of these needed to be regenerated so that the timestamp given to
freezegun would be withing tolerable distance of their simulated request
times -- given we're using a single freeze time for the entire project,
all externally-facing cassettes (in our case, facing yt-dlp) need to be
regenerated on each freezegun shift.
    @hseg
    hseg authored and gesh [email protected] committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    ca28b62 View commit details
    Browse the repository at this point in the history