Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

implementation of additional requests #22

Merged
merged 49 commits into from
Apr 27, 2022
Merged

implementation of additional requests #22

merged 49 commits into from
Apr 27, 2022

Commits on Feb 8, 2022

  1. add basic implementation of additional requests

    @BurnzZ
    BurnzZ committed Feb 8, 2022
    Configuration menu
    Copy the full SHA
    012a199 View commit details
    Browse the repository at this point in the history

  2. introduce concepts of GenericRequest and HttpClient

    @BurnzZ
    BurnzZ committed Feb 8, 2022
    Configuration menu
    Copy the full SHA
    6d38fd0 View commit details
    Browse the repository at this point in the history

  3. prevent HttpClient from being keyword arg in ItemWebPage

    @BurnzZ
    BurnzZ committed Feb 8, 2022
    Configuration menu
    Copy the full SHA
    cf7de8d View commit details
    Browse the repository at this point in the history

  4. revert changes to page.py

    @BurnzZ
    BurnzZ committed Feb 8, 2022
    Configuration menu
    Copy the full SHA
    391a449 View commit details
    Browse the repository at this point in the history

  5. assign perform_request as the default downloader for HttpClient

    @BurnzZ
    BurnzZ committed Feb 8, 2022
    Configuration menu
    Copy the full SHA
    f3bfa95 View commit details
    Browse the repository at this point in the history

Commits on Feb 9, 2022

  1. add status and headers to ResponseData

    @BurnzZ
    BurnzZ committed Feb 9, 2022
    Configuration menu
    Copy the full SHA
    e0f247b View commit details
    Browse the repository at this point in the history

  2. fix mypy errors on ResponseData.headers

    @BurnzZ
    BurnzZ committed Feb 9, 2022
    Configuration menu
    Copy the full SHA
    feff29d View commit details
    Browse the repository at this point in the history

Commits on Feb 18, 2022

  1. support multiple inputs in HttpClient.request

    @BurnzZ
    BurnzZ committed Feb 18, 2022
    Configuration menu
    Copy the full SHA
    15d0525 View commit details
    Browse the repository at this point in the history

  2. add tests for additional request components

    @BurnzZ
    BurnzZ committed Feb 18, 2022
    Configuration menu
    Copy the full SHA
    c5537ce View commit details
    Browse the repository at this point in the history

  3. refactor HttpClient's request interface

    @BurnzZ
    BurnzZ committed Feb 18, 2022
    Configuration menu
    Copy the full SHA
    74e5c89 View commit details
    Browse the repository at this point in the history

Commits on Mar 14, 2022

  1. add docs for additional requests

    @BurnzZ
    BurnzZ committed Mar 14, 2022
    Configuration menu
    Copy the full SHA
    f99a6e1 View commit details
    Browse the repository at this point in the history

  2. update CHANGELOG about HttpClient support for additional requests

    @BurnzZ
    BurnzZ committed Mar 14, 2022
    Configuration menu
    Copy the full SHA
    9025660 View commit details
    Browse the repository at this point in the history

  3. fix failing test on refactored _perform_request()

    @BurnzZ
    BurnzZ committed Mar 14, 2022
    Configuration menu
    Copy the full SHA
    4e803a4 View commit details
    Browse the repository at this point in the history

  4. fix incorrect type annotation for HttpClient.batch_requests()

    @BurnzZ
    BurnzZ committed Mar 14, 2022
    Configuration menu
    Copy the full SHA
    355294e View commit details
    Browse the repository at this point in the history

  5. introduce a concept of Meta to pass things around inside Page Objects

    @BurnzZ
    BurnzZ committed Mar 14, 2022
    Configuration menu
    Copy the full SHA
    1d29328 View commit details
    Browse the repository at this point in the history

  6. add __repr__ to Meta

    @BurnzZ
    BurnzZ committed Mar 14, 2022
    Configuration menu
    Copy the full SHA
    706f698 View commit details
    Browse the repository at this point in the history

  7. add tests for Meta

    @BurnzZ
    BurnzZ committed Mar 14, 2022
    Configuration menu
    Copy the full SHA
    9ae60e6 View commit details
    Browse the repository at this point in the history

  8. refactor Meta to subclass from a dict with added value restrictions a… 

    …nd key requirements
    @BurnzZ
    BurnzZ committed Mar 14, 2022
    Configuration menu
    Copy the full SHA
    658d1b7 View commit details
    Browse the repository at this point in the history

  9. remove feature to specify required data in Meta

    @BurnzZ
    BurnzZ committed Mar 14, 2022
    Configuration menu
    Copy the full SHA
    1e038fe View commit details
    Browse the repository at this point in the history

  10. rename is_restricted_value() into enforce_value_restriction() in Meta

    @BurnzZ
    BurnzZ committed Mar 14, 2022
    Configuration menu
    Copy the full SHA
    9d4940d View commit details
    Browse the repository at this point in the history

  11. add doc for Meta

    @BurnzZ
    BurnzZ committed Mar 14, 2022
    Configuration menu
    Copy the full SHA
    1caec0d View commit details
    Browse the repository at this point in the history

  12. update CHANGELOG for Meta

    @BurnzZ
    BurnzZ committed Mar 14, 2022
    Configuration menu
    Copy the full SHA
    a404189 View commit details
    Browse the repository at this point in the history

Commits on Mar 21, 2022

  1. remove value restriction of Meta

    @BurnzZ
    BurnzZ committed Mar 21, 2022
    Configuration menu
    Copy the full SHA
    a8d9530 View commit details
    Browse the repository at this point in the history

  2. remove unused import

    @BurnzZ
    BurnzZ committed Mar 21, 2022
    Configuration menu
    Copy the full SHA
    079ccc1 View commit details
    Browse the repository at this point in the history

  3. Merge pull request #23 from scrapinghub/meta 

    Introduce Meta as a way to pass information inside a PO
    @kmike
    kmike authored Mar 21, 2022
    Configuration menu
    Copy the full SHA
    396ab8e View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2022

  1. Merge branch 'master' into additional-requests 

    Adjust the code in line with the refactoring of
ResponseData into HttpResopnse in this PR:
#30
    @BurnzZ
    BurnzZ committed Mar 28, 2022
    Configuration menu
    Copy the full SHA
    f769147 View commit details
    Browse the repository at this point in the history

Commits on Apr 4, 2022

  1. update doc filenames to be consistent in naming format

    @BurnzZ
    BurnzZ committed Apr 4, 2022
    Configuration menu
    Copy the full SHA
    83839cf View commit details
    Browse the repository at this point in the history

  2. refactor Request to become HttpRequest 

    This makes it consistent with the recently refactored HttpResponse:
#30
    @BurnzZ
    BurnzZ committed Apr 4, 2022
    Configuration menu
    Copy the full SHA
    3a76bc3 View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2022

  1. update docs to have a dedicated section for HttpRequest

    @BurnzZ
    BurnzZ committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    a853d16 View commit details
    Browse the repository at this point in the history

  2. general doc improvements to additional requests and meta

    @BurnzZ
    BurnzZ committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    f0a3419 View commit details
    Browse the repository at this point in the history

  3. fix code examples in docs for HttpClient and Meta

    @BurnzZ
    BurnzZ committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    e74ca50 View commit details
    Browse the repository at this point in the history

  4. remove the alternative constructor from_anystr() from HttpRequestBody

    @BurnzZ
    BurnzZ committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    109a670 View commit details
    Browse the repository at this point in the history

Commits on Apr 6, 2022

  1. formalize exceptions and handling them

    @BurnzZ
    BurnzZ committed Apr 6, 2022
    Configuration menu
    Copy the full SHA
    eda3352 View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2022

  1. refactor to create base classes for HttpRequest and HttpBody

    @BurnzZ
    BurnzZ committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    65badca View commit details
    Browse the repository at this point in the history

  2. move HttpRequest* classes from into the page_inputs.py module

    @BurnzZ
    BurnzZ committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    99e9b41 View commit details
    Browse the repository at this point in the history

  3. use keyword args for HttpRequest in HttpClient

    @BurnzZ
    BurnzZ committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    6a53b90 View commit details
    Browse the repository at this point in the history

  4. remove return type annotation on HttpResponseBody.json()

    @BurnzZ
    BurnzZ committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    c1f7675 View commit details
    Browse the repository at this point in the history

  5. remove proposed _HttpBody

    @BurnzZ
    BurnzZ committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    3310c1b View commit details
    Browse the repository at this point in the history

  6. Merge branch 'master' into additional-requests

    @BurnzZ
    BurnzZ committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    29f84c6 View commit details
    Browse the repository at this point in the history

  7. make init params of HttpRequest/HttpResponse kw-only except for url

    @BurnzZ
    BurnzZ committed Apr 11, 2022
    2 Configuration menu
    Copy the full SHA
    d88a0e5 View commit details
    Browse the repository at this point in the history

  8. reverted change on HttpResponse.body being keyword-only

    @BurnzZ
    BurnzZ committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    cf6e663 View commit details
    Browse the repository at this point in the history

Commits on Apr 13, 2022

  1. update batch_requests() to return errors in failed requests

    @BurnzZ
    BurnzZ committed Apr 13, 2022
    Configuration menu
    Copy the full SHA
    5d2a751 View commit details
    Browse the repository at this point in the history

  2. improve docs to add more examples and fix existing ones

    @BurnzZ
    BurnzZ committed Apr 13, 2022
    Configuration menu
    Copy the full SHA
    7947d58 View commit details
    Browse the repository at this point in the history

  3. update test to check out positional args in HttpResponse's body

    @BurnzZ
    BurnzZ committed Apr 13, 2022
    Configuration menu
    Copy the full SHA
    e7b4f08 View commit details
    Browse the repository at this point in the history

Commits on Apr 19, 2022

  1. expose 'return_exceptions' param to HttpClient.batch_requests()

    @BurnzZ
    BurnzZ committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    bc4baea View commit details
    Browse the repository at this point in the history

  2. make Headers and Body type variables private only to requests.py

    @BurnzZ
    BurnzZ committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    9f43187 View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2022

  1. update batch_requests default param value of return_exceptions from T… 

    …rue to False
    @BurnzZ
    BurnzZ committed Apr 25, 2022
    Configuration menu
    Copy the full SHA
    80f95a8 View commit details
    Browse the repository at this point in the history

  2. expose execute() in HttpClient and rename batch_requests() into batch… 

    …_execute()
    @BurnzZ
    BurnzZ committed Apr 25, 2022
    Configuration menu
    Copy the full SHA
    5d82a37 View commit details
    Browse the repository at this point in the history

  3. improve additional request docs

    @BurnzZ
    BurnzZ committed Apr 25, 2022
    Configuration menu
    Copy the full SHA
    753e6ad View commit details
    Browse the repository at this point in the history