Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation for additional events. #3686

Merged
merged 4 commits into from
Oct 3, 2023
Merged

Add documentation for additional events. #3686

merged 4 commits into from
Oct 3, 2023

Conversation

jonathan343
Copy link
Contributor

@jonathan343 jonathan343 commented Apr 28, 2023

Overview

Our boto3 documentation website currently has a n Extensibility Guide which provides users an introduction and explanation of our event system in botocore/boto3. Despite botocore emitting various types of events, we currently only document 3 of them including creating-client-class, creating-resource-class, and provide-client-params. This PR adds the following:

  • Additional documentation for the following events:
    • before-call
    • request-created
    • before-send
    • needs-retry
    • after-call
    • after-call-error
  • A table providing the order in which all events are emitted along with a link to the location where they're emitted in code.

Testing

I ran make clean html to generate the html version of events.rstand inspected to ensure it was generated correctly.

Screenshot

Screen Shot 2023-07-10 at 11 27 40 AM

@jonathan343 jonathan343 requested review from jonemo and dlm6693 April 28, 2023 16:11
Copy link
Contributor

@dlm6693 dlm6693 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some initial comments/thoughts. I'll need to take another pass at it, but I did notice some inconsistent language to describe the same objects like AWSResponse. I think the wording should be exact and identical everywhere we write it.

docs/source/guide/events.rst Outdated Show resolved Hide resolved

:Description:
This event is emitted before the API request parameters are built.
Use this event to inject or modify parameters after they're validated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to be advertising the injection of parameters that are not validated?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's also just not correct. I used the sample script from below in this section:

    import boto3

    s3 = boto3.client('s3')

    # Access the event system on the S3 client
    event_system = s3.meta.events

    # Create a function
    def add_my_bucket(params, **kwargs):
        # Add the name of the bucket you want to default to.
        if 'Bucket' not in params:
            params['Bucket'] = 'mybucket'

    # Register the function to an event
    event_system.register('before-parameter-build.s3.ListObjectsV2', add_my_bucket)

    response = s3.list_objects_v2()

... and then modified the S3 service model such that the bucket name has a minimum length of 20 characters. If this were correct, there would be no validation error with mybucket. Instead:

botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid length for parameter Bucket, value: 8, valid min length: 20

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah Jonas is right. Input parameter validation happens during serialization, which is called in BaseClient._convert_to_request_dict. before-parameter-build is fired in BaseClient._emit_api_params, which is called earlier.

Could be worth mentioning this event is fired just before endpoint resolution, but not necessary.

docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Show resolved Hide resolved
Comment on lines 484 to 486
:param params: A dictionary where the keys are the names of the
parameters passed through the client method and the values are the values
of those parameters.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
:param params: A dictionary where the keys are the names of the
parameters passed through the client method and the values are the values
of those parameters.
:param params: A dictionary containing key value pairs consisting of the parameters passed through to the client method.

docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
Copy link
Contributor

@jonemo jonemo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of comments on phrasing and such, but most importantly:

  1. All statements about whether events occur before or after parameter validation need fact checking.
  2. Usage of "request parameters" throughout the doc needs to be cleaned up to avoid confusion between operation params and the request dict.

docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved

:Description:
This event is emitted before the API request parameters are built.
Use this event to inject or modify parameters after they're validated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's also just not correct. I used the sample script from below in this section:

    import boto3

    s3 = boto3.client('s3')

    # Access the event system on the S3 client
    event_system = s3.meta.events

    # Create a function
    def add_my_bucket(params, **kwargs):
        # Add the name of the bucket you want to default to.
        if 'Bucket' not in params:
            params['Bucket'] = 'mybucket'

    # Register the function to an event
    event_system.register('before-parameter-build.s3.ListObjectsV2', add_my_bucket)

    response = s3.list_objects_v2()

... and then modified the S3 service model such that the bucket name has a minimum length of 20 characters. If this were correct, there would be no validation error with mybucket. Instead:

botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid length for parameter Bucket, value: 8, valid min length: 20

docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
@Frederik-Baetens
Copy link

There's also a before-sign. Is there any point in documenting that?

Copy link
Contributor

@dlm6693 dlm6693 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple of minor things to address. Once fixed I think this will be good to merge.

docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
docs/source/guide/events.rst Outdated Show resolved Hide resolved
@dlm6693 dlm6693 self-requested a review October 3, 2023 00:12
Copy link
Contributor

@dlm6693 dlm6693 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📦

@jonathan343 jonathan343 merged commit 687f064 into boto:develop Oct 3, 2023
21 checks passed
aws-sdk-python-automation added a commit that referenced this pull request Oct 3, 2023
* release-1.28.59:
  Bumping version to 1.28.59
  Add changelog entries from botocore
  Add documentation for additional events. (#3686)
nateprewitt pushed a commit to nateprewitt/boto3 that referenced this pull request Nov 22, 2023
* Add documentation for additional events.

* Addressed all PR feedback.

* Fixed indentation for needs-retry code example.

* More PR Feedback.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants