Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

keyvaluetags: Add support for 'ListTags' with pagination methods #10834

Closed
wants to merge 3 commits into from

Conversation

ewbankkit
Copy link
Contributor

@ewbankkit ewbankkit commented Nov 10, 2019

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" comments, they generate extra noise for pull request followers and do not help prioritize the request

Closes #10831.

Release note for CHANGELOG:

NONE

Output from acceptance testing:

# Pagination tokens in request/reponse but no SDK pagination method
$ make testacc TEST=./aws TESTARGS='-run=TestAccAWSAthenaWorkGroup_Tags'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSAthenaWorkGroup_Tags -timeout 120m
=== RUN   TestAccAWSAthenaWorkGroup_Tags
=== PAUSE TestAccAWSAthenaWorkGroup_Tags
=== CONT  TestAccAWSAthenaWorkGroup_Tags
--- PASS: TestAccAWSAthenaWorkGroup_Tags (40.63s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	40.665s
# SDK pagination method
$ make testacc TEST=./aws TESTARGS='-run=TestAccAwsBackupPlan_withTags'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAwsBackupPlan_withTags -timeout 120m
=== RUN   TestAccAwsBackupPlan_withTags
=== PAUSE TestAccAwsBackupPlan_withTags
=== CONT  TestAccAwsBackupPlan_withTags
--- PASS: TestAccAwsBackupPlan_withTags (48.85s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	48.875s

@ewbankkit ewbankkit requested a review from a team November 10, 2019 23:30
@ghost ghost added needs-triage Waiting for first response or review from a maintainer. size/L Managed by automation to categorize the size of a PR. service/backup Issues and PRs that pertain to the backup service. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. size/XL Managed by automation to categorize the size of a PR. and removed size/L Managed by automation to categorize the size of a PR. labels Nov 10, 2019
@ewbankkit ewbankkit changed the title [WIP] keyvaluetags: Add support for 'ListTags' with pagination methods keyvaluetags: Add support for 'ListTags' with pagination methods Nov 11, 2019
@ewbankkit
Copy link
Contributor Author

The list of pagination tokens was generated by an adaption of @kapilt's script:

import boto3
import yaml

def main():
    session = boto3.Session()
    services = session.get_available_services()
    op_missing_paginator = {}
    for s in services:
        c = session.client(s)
        op_to_py = {v: k for k, v in c._PY_TO_OP_NAME.items()}
        for op in c.meta.service_model.operation_names:
            opm = c.meta.service_model.operation_model(op)
            if opm.output_shape is None:
                continue

            input_token = ''
            output_token = ''
            if 'NextToken' in opm.output_shape.members or 'nextToken' in opm.output_shape.members:
                input_token = 'NextToken'
                output_token = 'NextToken'
            elif 'PaginationToken' in opm.output_shape.members:
                input_token = 'PaginationToken'
                output_token = 'PaginationToken'
            elif 'PageToken' in opm.output_shape.members or 'pageToken' in opm.output_shape.members:
                input_token = 'PageToken'
                output_token = 'PageToken'
            elif 'NextPageToken' in opm.output_shape.members or 'nextPageToken' in opm.output_shape.members:
                input_token = 'NextPageToken'
                output_token = 'NextPageToken'
            elif 'Marker' in opm.output_shape.members and 'NextMarker' in opm.output_shape.members or 'marker' in opm.output_shape.members and 'nextMarker' in opm.output_shape.members:
                input_token = 'Marker'
                output_token = 'NextMarker'
            elif 'NextMarker' in opm.output_shape.members or 'nextMarker' in opm.output_shape.members:
                input_token = 'NextMarker'
                output_token = 'NextMarker'
            else:
                continue

            py_name = op_to_py[opm.name]
            if 'tags' in py_name:
                op_missing_paginator.setdefault(s, []).append((py_name, input_token, output_token))

    for s, tokens in op_missing_paginator.items():
        print('case "%s":\nreturn &PaginationTokens{InputToken: "%s", OutputToken: "%s"}' % (s, tokens[0][1], tokens[0][2]))

if __name__ == '__main__':
    main()

@ewbankkit ewbankkit changed the title keyvaluetags: Add support for 'ListTags' with pagination methods [WIP] keyvaluetags: Add support for 'ListTags' with pagination methods Nov 13, 2019
@ewbankkit ewbankkit changed the title [WIP] keyvaluetags: Add support for 'ListTags' with pagination methods keyvaluetags: Add support for 'ListTags' with pagination methods Nov 13, 2019
@ewbankkit
Copy link
Contributor Author

See issue comments.

@ewbankkit ewbankkit closed this Dec 15, 2019
@ewbankkit ewbankkit deleted the issue-10831 branch December 15, 2019 22:36
@ghost
Copy link

ghost commented Jan 15, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Jan 15, 2020
@breathingdust breathingdust removed the needs-triage Waiting for first response or review from a maintainer. label Sep 17, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
service/backup Issues and PRs that pertain to the backup service. size/XL Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

keyvaluetags and paginated "ListTags" results
2 participants