Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ForEachResource and CollectResources utils functions #48729

Merged

Conversation

smallinsky
Copy link
Contributor

@smallinsky smallinsky commented Nov 10, 2024

What

This PR introduces the helper functionsForEachResource and CollectResources, which handle pagination automatically. These functions support the following function signatures:

Why

These helper functions simplify resource collection by eliminating the need for manually handling pagination logic. Instead of writing code like this:

	var accessLists []*accesslist.AccessList
	var nextKey string
	for {
		var page []*accesslist.AccessList
		var err error
		page, nextKey, err = client.AccessListClient().ListAccessLists(ctx, 0, nextKey)
		if err != nil {
			return trace.Wrap(err)
		}

		accessLists = append(accessLists, page...)

		if nextKey == "" {
			break
		}
	}

which is prone to bugs e.g.,

  • passing an incorrect nextKey to the list function
  • Forgetting to update nextKey
  • creating a loop scoped variables like:
    var nextKey string
    for {
     page, nextKey, err := client.ListAccessLists(ctx, 0, nextKey)
    ...
    }

users can now simply call:

accessLists err := utils.CollectResources(ctx, svc.ListAccessLists)

This approach minimizes potential errors and makes code more concise and readable.

@smallinsky smallinsky force-pushed the smallinsky/add-collect-resources-for-each-resource branch from 2847c4a to 4866a7b Compare November 10, 2024 17:47
@smallinsky smallinsky marked this pull request as ready for review November 10, 2024 17:49
@smallinsky smallinsky requested a review from r0mant November 10, 2024 17:49
@smallinsky smallinsky added the no-changelog Indicates that a PR does not require a changelog entry label Nov 10, 2024
@smallinsky smallinsky force-pushed the smallinsky/add-collect-resources-for-each-resource branch from 3d824c9 to 795dc1f Compare November 10, 2024 20:35
Copy link
Contributor

@flyinghermit flyinghermit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice

var options ForEachOptions
for _, opt := range opts {
opt(&options)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: set default pageSize if not provided with WithPageSize ? its usually apidefaults.DefaultChunkSize value used as default.

should we also validate the pagesize provided with WithPageSize?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 0 value is valid as page side. I it will mean that the server will use default page size and it is commonly used patter in our codebase for instance

page, nextKey, err = client.AccessListClient().ListAccessLists(ctx, 0, nextKey)

@smallinsky smallinsky force-pushed the smallinsky/add-collect-resources-for-each-resource branch from 04b3d2a to fdf7981 Compare November 11, 2024 19:19
@smallinsky smallinsky added this pull request to the merge queue Nov 11, 2024
Merged via the queue into master with commit d1fdd85 Nov 11, 2024
39 checks passed
@smallinsky smallinsky deleted the smallinsky/add-collect-resources-for-each-resource branch November 11, 2024 20:05
@public-teleport-github-review-bot

@smallinsky See the table below for backport results.

Branch Result
branch/v17 Create PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport/branch/v17 no-changelog Indicates that a PR does not require a changelog entry size/md
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants