Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[core] Improve the performance of show tables #4592

Merged
merged 1 commit into from
Nov 26, 2024
Merged

[core] Improve the performance of show tables #4592

merged 1 commit into from
Nov 26, 2024

Conversation

xleoken
Copy link
Member

@xleoken xleoken commented Nov 26, 2024 •
edited
Loading

Purpose

When store the catalog in s3 object store, as the number of tables increases, the performance of the show tables deteriorates.

Because the FileUtils#listVersionedFileStatus method will send many list request to the s3 server.

Tests

List 100 tables

Before patch

test_table98
test_table99
Time taken: 68.043 seconds, Fetched 100 row(s)

After patch

test_table98
test_table99
Time taken: 2.298 seconds, Fetched 100 row(s)

API and Format

Documentation

@JingsongLi
Copy link
Contributor

+1

@JingsongLi JingsongLi merged commit 139b5a7 into apache:master Nov 26, 2024
12 checks passed
@xleoken xleoken mentioned this pull request Nov 27, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JingsongLi JingsongLi JingsongLi left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@xleoken @JingsongLi