Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement](MS) Add fix tablet data size api for meta service #41782

Merged
merged 20 commits into from
Oct 30, 2024

Conversation

Yukang-Lian
Copy link
Collaborator

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Yukang-Lian
Copy link
Collaborator Author

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@Yukang-Lian Yukang-Lian force-pushed the Add-Api-For-Meta-Service branch from 17f8700 to bd799ef Compare October 15, 2024 17:09
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

gensrc/proto/cloud.proto Outdated Show resolved Hide resolved
@apache apache deleted a comment from github-actions bot Oct 16, 2024
@apache apache deleted a comment from github-actions bot Oct 16, 2024
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@@ -2188,4 +2188,238 @@ std::pair<MetaServiceCode, std::string> MetaServiceImpl::get_instance_info(
return {code, std::move(msg)};
}

MetaServiceResponseStatus MetaServiceImpl::fix_tablet_stats(std::string cloud_unique_id_str,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'fix_tablet_stats' has cognitive complexity of 53 (threshold 50) [readability-function-cognitive-complexity]

MetaServiceResponseStatus MetaServiceImpl::fix_tablet_stats(std::string cloud_unique_id_str,
                                           ^
Additional context

cloud/src/meta-service/meta_service.cpp:2200: +1, including nesting penalty of 0, nesting level increased to 1

    } catch (...) {
      ^

cloud/src/meta-service/meta_service.cpp:2207: +1, including nesting penalty of 0, nesting level increased to 1

    if (instance_id.empty()) {
    ^

cloud/src/meta-service/meta_service.cpp:2218: +1, including nesting penalty of 0, nesting level increased to 1

    if (err != TxnErrorCode::TXN_OK) {
    ^

cloud/src/meta-service/meta_service.cpp:2236: +1, including nesting penalty of 0, nesting level increased to 1

    do {
    ^

cloud/src/meta-service/meta_service.cpp:2238: +2, including nesting penalty of 1, nesting level increased to 2

        if (err != TxnErrorCode::TXN_OK) {
        ^

cloud/src/meta-service/meta_service.cpp:2244: +2, including nesting penalty of 1, nesting level increased to 2

        while (it->has_next()) {
        ^

cloud/src/meta-service/meta_service.cpp:2251: +3, including nesting penalty of 2, nesting level increased to 3

            if (out.size() == 7) {
            ^

cloud/src/meta-service/meta_service.cpp:2268: +4, including nesting penalty of 3, nesting level increased to 4

                if (code != MetaServiceCode::OK) {
                ^

cloud/src/meta-service/meta_service.cpp:2291: +4, including nesting penalty of 3, nesting level increased to 4

                if (!tablet_stat.SerializeToString(&tablet_stat_value)) {
                ^

cloud/src/meta-service/meta_service.cpp:2324: +4, including nesting penalty of 3, nesting level increased to 4

                if (err != TxnErrorCode::TXN_OK) {
                ^

cloud/src/meta-service/meta_service.cpp:2338: +4, including nesting penalty of 3, nesting level increased to 4

                if (tablet_stat_check.DebugString() != tablet_stat.DebugString()) {
                ^

cloud/src/meta-service/meta_service.cpp:2349: +4, including nesting penalty of 3, nesting level increased to 4

                if (err != TxnErrorCode::TXN_OK) {
                ^

cloud/src/meta-service/meta_service.cpp:2358: +4, including nesting penalty of 3, nesting level increased to 4

                if (tablet_stat_data_size_value.size() != sizeof(tablet_stat_data_size_check))
                ^

cloud/src/meta-service/meta_service.cpp:2368: +4, including nesting penalty of 3, nesting level increased to 4

                if constexpr (std::endian::native == std::endian::big) {
                ^

cloud/src/meta-service/meta_service.cpp:2375: +4, including nesting penalty of 3, nesting level increased to 4

                if (tablet_stat_data_size_check != tablet_stat_data_size) {
                ^

cloud/src/meta-service/meta_service.cpp:2385: +3, including nesting penalty of 2, nesting level increased to 3

            if (sub_txn_id % 50 == 0) {
            ^

cloud/src/meta-service/meta_service.cpp:2387: +4, including nesting penalty of 3, nesting level increased to 4

                if (err != TxnErrorCode::TXN_OK) {
                ^

cloud/src/meta-service/meta_service.cpp:2393: +1, nesting level increased to 4

                } else {
                  ^

cloud/src/meta-service/meta_service.cpp:2405: +1, including nesting penalty of 0, nesting level increased to 1

    if (err != TxnErrorCode::TXN_OK) {
    ^

cloud/src/meta-service/meta_service.cpp:2413: +1, nesting level increased to 1

    } else {
      ^

@@ -2188,4 +2188,238 @@
return {code, std::move(msg)};
}

MetaServiceResponseStatus MetaServiceImpl::fix_tablet_stats(std::string cloud_unique_id_str,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'fix_tablet_stats' exceeds recommended size/complexity thresholds [readability-function-size]

MetaServiceResponseStatus MetaServiceImpl::fix_tablet_stats(std::string cloud_unique_id_str,
                                           ^
Additional context

cloud/src/meta-service/meta_service.cpp:2190: 231 lines including whitespace and comments (threshold 80)

MetaServiceResponseStatus MetaServiceImpl::fix_tablet_stats(std::string cloud_unique_id_str,
                                           ^

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@@ -2188,4 +2188,342 @@ std::pair<MetaServiceCode, std::string> MetaServiceImpl::get_instance_info(
return {code, std::move(msg)};
}

MetaServiceResponseStatus MetaServiceImpl::fix_tablet_stats(std::string cloud_unique_id_str,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'fix_tablet_stats' exceeds recommended size/complexity thresholds [readability-function-size]

MetaServiceResponseStatus MetaServiceImpl::fix_tablet_stats(std::string cloud_unique_id_str,
                                           ^
Additional context

cloud/src/meta-service/meta_service.cpp:2190: 335 lines including whitespace and comments (threshold 80)

MetaServiceResponseStatus MetaServiceImpl::fix_tablet_stats(std::string cloud_unique_id_str,
                                           ^

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@@ -18,10 +18,17 @@
#include "meta-service/meta_service_tablet_stats.h"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: 'meta-service/meta_service_tablet_stats.h' file not found [clang-diagnostic-error]

#include "meta-service/meta_service_tablet_stats.h"
         ^

@@ -19,6 +19,8 @@

#include <gen_cpp/cloud.pb.h>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: 'gen_cpp/cloud.pb.h' file not found [clang-diagnostic-error]

#include <gen_cpp/cloud.pb.h>
         ^

@Yukang-Lian Yukang-Lian force-pushed the Add-Api-For-Meta-Service branch from 63eb9a8 to a745f7e Compare October 24, 2024 15:01
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

return st;
}

MetaServiceResponseStatus fix_tablet_stats_internal(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'fix_tablet_stats_internal' exceeds recommended size/complexity thresholds [readability-function-size]

MetaServiceResponseStatus fix_tablet_stats_internal(
                          ^
Additional context

cloud/src/meta-service/meta_service_tablet_stats.cpp:196: 104 lines including whitespace and comments (threshold 80)

MetaServiceResponseStatus fix_tablet_stats_internal(
                          ^

@Yukang-Lian Yukang-Lian force-pushed the Add-Api-For-Meta-Service branch from 2d305ac to fa2095a Compare October 28, 2024 12:00
@Yukang-Lian
Copy link
Collaborator Author

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

return st;
}

MetaServiceResponseStatus fix_tablet_stats_internal(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'fix_tablet_stats_internal' exceeds recommended size/complexity thresholds [readability-function-size]

MetaServiceResponseStatus fix_tablet_stats_internal(
                          ^
Additional context

cloud/src/meta-service/meta_service_tablet_stats.cpp:196: 126 lines including whitespace and comments (threshold 80)

MetaServiceResponseStatus fix_tablet_stats_internal(
                          ^

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.46% (9717/25937)
Line Coverage: 28.73% (80613/280543)
Region Coverage: 28.16% (41695/148049)
Branch Coverage: 24.72% (21182/85682)
Coverage Report: http://coverage.selectdb-in.cc/coverage/fa2095a202d0e920870fb17172c227dd260acb10_fa2095a202d0e920870fb17172c227dd260acb10/report/index.html

@Yukang-Lian
Copy link
Collaborator Author

run cloud_p0

@Yukang-Lian
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.46% (9715/25937)
Line Coverage: 28.72% (80578/280543)
Region Coverage: 28.14% (41667/148049)
Branch Coverage: 24.71% (21170/85682)
Coverage Report: http://coverage.selectdb-in.cc/coverage/8fa5823d2afc21c28ad6c42fc90a0f918f2c67cc_8fa5823d2afc21c28ad6c42fc90a0f918f2c67cc/report/index.html

@Yukang-Lian Yukang-Lian force-pushed the Add-Api-For-Meta-Service branch from a0029ff to f125f11 Compare October 29, 2024 18:29
@Yukang-Lian
Copy link
Collaborator Author

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

return st;
}

MetaServiceResponseStatus fix_tablet_stats_internal(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'fix_tablet_stats_internal' exceeds recommended size/complexity thresholds [readability-function-size]

MetaServiceResponseStatus fix_tablet_stats_internal(
                          ^
Additional context

cloud/src/meta-service/meta_service_tablet_stats.cpp:196: 125 lines including whitespace and comments (threshold 80)

MetaServiceResponseStatus fix_tablet_stats_internal(
                          ^

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.48% (9721/25938)
Line Coverage: 28.74% (80644/280620)
Region Coverage: 28.15% (41674/148042)
Branch Coverage: 24.72% (21173/85668)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f125f11a55a7746211f1b72603c055938a9360fe_f125f11a55a7746211f1b72603c055938a9360fe/report/index.html

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 30, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Collaborator

@TangSiyang2001 TangSiyang2001 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gavinchou gavinchou merged commit 3c7fe3f into apache:master Oct 30, 2024
24 of 27 checks passed
Yukang-Lian added a commit to Yukang-Lian/doris that referenced this pull request Nov 7, 2024
dataroaring pushed a commit that referenced this pull request Nov 11, 2024
… size api for meta service (#41782)" (#43460)

Pick #41782

---------

Co-authored-by: Yukang-Lian <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.3-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants