Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BlazingMQ Storage Tool #158

Merged
merged 130 commits into from
Mar 15, 2024
Merged

BlazingMQ Storage Tool #158

merged 130 commits into from
Mar 15, 2024

Conversation

waldgange
Copy link
Collaborator

@waldgange waldgange commented Nov 28, 2023 •
edited
Loading

BMQstoragetool is a CLI tool which can inspect BlazingMQ storage files, and answer various queries.
For example:
$ bmqstoragetool --path /path/to/bmq/storage/files --guid=ABCDEF --queue="foo.bar" --timestamp-gt=123 --timestamp-lt=987 --details --dump-payload --dump-limit=10

alexander-e1off and others added 7 commits November 28, 2023 03:21
@alexander-e1off
Add processor factory and base processor
6e6e89b 
Signed-off-by: Aleksandr Ivanov <[email protected]>
@alexander-e1off
Merge branch 'bmqstoragetool' of https://github.com/waldgange/blazingmq…
6650273 
… into bmqstoragetool
@alexander-e1off
Fix unit test
cf32325 
Signed-off-by: Aleksandr Ivanov <[email protected]>
@alexander-e1off
Fix merge conflicts
1f5d078 
Signed-off-by: Aleksandr Ivanov <[email protected]>
@waldgange
BMQstoragetool initial commit
0678d64 
Signed-off-by: Anton Pryakhin <[email protected]>
@waldgange
Formatting
d32c69d 
Signed-off-by: Anton Pryakhin <[email protected]>
@waldgange
Help parameter
2245226 
Signed-off-by: Anton Pryakhin <[email protected]>
@waldgange waldgange changed the title BMQstoragetool initial commit BMQstoragetool Nov 28, 2023
waldgange and others added 4 commits November 28, 2023 17:01
@waldgange
Dummy
fcfe317 
Signed-off-by: Anton Pryakhin <[email protected]>
Dummy
c93fba3 
Signed-off-by: Anton Pryakhin <[email protected]>
@alexander-e1off
Merge branch 'bmqstoragetool' of https://github.com/waldgange/blazingmq…
6f441c8 
… into bmqstoragetool
@waldgange
Plugins come back
d770a6c 
Signed-off-by: Anton Pryakhin <[email protected]>
@quarter-note quarter-note changed the title BMQstoragetool BlazingMQstoragetool Nov 28, 2023
@quarter-note quarter-note changed the title BlazingMQstoragetool BlazingMQ Storage Tool Nov 28, 2023
@waldgange
Formatting
4e36022 
Signed-off-by: Anton Pryakhin <[email protected]>
@quarter-note quarter-note changed the title BlazingMQ Storage Tool [WIP] BlazingMQ Storage Tool Nov 28, 2023
alexander-e1off and others added 14 commits November 29, 2023 01:40
@alexander-e1off
Add logic to search by GUID
85701aa 
Signed-off-by: Aleksandr Ivanov <[email protected]>
@alexander-e1off
Update search logic and tests
7a691aa 
Signed-off-by: Aleksandr Ivanov <[email protected]>
@alexander-e1off
Add outstanding messages search
0c2e2ab 
Signed-off-by: Aleksandr Ivanov <[email protected]>
@waldgange
Validate cmd arguments and create a command processor
be00e06 
Signed-off-by: Anton Pryakhin <[email protected]>
@waldgange
Validate cmd arguments and create a command processor
504d000 
Signed-off-by: Anton Pryakhin <[email protected]>
@waldgange
Formatting
e9cbafd 
Signed-off-by: Anton Pryakhin <[email protected]>
@alexander-e1off
Add calculation of outstanding ratio
88eb30f 
Signed-off-by: Aleksandr Ivanov <[email protected]>
@alexander-e1off
Merge from bmqtool branch, fix conflicts
cfa3df8 
Signed-off-by: Aleksandr Ivanov <[email protected]>
@alexander-e1off
Get rid of SearchParameters
164e7cb 
Signed-off-by: Aleksandr Ivanov <[email protected]>
@alexander-e1off
Add searching of confirmed and partially confirmed messages
e5852d0 
Signed-off-by: Aleksandr Ivanov <[email protected]>
@waldgange
Fix guids check
91bf21d 
Signed-off-by: Anton Pryakhin <[email protected]>
@alexander-e1off
Refactor to use SearchResult classes
8e95247 
Signed-off-by: Aleksandr Ivanov <[email protected]>
@waldgange
Map files in validator
885ea53 
Signed-off-by: Anton Pryakhin <[email protected]>
@alexander-e1off
Integrate SearchProcessor with updated Parameters, fix integration is…
dfcc4c8 
…sues

Signed-off-by: Aleksandr Ivanov <[email protected]>
@alexander-e1off
Copy link
Collaborator

Hi @kaikulimu, thanks for found bug. We fixed it with case 1. Extend the logic to make sure both queue names and queue keys are accounted for. Also added test for this case. Now with your files it shows correct number:

bmqstoragetool.tsk --journal-path=bmq_test_files/bmqc01\#B/bmq_2*  --csl-file=bmq_test_files/bmqc01\#B/bmq_csl_20230522_145855_37146FB620.bmq_csl --queue-name=bmq://fi.imgr.rfiosrv.~dv/fiom-sheet-2 --queue-key=AEAA830F17
389295 message GUID(s) found.

@waldgange
Handle review notes
392edc5 
Signed-off-by: Anton Pryakhin <[email protected]>
@waldgange
Formatting
8045242 
Signed-off-by: Anton Pryakhin <[email protected]>
@waldgange
Handle review notes
7d9f8b3 
Signed-off-by: Anton Pryakhin <[email protected]>
@waldgange
Check hex
58fba50 
Signed-off-by: Anton Pryakhin <[email protected]>
@kaikulimu
Copy link
Collaborator

Finished my User-Acceptance Testing and everything was satisfactory. Great job!

syuzvinsky
syuzvinsky previously approved these changes Feb 27, 2024
View reviewed changes
kaikulimu
kaikulimu requested changes Feb 27, 2024
View reviewed changes
Copy link
Collaborator

@kaikulimu kaikulimu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments, mostly stylistic. The most important thing is that auto and lambda functions cannot be used, because we need support C++03 compilers on IBM and Solaris.

src/applications/bmqstoragetool/README.md Outdated Show resolved Hide resolved
src/applications/bmqstoragetool/README.md Outdated Show resolved Hide resolved
src/applications/bmqstoragetool/README.md Outdated Show resolved Hide resolved
src/applications/bmqstoragetool/package/bmqstoragetool.mem Show resolved Hide resolved
src/applications/bmqstoragetool/bmqstoragetool.m.cpp Show resolved Hide resolved
src/applications/bmqstoragetool/m_bmqstoragetool_searchresultfactory.cpp Outdated Show resolved Hide resolved
src/applications/bmqstoragetool/m_bmqstoragetool_testutils.h Show resolved Hide resolved
src/applications/bmqstoragetool/m_bmqstoragetool_testutils.h Outdated Show resolved Hide resolved
src/applications/bmqstoragetool/m_bmqstoragetool_testutils.h Show resolved Hide resolved
src/applications/bmqstoragetool/m_bmqstoragetool_testutils.h Show resolved Hide resolved
alexander-e1off and others added 2 commits February 28, 2024 22:56
@alexander-e1off
Fix code style and add code documentation
62beba3 
Signed-off-by: Aleksandr Ivanov <[email protected]>
@waldgange
Merge pull request #5 from waldgange/search-processor
aef96d7 
Fix code style and add code documentation
kaikulimu
kaikulimu reviewed Mar 14, 2024
View reviewed changes
Copy link
Collaborator

@kaikulimu kaikulimu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comments. Should be good to go after this.

src/applications/bmqstoragetool/m_bmqstoragetool_searchresult.h Outdated Show resolved Hide resolved
kaikulimu
kaikulimu approved these changes Mar 15, 2024
View reviewed changes
Copy link
Collaborator

@kaikulimu kaikulimu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kaikulimu kaikulimu merged commit b6a9727 into bloomberg:main Mar 15, 2024
10 checks passed
@waldgange waldgange deleted the bmqstoragetool branch March 22, 2024 10:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alexander-e1off alexander-e1off alexander-e1off left review comments

@678098 678098 678098 left review comments

@kaikulimu kaikulimu kaikulimu approved these changes

@syuzvinsky syuzvinsky syuzvinsky left review comments

Assignees

@kaikulimu kaikulimu

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@waldgange @kaikulimu @alexander-e1off @678098 @syuzvinsky