neofs-lens: storage status new command #2591

AliceInHunterland · 2023-09-25T16:05:11Z

New command:

neofs-lens storage status --address ApueGak8ZwgXtxhhV3Bf97sbuByBDFkJSAZ4921g7hUr/E6bMcEHxjnrMRaRx5fjPe6WGsFS3jW9Q6SmMvHSskmNr  --config ./data/config/config-sn.yaml

Expected result looks like:

Shard ID:  7cCVnpBHZMZdfo8GfhKCSf
	Substorage  0
		Blob Type:  peapod
		Blob Path:  ./data/peapod1.db
		Blob Error:  status: code = 2049 message = object not found
	Substorage  1
		Blob Type:  fstree
		Blob Path:  ./data/fstree1
		Blob Error:  status: code = 2049 message = object not found
	Metabase
		Metabase storage ID:  
		Metabase path:  ./data/meta1
		Metabase object status:  NOT IN GRAVEYARD
		Metabase object error:  <nil>
	Writecache
		Writecache DB path:  
		Writecache DB error:  <nil>
		Writecache  FSTree path:  
		Writecache FSTree error:  <nil>
Shard ID:  QQD7WcPs6crwwSSZ3A3Ak6
	Substorage  0
		Blob Type:  blobovnicza
		Blob Path:  ./data/blob/blobovnicza
		Blob Error:  <nil>
	Substorage  1
		Blob Type:  fstree
		Blob Path:  ./data/blob
		Blob Error:  status: code = 2049 message = object not found
	Metabase
		Metabase storage ID:  0/0
		Metabase path:  ./data/metabase.db
		Metabase object status:  AVAILABLE NOT IN GRAVEYARD
		Metabase object error:  <nil>
	Writecache
		Writecache DB path:  data/wc0/small.bolt
		Writecache DB error:  object not found
		Writecache  FSTree path:  ./data/wc0
		Writecache FSTree error:  object not found

In the initial issue was proposed to display only parts of the storage where the object belongs. As for now, this command displays all parts of storage. what do you think about it? also, there is a question about word choice and error display.
Refs: #2550.

codecov · 2023-09-25T16:09:15Z

Codecov Report

Merging #2591 (e9ffb8e) into master (1d7bd26) will decrease coverage by 0.09%.
The diff coverage is 0.00%.

❗ Current head e9ffb8e differs from pull request most recent head 4c1ab5e. Consider uploading reports for the commit 4c1ab5e to get more accurate results

@@            Coverage Diff             @@
##           master    #2591      +/-   ##
==========================================
- Coverage   29.71%   29.62%   -0.09%     
==========================================
  Files         411      416       +5     
  Lines       31431    31523      +92     
==========================================
  Hits         9339     9339              
- Misses      21280    21372      +92     
  Partials      812      812

Files	Coverage Δ
pkg/local_object_storage/writecache/writecache.go	`77.55% <ø> (ø)`
pkg/local_object_storage/shard/status.go	`0.00% <0.00%> (ø)`
pkg/local_object_storage/engine/status.go	`0.00% <0.00%> (ø)`
pkg/local_object_storage/blobstor/status.go	`0.00% <0.00%> (ø)`
pkg/local_object_storage/writecache/status.go	`0.00% <0.00%> (ø)`
pkg/local_object_storage/metabase/status.go	`0.00% <0.00%> (ø)`

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

roman-khimov

I'd try simplifying the interface, even though metabase one does require some specifics. And maybe only deal with cases where there is some data for the object, lots of inexistence errors are not very useful (we can have a lot of shards).

cmd/neofs-lens/internal/printers.go

roman-khimov · 2023-09-27T20:38:36Z

pkg/local_object_storage/shard/status.go

+	Blob       blobstor.ObjectStatus
+	Metabase   meta.ObjectStatus
+	Writecache writecache.ObjectStatus
+	Errors     []error


It's really unobvious what this slice means without looking into the implementation.

cmd/neofs-lens/internal/printers.go

carpawell · 2023-09-29T11:27:57Z

pkg/local_object_storage/metabase/status.go

+			res.State = append(res.State, "AVAILABLE")
+		}
+		if removedStatus == 1 {
+			res.State = append(res.State, "GC MARKED")


so in the worst scenario, could it print LOCKED AVAILABLE AVAILABLE GC MARKED?

it could be such a situation when an object is inBucket and inGraveyardWithKey at the same time?

if so it could be LOCKED AVAILABLE AVAILABLE GC MARKED

"AVAILABLE AVAILABLE" is better be avoided, you can use some different terms.

i would expect printing statuses if they are interested, e.g.: a regular available object -- shard ID, if locked, another line in the print about it, if removed but locked -- one more line. if not found at all -- just "not found" line or smth like that

it could be such a situation when an object is inBucket

depends on what bucket we are talking about. but an object can be placed in a shard, be removed but be locked so LOCKED AVAILABLE AVAILABLE GC MARKED seems reachable (but not clear)

AliceInHunterland · 2023-10-02T08:45:06Z

@roman-khimov what do you mean by

metabase one does require some specifics

it seems that metabase does not have any other specifications.
for this moment we have:

neofs-lens storage status --address ApueGak8ZwgXtxhhV3Bf97sbuByBDFkJSAZ4921g7hUr/E6bMcEHxjnrMRaRx5fjPe6WGsFS3jW9Q6SmMvHSskmNr  --config ./data/config/config-sn.yaml

Shard ID:   QQD7WcPs6crwwSSZ3A3Ak6
            Substorage 0
               Blob Type:   blobovnicza
               Blob Path:   ./data/blob/blobovnicza
            Metabase
               Metabase storage ID:      0/0
               Metabase path:            ./data/metabase.db
               Metabase object status:   AVAILABLE NOT IN GRAVEYARD

roman-khimov · 2023-10-02T14:41:11Z

I mean it differs from "normal" storages.

roman-khimov · 2023-10-03T15:44:24Z

pkg/local_object_storage/metabase/status.go

+			res.State = append(res.State, "AVAILABLE")
+		}
+		if removedStatus == 1 {
+			res.State = append(res.State, "GC MARKED")


"AVAILABLE AVAILABLE" is better be avoided, you can use some different terms.

pkg/local_object_storage/writecache/status.go

pkg/local_object_storage/metabase/status.go

cmd/neofs-lens/internal/printers.go

carpawell · 2023-10-04T09:59:50Z

pkg/local_object_storage/metabase/status.go

+			res.State = append(res.State, "AVAILABLE")
+		}
+		if removedStatus == 1 {
+			res.State = append(res.State, "GC MARKED")


i would expect printing statuses if they are interested, e.g.: a regular available object -- shard ID, if locked, another line in the print about it, if removed but locked -- one more line. if not found at all -- just "not found" line or smth like that

it could be such a situation when an object is inBucket

depends on what bucket we are talking about. but an object can be placed in a shard, be removed but be locked so LOCKED AVAILABLE AVAILABLE GC MARKED seems reachable (but not clear)

pkg/local_object_storage/writecache/status.go

roman-khimov

Please rebase again, the CHANGELOG has changed.

pkg/local_object_storage/writecache/status.go

This command prints out information about storage and its components, where object is located. It can be useful for object inspection and storage health check. Refs: #2550. Signed-off-by: Ekaterina Pavlova <[email protected]>

AliceInHunterland force-pushed the feature/2550-neofs-lens-storage-status-command branch from 6163fa2 to 345cc44 Compare September 26, 2023 07:35

AliceInHunterland marked this pull request as ready for review September 26, 2023 07:37

AliceInHunterland requested review from roman-khimov, carpawell and cthulhu-rider as code owners September 26, 2023 07:37

roman-khimov reviewed Sep 27, 2023

View reviewed changes

carpawell reviewed Sep 29, 2023

View reviewed changes

AliceInHunterland force-pushed the feature/2550-neofs-lens-storage-status-command branch 2 times, most recently from 9067829 to 8222ff4 Compare September 29, 2023 11:52

AliceInHunterland requested review from roman-khimov and carpawell October 2, 2023 10:00

roman-khimov reviewed Oct 3, 2023

View reviewed changes

carpawell reviewed Oct 4, 2023

View reviewed changes

AliceInHunterland force-pushed the feature/2550-neofs-lens-storage-status-command branch from 8222ff4 to 39c2cfe Compare October 5, 2023 08:22

AliceInHunterland requested review from roman-khimov and carpawell October 9, 2023 05:17

AliceInHunterland force-pushed the feature/2550-neofs-lens-storage-status-command branch from 39c2cfe to 0be3397 Compare October 10, 2023 14:05

roman-khimov reviewed Oct 11, 2023

View reviewed changes

pkg/local_object_storage/writecache/status.go Outdated Show resolved Hide resolved

AliceInHunterland force-pushed the feature/2550-neofs-lens-storage-status-command branch 2 times, most recently from 2549f5d to b9c6606 Compare October 11, 2023 19:44

neofs-lens: storage status new command

4c1ab5e

This command prints out information about storage and its components, where object is located. It can be useful for object inspection and storage health check. Refs: #2550. Signed-off-by: Ekaterina Pavlova <[email protected]>

AliceInHunterland force-pushed the feature/2550-neofs-lens-storage-status-command branch from b9c6606 to 4c1ab5e Compare October 11, 2023 19:49

roman-khimov approved these changes Oct 11, 2023

View reviewed changes

roman-khimov merged commit 41e17ce into master Oct 12, 2023
8 checks passed

roman-khimov deleted the feature/2550-neofs-lens-storage-status-command branch October 12, 2023 06:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

neofs-lens: storage status new command #2591

neofs-lens: storage status new command #2591

AliceInHunterland commented Sep 25, 2023 •

edited

Loading

codecov bot commented Sep 25, 2023 •

edited

Loading

roman-khimov left a comment

roman-khimov Sep 27, 2023

carpawell Sep 29, 2023

AliceInHunterland Sep 29, 2023 •

edited

Loading

roman-khimov Oct 3, 2023

carpawell Oct 4, 2023

AliceInHunterland commented Oct 2, 2023 •

edited

Loading

roman-khimov commented Oct 2, 2023

roman-khimov Oct 3, 2023

carpawell Oct 4, 2023

roman-khimov left a comment

neofs-lens: storage status new command #2591

neofs-lens: storage status new command #2591

Conversation

AliceInHunterland commented Sep 25, 2023 • edited Loading

codecov bot commented Sep 25, 2023 • edited Loading

Codecov Report

roman-khimov left a comment

Choose a reason for hiding this comment

roman-khimov Sep 27, 2023

Choose a reason for hiding this comment

carpawell Sep 29, 2023

Choose a reason for hiding this comment

AliceInHunterland Sep 29, 2023 • edited Loading

Choose a reason for hiding this comment

roman-khimov Oct 3, 2023

Choose a reason for hiding this comment

carpawell Oct 4, 2023

Choose a reason for hiding this comment

AliceInHunterland commented Oct 2, 2023 • edited Loading

roman-khimov commented Oct 2, 2023

roman-khimov Oct 3, 2023

Choose a reason for hiding this comment

carpawell Oct 4, 2023

Choose a reason for hiding this comment

roman-khimov left a comment

Choose a reason for hiding this comment

AliceInHunterland commented Sep 25, 2023 •

edited

Loading

codecov bot commented Sep 25, 2023 •

edited

Loading

AliceInHunterland Sep 29, 2023 •

edited

Loading

AliceInHunterland commented Oct 2, 2023 •

edited

Loading