Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify engine interface #3008

Merged
merged 16 commits into from
Nov 12, 2024
Merged

Simplify engine interface #3008

merged 16 commits into from
Nov 12, 2024

Conversation

roman-khimov
Copy link
Member

Mostly similar to #3001, we have a number of Prm/Res structures that make it harder to used engine and trace things going on. But there are some functional changes here as well and also things that go beyond Prm/Res. Overall we get rid of many pointless conversions.

These APIs seem to be totally unused.

Signed-off-by: Roman Khimov <[email protected]>
We _never_ used non-forced deletion, so only a forced version is provided now.
But maybe we can just Inhume().

Signed-off-by: Roman Khimov <[email protected]>
No Prm/Res and ignore option is dropped too since there were no users of it.

Signed-off-by: Roman Khimov <[email protected]>
Pure refactoring, no functional changes.

Signed-off-by: Roman Khimov <[email protected]>
I believe it's more correct, since an object can be locked, but if we're
gholding a copy no one needs it still should go away.

Signed-off-by: Roman Khimov <[email protected]>
Both can check for locked status, but this doesn't make much sense, reuse
code.

Signed-off-by: Roman Khimov <[email protected]>
Delete is a forced removal of a given address, use it this way.

Signed-off-by: Roman Khimov <[email protected]>
It's not used by any real code, Delete() is sufficient and Inhume can now be
used for tombstone handling only.

Signed-off-by: Roman Khimov <[email protected]>
It's specifically for tombstones now since other deletion cases work via
Delete().

Signed-off-by: Roman Khimov <[email protected]>
Can be split into two methods in future, but works fine for now.

Signed-off-by: Roman Khimov <[email protected]>
I believe these should be counted irrespective of blocked/unblocked state,
doing this in internal functions is also error-prone since we can count for
some unrelated calls as well.

Signed-off-by: Roman Khimov <[email protected]>
The only reason they existed is because StorageEngine interfaces were unusable.
Now they're not and these wrappers are trivial and do nothing in most of the
cases.

Signed-off-by: Roman Khimov <[email protected]>
@roman-khimov roman-khimov added enhancement Improving existing functionality U4 Nothing urgent S3 Minimally significant I4 No visible changes labels Nov 11, 2024
@roman-khimov roman-khimov added this to the v0.44.0 milestone Nov 11, 2024
Copy link

codecov bot commented Nov 11, 2024

Codecov Report

Attention: Patch coverage is 52.55814% with 102 lines in your changes missing coverage. Please review.

Project coverage is 23.01%. Comparing base (007e992) to head (e482dbf).
Report is 17 commits behind head on master.

Files with missing lines Patch % Lines
pkg/local_object_storage/engine/container.go 0.00% 27 Missing ⚠️
pkg/local_object_storage/engine/select.go 44.44% 14 Missing and 1 partial ⚠️
pkg/services/object/get/util.go 0.00% 7 Missing ⚠️
pkg/local_object_storage/engine/evacuate.go 70.00% 5 Missing and 1 partial ⚠️
pkg/services/control/server/list_objects.go 0.00% 6 Missing ⚠️
cmd/neofs-lens/internal/storage/list.go 0.00% 5 Missing ⚠️
pkg/local_object_storage/engine/range.go 80.00% 4 Missing and 1 partial ⚠️
pkg/local_object_storage/engine/head.go 77.77% 3 Missing and 1 partial ⚠️
pkg/local_object_storage/engine/writecache.go 0.00% 4 Missing ⚠️
cmd/neofs-node/object.go 0.00% 3 Missing ⚠️
... and 13 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3008      +/-   ##
==========================================
- Coverage   23.06%   23.01%   -0.06%     
==========================================
  Files         790      790              
  Lines       58709    58500     -209     
==========================================
- Hits        13542    13461      -81     
+ Misses      44285    44161     -124     
+ Partials      882      878       -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@carpawell carpawell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to see tests before the merge.

inhumePrm.WithForceRemoval()

_, err := ls.Inhume(inhumePrm)
err := ls.Delete(addr)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now, there is more than one thing that can delete objects, not sure it is expected. GC can be controlled, GC can be tuned or even stopped to prevent unnecessary disk load (#2992) but with this PR there is also a policer that does things on its own

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It did the same thing previously. The only difference is force, that's fa47327.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i meant deletion through the garbage bucket is more predictable to me. looking at this change makes me think we do not need the garbage bucket at all then

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic is the same, at least.

@@ -20,6 +20,10 @@ func (e *StorageEngine) ContainerSize(cnr cid.ID) (uint64, error) {
err error
size uint64
)
if e.metrics != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so this changed the game: it is not a "load hardware" metric but a logic one now? at least, i would add this to CHANGELOG

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We completely missed invocations previously. For a normal engine this doesn't change anything. For blocked (closed? I've not found any way to block, really) it'd at least count invocations (time is meaningless in this case).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

closed? I've not found any way to block, really

should be dropped then or try to write code keeping in mind that it can be blocked after some future feature

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was itching for me as well, but I've kept it for now.

@roman-khimov
Copy link
Member Author

Copy link
Member

@carpawell carpawell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@roman-khimov
Copy link
Member Author

Green.

@roman-khimov roman-khimov merged commit 6ea74d2 into master Nov 12, 2024
22 of 24 checks passed
@roman-khimov roman-khimov deleted the simplify-engine-interface branch November 12, 2024 17:39
roman-khimov added a commit that referenced this pull request Nov 13, 2024
This continues #3008 with internal refactoring that is mostly about
making code simpler and dropping useless code.
carpawell added a commit that referenced this pull request Nov 25, 2024
Follow #3001, #3008. With some additions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Improving existing functionality I4 No visible changes S3 Minimally significant U4 Nothing urgent
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants