Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add schema entities and mappings to process set node operational status protobuf message #19

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

zeeshanakram3
Copy link
Collaborator

@kdembler kdembler self-assigned this Mar 1, 2024

type DistributionNodeOperationalStatusSetEvent {
"Distribution bucket operator"
bucketOperator: DistributionBucketOperator
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this field should be required?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed

@@ -60,6 +67,9 @@ export async function processStorageOperatorMetadata(
if (isSet(metadataUpdate.location)) {
processNodeLocationMetadata(operatorMetadata, metadataUpdate.location)
}
if (isSet(metadataUpdate.operationalStatus)) {
processNodeOperationalStatusMetadata('worker', undefined, metadataUpdate.operationalStatus)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we need to pass the current status as the second argument so worker is not able to override the lead?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Addressed

@@ -121,6 +203,9 @@ export async function processDistributionOperatorMetadata(
if (isSet(metadataUpdate.location)) {
processNodeLocationMetadata(operatorMetadata, metadataUpdate.location)
}
if (isSet(metadataUpdate.operationalStatus)) {
processNodeOperationalStatusMetadata('worker', undefined, metadataUpdate.operationalStatus)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed

.getById(`${workingGroup}-${meta.workerId}`)

if (!maybeWorker) {
return invalidMetadata(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We return invalidMetadata here but processNodeOperationalStatusMetadata in mappings/storage/metadata.ts only calls it, should we be consistent about it?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the return is just to break the function, alternatively we could just do

invalidMetadata()
return 

`The storage bucket ${bucketId} is not active`
)
// If the actor is a worker, check if the worker is the operator of the storage bucket
} else if (actor && storageBucket.operatorStatus.workerId !== actor.runtimeId) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, this check will ensure that regular worker can only update status of their own bucket and not somebody's elses. Should we also add a check for when the lead is the actor that ensures the meta.bucketId and meta.workerId matches storageBucket.operatorStatus.workerId?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, there is no validation currently to check meta.workerId = bucket.operatorStatus.workerId. So the drawback of the lack of this validation is that in the case of lead remark the operation would be performed even though meta.bucketId and meta.workerId do not match for any bucket, so, mappings are just looking for any valid meta.bucketId and set the status on that, (Simply ignoring what is the value of meta.workerId). And this is fine for the Lead context I think.

Comment on lines +269 to +273
const metadataEntity =
(await overlay.getRepository(StorageBucketOperatorMetadata).getById(bucketId)) ||
overlay
.getRepository(StorageBucketOperatorMetadata)
.new({ id: bucketId, storageBucketId: bucketId })
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to confirm, .new() call is sync and doesn't need await?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, it just creates in-memory objects, which are scheduled to be saved on DB later.

const operationalStatusSetEvent = new StorageNodeOperationalStatusSetEvent({
...genericEventFields(overlay, block, indexInBlock, extrinsicHash),
storageBucket: storageBucket.id,
operationalStatus: metadataEntity.nodeOperationalStatus || undefined,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we are emitting metadataEntity.nodeOperationalStatus, will this be already updated value after call to processNodeOperationalStatusMetadata?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will be set on the line.

metadataEntity.nodeOperationalStatus = processNodeOperationalStatusMetadata(

But I just noticed that this StorageNodeOperationalStatusSetEvent evet is being emitted regardless of the outcome of the processNodeOperationalStatusMetadata.

So, I have changed the implementation only to emit the event when the status changes.

SetNodeOperationalStatus,
`The distribution bucket operator ${distributionOperatorId} does not exist`
)
} else if (actor && operator.workerId !== actor.runtimeId) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question about the condition as above for storage

@zeeshanakram3 zeeshanakram3 force-pushed the process_set_node_operational_status_protobuf_message branch from d9716de to 692abf5 Compare March 27, 2024 03:25
@kdembler
Copy link
Member

kdembler commented Mar 27, 2024

@zeeshanakram3 build fails nvm, found Joystream/joystream#4793 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Review
Development

Successfully merging this pull request may close these issues.

2 participants