Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixes load job counter #1702

Merged
merged 2 commits into from
Aug 16, 2024
Merged

fixes load job counter #1702

merged 2 commits into from
Aug 16, 2024

Conversation

rudolfix
Copy link
Collaborator

Description

Load job counters were incorrectly increased (absolute value was used). This also will skip loading full package details on each completed job decreasing cpu usage on large packages

@rudolfix rudolfix self-assigned this Aug 15, 2024
Copy link

netlify bot commented Aug 15, 2024

Deploy Preview for dlt-hub-docs ready!

Name Link
🔨 Latest commit 1d54f8e
🔍 Latest deploy log https://app.netlify.com/sites/dlt-hub-docs/deploys/66be3d9a3f07550008e36fe7
😎 Deploy Preview https://deploy-preview-1702--dlt-hub-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@rudolfix rudolfix added the bug Something isn't working label Aug 15, 2024
@@ -170,6 +170,7 @@ def update(
total=total,
)
self.messages[counter_key] = None
self.last_log_time = None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Load job counters were incorrectly increased (absolute value was used).

How does adding this line solve the issue of incorrectly increasing the counter?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually main issue was solved by deleting this:

# update load package info if any jobs where finalized
                if finalized_jobs:
                    self.update_loadpackage_info(load_id)

which were adding absolute number of completed packages on each loop.

the line above makes sure that we show counter with 0 jobs at the moment the load starts.

@rudolfix rudolfix merged commit 5bf78ae into devel Aug 16, 2024
56 checks passed
@rudolfix rudolfix deleted the fix/fixes-load-job-counter branch August 16, 2024 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

2 participants