Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](job)Fix millisecond offset issue in time window scheduling trigger time calculation #45176

Merged
merged 1 commit into from
Dec 12, 2024

Conversation

CalvinKirs
Copy link
Member

Abstract:

In the current time window scheduling logic, the calculation of trigger times was not strictly aligned to the second level, which could lead to millisecond offsets. This offset caused issues such as consecutive trigger times at 14:56:59 and 14:57:00, disrupting the correctness of the scheduling.

This PR optimizes the calculation of trigger times to ensure that time points are strictly aligned to the second level, preventing the accumulation of millisecond errors.

Issue Description:

Under a specified window (e.g., 14:50:00 to 14:59:00) and a fixed interval (e.g., every minute), the scheduler generated erroneous trigger times such as:

| 2024-12-04 14:56:59 |
| 2024-12-04 14:57:00 |
| 2024-12-04 14:57:59 |
| 2024-12-04 14:58:00 |

Cause:

The current firstTriggerTime and the loop calculation did not strictly align trigger times to the second level, resulting in erroneous trigger points due to floating-point or millisecond offset accumulation. The end condition for the time window was not aligned to the second level, which could lead to additional trigger times being included.

Fix:

Modification 1: Strictly align the trigger time to the second level.

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

…ger time calculation

### Abstract:
In the current time window scheduling logic, the calculation of trigger times was not strictly aligned to the second level, which could lead to millisecond offsets. This offset caused issues such as consecutive trigger times at 14:56:59 and 14:57:00, disrupting the correctness of the scheduling.

This PR optimizes the calculation of trigger times to ensure that time points are strictly aligned to the second level, preventing the accumulation of millisecond errors.

### Issue Description:

Under a specified window (e.g., 14:50:00 to 14:59:00) and a fixed interval (e.g., every minute), the scheduler generated erroneous trigger times such as:

```
| 2024-12-04 14:56:59 |
| 2024-12-04 14:57:00 |
| 2024-12-04 14:57:59 |
| 2024-12-04 14:58:00 |
```
#### Cause:
The current firstTriggerTime and the loop calculation did not strictly align trigger times to the second level, resulting in erroneous trigger points due to floating-point or millisecond offset accumulation. The end condition for the time window was not aligned to the second level, which could lead to additional trigger times being included.

### Fix:
Modification 1: Strictly align the trigger time to the second level.
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@CalvinKirs
Copy link
Member Author

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 9, 2024
Copy link
Contributor

github-actions bot commented Dec 9, 2024

PR approved by at least one committer and no changes requested.

Copy link
Contributor

github-actions bot commented Dec 9, 2024

PR approved by anyone and no changes requested.

Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@CalvinKirs CalvinKirs merged commit 13add3c into apache:master Dec 12, 2024
26 of 28 checks passed
@CalvinKirs CalvinKirs deleted the master-job-count branch December 12, 2024 08:42
github-actions bot pushed a commit that referenced this pull request Dec 12, 2024
…ger time calculation (#45176)

### Abstract:
In the current time window scheduling logic, the calculation of trigger
times was not strictly aligned to the second level, which could lead to
millisecond offsets. This offset caused issues such as consecutive
trigger times at 14:56:59 and 14:57:00, disrupting the correctness of
the scheduling.

This PR optimizes the calculation of trigger times to ensure that time
points are strictly aligned to the second level, preventing the
accumulation of millisecond errors.

### Issue Description:

Under a specified window (e.g., 14:50:00 to 14:59:00) and a fixed
interval (e.g., every minute), the scheduler generated erroneous trigger
times such as:

```
| 2024-12-04 14:56:59 |
| 2024-12-04 14:57:00 |
| 2024-12-04 14:57:59 |
| 2024-12-04 14:58:00 |
```
#### Cause:
The current firstTriggerTime and the loop calculation did not strictly
align trigger times to the second level, resulting in erroneous trigger
points due to floating-point or millisecond offset accumulation. The end
condition for the time window was not aligned to the second level, which
could lead to additional trigger times being included.

### Fix:
Modification 1: Strictly align the trigger time to the second level.
github-actions bot pushed a commit that referenced this pull request Dec 12, 2024
…ger time calculation (#45176)

### Abstract:
In the current time window scheduling logic, the calculation of trigger
times was not strictly aligned to the second level, which could lead to
millisecond offsets. This offset caused issues such as consecutive
trigger times at 14:56:59 and 14:57:00, disrupting the correctness of
the scheduling.

This PR optimizes the calculation of trigger times to ensure that time
points are strictly aligned to the second level, preventing the
accumulation of millisecond errors.

### Issue Description:

Under a specified window (e.g., 14:50:00 to 14:59:00) and a fixed
interval (e.g., every minute), the scheduler generated erroneous trigger
times such as:

```
| 2024-12-04 14:56:59 |
| 2024-12-04 14:57:00 |
| 2024-12-04 14:57:59 |
| 2024-12-04 14:58:00 |
```
#### Cause:
The current firstTriggerTime and the loop calculation did not strictly
align trigger times to the second level, resulting in erroneous trigger
points due to floating-point or millisecond offset accumulation. The end
condition for the time window was not aligned to the second level, which
could lead to additional trigger times being included.

### Fix:
Modification 1: Strictly align the trigger time to the second level.
CalvinKirs added a commit that referenced this pull request Dec 12, 2024
…eduling trigger time calculation #45176 (#45353)

Cherry-picked from #45176

Co-authored-by: Calvin Kirs <[email protected]>
morningman pushed a commit that referenced this pull request Dec 20, 2024
…eduling trigger time calculation #45176 (#45352)

Cherry-picked from #45176

Co-authored-by: Calvin Kirs <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.8-merged dev/3.0.4-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants