run_every time shifting behaviour

[CaGix22]

Hello everyone!

I've noticed that rules not using the limit_execution parameter are shifting slightly each hour (by seconds), and I'm trying to understand why. I Imagine it is expected behavior, but I would like to know more about it. Is it something I can control with another parameter?

My rule has the use_count_query: true parameter, and if I understood correctly, since buffer_time is disregarded by it, my understanding is that the start and end query times are determined by run_every.

global config: run_every: hours: 1

Here are some of the logs in the elastalert_status index illustrating the issue:

starttime column                                   endtime column
May 30, 2024 @ 21:16:09.335             May 30, 2024 @ 22:16:13.849

May 30, 2024 @ 20:16:07.082             May 30, 2024 @ 21:16:09.335

May 30, 2024 @ 19:16:03.850             May 30, 2024 @ 20:16:07.082

May 30, 2024 @ 18:16:00.622             May 30, 2024 @ 19:16:03.850

May 30, 2024 @ 17:16:00.332             May 30, 2024 @ 18:16:00.622

May 30, 2024 @ 16:15:58.464             May 30, 2024 @ 17:16:00.332

May 30, 2024 @ 15:15:56.880             May 30, 2024 @ 16:15:58.464

May 30, 2024 @ 14:15:54.640             May 30, 2024 @ 15:15:56.880

May 30, 2024 @ 13:15:50.925             May 30, 2024 @ 14:15:54.640

May 30, 2024 @ 12:15:48.620             May 30, 2024 @ 13:15:50.925

May 30, 2024 @ 11:15:44.867             May 30, 2024 @ 12:15:48.620

May 30, 2024 @ 10:15:41.553             May 30, 2024 @ 11:15:44.867

May 30, 2024 @ 09:15:40.259             May 30, 2024 @ 10:15:41.553

May 30, 2024 @ 08:15:39.726             May 30, 2024 @ 09:15:40.259

May 30, 2024 @ 07:15:39.119             May 30, 2024 @ 08:15:39.726

May 30, 2024 @ 06:15:37.404             May 30, 2024 @ 07:15:39.119

Thanks in advance!!

[jertel]

Yes, it's expected behavior. ElastAlert 2 randomly adds up to 5 seconds of jitter to the job scheduler, to prevent multiple rules from triggering at the exact same millisecond. This is not currently configurable.

[CaGix22]

Thanks Jertel for the fast reply! This info is really helpful.