Rules not triggered until timepart of --start and/or --end is changed

[litsegaard]

I'm still struggling to get a smooth experience running EA on historical data (I know it's not meant for that but...). The process which seems to almost work for our use case is

1. Re-index "historical" data gathered two months ago replacing the date with today's date but with time intact
2. Run EA as a docker container using today's date and an arbitrary selected time-span

The command I use for interactively invoke EA is:

docker run --rm -it --net elkprodioc-kursse_elk \
  -e NO_PROXY=elasticsearch,<HOST_IP> \
  -v /home/vmadmin/compose-templates/tls/certs/ca/:/opt/elastalert/certs/ \
  -v $(pwd)/config/elastalert.yaml:/opt/elastalert/config.yaml \
  -v $(pwd)/rules:/opt/elastalert/rules \
  jertel/elastalert2:2.15.0 --start "$(date +'%Y'-'%m'-'%d')T07:00:00" --end "$(date +'%Y'-'%m'-'%d')T18:00:00" --verbose

This seems to work, however, when I've re-indexed the data, the run operation does not trigger any alarms until I manually tweak one of the time parts in --start or --end. If I do that and invoke the command the rule(s) get triggered and alarms are sent. Why is this?

[jertel]

ElastAlert 2 records the previous run time for a rule so that the next run doesn't re-query the same records. So if your newly re-indexed data is occupying the same time frames as a previous rule run then that would explain it. You stated that you are tweaking the start/end times but without specific examples and their corresponding logs this is the best explanation I can give you.