Check on Welsch cron jobs

[Chris-Schnaufer]

Make sure the cron jobs are running correctly

[Chris-Schnaufer]

There are two problems found with the cron jobs:

1. the /home/dlebauer/neon-datasets/models/simple/ets_forecast.R script is erroring out

...
Registered S3 method overwritten by 'quantmod':
  method            from
  as.zoo.data.frame zoo 
Error: Problem with `mutate()` input `time`.
✖ Input `time` can't be recycled to size 3240.
ℹ Input `time` is `rep(now + days(1:180), 8)`.
ℹ Input `time` must be size 3240 or 1, not 1440.
Backtrace:
     █
  1. ├─`%>%`(...)
  2. │ ├─base::withVisible(eval(quote(`_fseq`(`_lhs`)), env, env))
  3. │ └─base::eval(quote(`_fseq`(`_lhs`)), env, env)
  4. │   └─base::eval(quote(`_fseq`(`_lhs`)), env, env)
  5. │     └─`_fseq`(`_lhs`)
  6. │       └─magrittr::freduce(value, `_function_list`)
  7. │         └─function_list[[i]](value)
  8. │           ├─dplyr::mutate(., time = rep(now + days(1:180), 8))
  9. │           └─dplyr:::mutate.data.frame(., time = rep(now + days(1:180), 8))
 10. │             └─dplyr:::mutate_cols(.data, ...)
 11. │               ├─base::withCallingHandlers(...)
 12. │               └─mask$eval_all_mutate(dots[[i]])
 13. ├─dplyr:::abort_glue(character(0), list(x_size = 1440L), "dplyr:::mutate_incompatible_size")
 14. │ ├─rlang::exec(abort, class = class, !!!.envir)
 15. │ └─(function (message = NULL, class = NULL, ..., trace = NULL, parent = NULL, ...
 16. │   └─rlang:::signal_abort(cnd)
 17. │     └─base::signalCondition(cnd)
 18. └─(function (e) ...
Execution halted

2. the BETYdb database associated with REMOTE_SERVERS=1 is no longer available.

[Chris-Schnaufer]

@dlebauer Any thoughts on what to do with these two findings?

[dlebauer]

@Chris-Schnaufer

• the cron job associated with /home/dlebauer/neon-datasets/models/simple/ets_forecast.R can be deleted. It is now running via Github actions
• for BETYdb sync, what error are you getting. Is it related to Extremely long sync with some DBs (e.g. BU) PecanProject/bety#727 ? Do you know where it is looking for the dump from that db (I think it should be https://psql-pecan.bu.edu/sync/dump/bety.tar.gz)?

Once the db is syncing again, this query should return >1:

select count(*) from dbfiles 
   where created_at > '2022-08-01' 
   and id between 1000000000 and 1001000000; 

[Chris-Schnaufer]

Need to wait to update the database to resolve the update for REMOTE_SERVER=1 issue (PeCAn is being used the week of Octonber 10

[dlebauer]

Also todo:

• update cron jobs to use the most recent version of load.bety.sh, which was moved to the bety repository two years ago: https://github.com/PecanProject/bety/blob/develop/script/load.bety.sh
  • is it worth starting with a fresh wget https://raw.githubusercontent.com/PecanProject/bety/develop/script/load.bety.sh every day?
  • I suspect this is why the REMOTE_SERVER=1 is failing. The location of dumps are now stored in the machines table
• need to make sure that https://welsch.cyverse.org/pecan/dump/bety.tar.gz is available for others to sync; I can't access it and /root/cron-output/irsync-backups says Level 0: collection is unknown (related?)
• check other files in /root/cron-output e.g. should backup.bety and dump.bety be empty?

[dlebauer]

Reopening to confirm that:

• load.bety.sh, dump.bety.sh and backup.bety.sh scripts use the updated versions from https://github.com/PecanProject/bety/tree/develop/script (crontab seems to indicate it is still using the old versions)
• https://welsch.cyverse.org/pecan/dump/bety.tar.gz is available so that other databases can sync
• updated docs on db migration if what you did is different

[Chris-Schnaufer]

Closing this issue again since the original task has been completed. Moving these comments to issue #1098 since the changes can't be done until the docker images are brought up to date

• these steps should be part of that upgrade due to the dependencies (current code is already staged on Welsch)
• the original task of this issue has been completed - therefore these comments should be in another issue