EPIC: FIM Sys Admin Tasks (and a few related FIM tasks)

[RobHanna-NOAA]

This is a rollup of a large number of task recently recorded in our GIT repo that are more sys admin type tasks. While some of them are actual tasks that affect the FIM code base, they are not related to HAND outputs but environment tools to create HAND outputs. Many of these are also based in new OWP requirements, AWS fixes and new requirements, and various other sys admin tasks.

These are in no particular order and priority has been changing on them quickly. Not all tasks have a card associated with them but most do. Many have had at least some work into them already with significant notes and discoveries, but those notes are very complex and long and hard to put in git cards, especially due to security issues. Rob has notes on all of these and we might find a way to move some of those notes to google docs or something over time.

1) 1371: [8pt] Address AWS Security Notices (Critical and High status)

• priority very high. Need by Dec 31, but Matt and some other folks can help to some degree
• partially WIP
• Update: 2024/12/30: Done.

2) 1363: EPIC ; Rebuild / Upgraded AWS EC2's
- Part of this is priority of very high as one of the key problems is Ubuntu OS update and possible stability

• This also includes the segmentation fault issue (see item below)
• Also includes the QGIS / Parquet problem.
• various other issue but not as urgent

3) 1376 [21pt] Segmentation Fault

• priority very, very high
• already WIP by Rob and Matt
• Might be related or fixable via a pending rebuild / or upgrade or replacement of EC2's (see above)

4) 1335 Figure out the permissions for non-root users

• priority high
• Related to next topic of docker to podman (1375)

5) 1375; [21 pt (more)] Move from Docker to Podman

• priority high
• Needed by ITSG to allow them to replace the two OWP servers and it is overdue
• See number 6 below which might be related.

6) 1373 Change Docker builds of Dockerfile to docker compose

• priority unknown but might need to be high depending on the Podman item above (1375)
• Update: Dec 31, 2024: Matt validated using docker.dev that it will build against podman against docker build. Docker compose not needed at this time. Deferring card to backlog, log priority

7) 1364 Get AWS Stuff into Terraform

• priority low but currently doing manual backups so there is a risk element here.

8) 1365 [21pt] Rebuild the process of running AWS processing runs

• priority low but could have high value in light of the next one down, item 9, 1366: Develop script for developers to kick of step function runs.
• Also related to item 10 below AWS Step Function notification system.

9) 1366 [13pt] Build Script that Developers can kick off their own UAT runs against AWS Step Functions

• priority medium but high value
• See item 8 above (rebuild process of AWS processing runs), but not manditory.
• Also see item 10 below (AWS Step Function notification system), which is also optional, but for the average non sys admin to run this tool, will require some training, docs and a fair bit of babysitting of the run.
• At a min, it has a lot of value for us sys admin people.

10) 1367 [21pt] AWS Step Function notification system

• priority low
• Without this, item 8 can be much more complex and harder to use for average developers.

11) 1368 Add Alpha Testing to CI / CD (Step Functions)

• priority low
• A prerequisite which is a QA tool and already WIP by Matt 1269 QA Tool outline

12) 1370 Look into OverClocking for some FIM pipeline py scripts that have multi-processing

• priority low but value can be high. LOE can be low to moderate
• has cost savings impact and can have major performance impacts on various tools such as post_processing tools

13) 1372: [13pt] Address AWS Security Notices (medium and low status)

• priority medium. Need by end of Feb 2025 (ish). Matt and some other folks can help to some degree.
• Also need a process to monitor the security logs potentially

14) 1374 Look into S3 storage types (infrequent versus active versus xxxx)

• priority low but impact may/may not be high as it is related to cost saving.
• HydroVIS Gamma staff an help guilde this easily.
• LOE is likely very low but unknown.

15) 1369 [8pt] GitHub Linting warning

• priority low

16) Added Dec 16/24- 1380 Get docker build running again on Mac's and MS Windows.

• priority medium
• Problem related to GDAL and python versions on the host file.
• Fix in and is in testing now.