Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adding script to auto restart task #6072

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
9c76ae8
adding script to auto restart task
pvisweswar Aug 1, 2023
915f345
correcting the syntax
pvisweswar Aug 1, 2023
e3a7508
changes in the script
pvisweswar Aug 8, 2023
8e31dc4
updating autoreboot script
pvisweswar Sep 5, 2023
5569179
Delete uptime.sh.j2 file
pvisweswar Sep 5, 2023
9e080a5
updating staging maintainance_window
pvisweswar Sep 5, 2023
a79292c
updating the script
pvisweswar Sep 5, 2023
f1b78b3
Update autoreboot.sh.j2
pvisweswar Sep 5, 2023
1f5dc67
Update public.yml
pvisweswar Sep 6, 2023
6f32e9e
fixing the script
pvisweswar Sep 6, 2023
8607867
Update public.yml by removing spaces after equal to
pvisweswar Sep 6, 2023
3ff3bd4
Update main.yml with 'when' condition
pvisweswar Sep 6, 2023
827b4bb
change in main.yml file for when condition
pvisweswar Sep 6, 2023
44fe6ae
Update main.yml file
pvisweswar Sep 6, 2023
8b08da5
Update public.yml file
pvisweswar Sep 7, 2023
cb5321a
modifying the command in the script
pvisweswar Sep 13, 2023
efe717e
changing conditions in the script
pvisweswar Sep 13, 2023
7098d3a
adding script to auto restart task
pvisweswar Aug 1, 2023
101142c
correcting the syntax
pvisweswar Aug 1, 2023
3cebb69
changes in the script
pvisweswar Aug 8, 2023
93f944a
updating autoreboot script
pvisweswar Sep 5, 2023
efa7265
Delete uptime.sh.j2 file
pvisweswar Sep 5, 2023
56eed59
updating staging maintainance_window
pvisweswar Sep 5, 2023
c47ce35
updating the script
pvisweswar Sep 5, 2023
bbbbdad
Update autoreboot.sh.j2
pvisweswar Sep 5, 2023
5e9e99c
Update public.yml
pvisweswar Sep 6, 2023
1a5466b
fixing the script
pvisweswar Sep 6, 2023
4df6cb0
Update public.yml by removing spaces after equal to
pvisweswar Sep 6, 2023
ea35a6b
Update main.yml with 'when' condition
pvisweswar Sep 6, 2023
7747219
change in main.yml file for when condition
pvisweswar Sep 6, 2023
b86a6ee
Update main.yml file
pvisweswar Sep 6, 2023
7079ebf
Update public.yml file
pvisweswar Sep 7, 2023
f83fec2
modifying the command in the script
pvisweswar Sep 13, 2023
367eac7
changing conditions in the script
pvisweswar Sep 13, 2023
5af5861
Merge remote-tracking branch 'dimagi/master' into SAAS-14342/autorest…
pvisweswar Sep 20, 2023
7743f5b
Merge branch 'SAAS-14342/autorestart_of_ec2_servers' of https://githu…
pvisweswar Sep 20, 2023
82b0aaa
updated the crontab to trigger the script on Tuesday, Wednesday, and …
pvisweswar Sep 22, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions environments/staging/public.yml
Original file line number Diff line number Diff line change
Expand Up @@ -230,3 +230,9 @@ allow_aws_ssm_proxy: true
release_bucket: dimagi-commcare-staging-release

formplayer_java_version: "{{ java_17_bin_path }}/java"

# formplayer & web_worker

maintenance_start_time: "06:00"

maintenance_end_time: "08:00"
23 changes: 23 additions & 0 deletions src/commcare_cloud/ansible/roles/devops_scripts/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,29 @@
mode: "u=rwx,g=rwx,o=r"
when: "'postgresql' in group_names or 'citusdb' in group_names or 'pg_standby' in group_names"

- name: Copy autoreboot.sh script to /usr/local/sbin
template:
src: autoreboot.sh.j2
dest: /usr/local/sbin/autoreboot.sh
owner: ansible
group: ansible
mode: "u=rwx,g=rwx,o=r"
when: "'formplayer' in group_names or 'webworkers' in group_names"

- name: Creates a cron to check the autoreboot details
become: yes
cron:
name: Run autoreboot.sh at minute 0 and 30 on Tuesday, Wednesday, and Thursday
minute: "0,30"
hour: "*"
day: "*"
month: "*"
weekday: "2-4" # Tuesday (2) through Thursday (4)
user: root
job: "/usr/local/sbin/autoreboot.sh"
cron_file: autoreboot_sh
when: maintenance_start_time is defined and ("'formplayer' in group_names or 'webworkers' in group_names")

- name: Copy intruder.sh script to /usr/local/sbin
copy:
src: intruder.sh
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#!/bin/bash

current_availability_zone=$(ec2metadata --availability-zone | awk -F'-' '{print $3}' | cut -c2-)
maintenance_start_time={{ maintenance_start_time }}
maintenance_end_time={{ maintenance_end_time }}
current_time=$(date +%R)
uptime_days=$(uptime | awk '/days?/ {print $3; next}; {print 0}')

if [ $uptime_days -ge 90 ] && [[ $current_time > $maintenance_start_time ]] && [[ $current_time < $maintenance_end_time ]]; then
if [ "$current_availability_zone" == "a" ]; then
logger '[Maintenance] Rebooting server as uptime is greater than or equal to ninety days in az:A'
echo "Starting reboot"
shutdown -r +1
elif [ "$current_availability_zone" == "b" ]; then
logger '[Maintenance] Rebooting server as uptime is greater than or equal to ninety days in az:B'
echo "Starting reboot"
shutdown -r +30
elif [ "$current_availability_zone" == "c" ]; then
logger '[Maintenance] Rebooting server as uptime is greater than or equal to ninety days in az:C'
echo "Starting reboot"
shutdown -r +60
fi
fi