Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IA-5048] init script documentation #4778

Merged
merged 13 commits into from
Sep 20, 2024
Merged

Conversation

LizBaldo
Copy link
Collaborator

@LizBaldo LizBaldo commented Sep 4, 2024

Jira ticket: https://broadworkbench.atlassian.net/browse/IA-5048

Summary of changes

What

  • Remove unused legacy code and document the many complexities of our init scripts

Why

  • We had a couple of PROD incident related to some legacy code in the init script, and this is an attempt to better document and understand what these scripts are doing

Testing these changes

What to test

Who tested and where

  • This change is covered by automated tests
    • NB: Rerun automation tests on this PR by commenting jenkins retest or jenkins multi-test.
  • I validated this change
  • Primary reviewer validated this change
  • I validated this change in the dev environment

http/src/main/resources/init-resources/gce-init.sh Outdated Show resolved Hide resolved
http/src/main/resources/init-resources/gce-init.sh Outdated Show resolved Hide resolved
http/src/main/resources/init-resources/gce-init.sh Outdated Show resolved Hide resolved
http/src/main/resources/init-resources/gce-init.sh Outdated Show resolved Hide resolved
http/src/main/resources/init-resources/startup.sh Outdated Show resolved Hide resolved
http/src/main/resources/init-resources/startup.sh Outdated Show resolved Hide resolved
@@ -164,18 +253,6 @@ END
docker restart $JUPYTER_SERVER_NAME
docker restart $WELDER_SERVER_NAME

# This line is only for migration (1/26/2022). Say you have an existing runtime where jupyter container's PD is mapped at $HOME/notebooks,
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should simplify and not support these old runtimes anymore

Copy link

codecov bot commented Sep 4, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.78%. Comparing base (629bb69) to head (6e8781c).
Report is 1 commits behind head on develop.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff            @@
##           develop    #4778   +/-   ##
========================================
  Coverage    74.78%   74.78%           
========================================
  Files          165      165           
  Lines        14948    14948           
  Branches      1178     1178           
========================================
  Hits         11179    11179           
  Misses        3769     3769           
Flag Coverage Δ
74.78% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 629bb69...6e8781c. Read the comment docs.

@@ -376,7 +400,9 @@ if [ ! -z "$JUPYTER_DOCKER_IMAGE" ] ; then
mkdir -p ${WORK_DIRECTORY}/packages
chmod a+rwx ${WORK_DIRECTORY}/packages

# TODO: update this if we upgrade python version
# Install everything after having mounted the empty PD
# This should not be needed anymore if the jupyter home is a directory of the PD mount point
Copy link
Collaborator

@rtitle rtitle Sep 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 this is a strong candidate for deletion, I can't imagine we still support runtimes where the home directory is not /home/jupyter.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I mean here is that if we do not mount the PD in the home.directory then we can install these in the container directly

Copy link
Collaborator

@rtitle rtitle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this! I got through gce-init.sh, will pick up the rest later.

@rtitle
Copy link
Collaborator

rtitle commented Sep 5, 2024

Made it, overall looks great. We should make sure to do some testing, especially around pause/resume which has more bash restructuring. Looks like some leo-swat tests failed, not sure why.

@LizBaldo
Copy link
Collaborator Author

LizBaldo commented Sep 5, 2024

Thanks a bunch for the review, let me do a second pass and address your comments and I'll dig into the testing part next.

@LizBaldo
Copy link
Collaborator Author

LizBaldo commented Sep 6, 2024

@rtitle and @lucymcnatt I just tested creating, pausing, resuming, deleting with and without the PD for both a GCE and dataproc instance on my BEE and everything worked as expected 👌

@LizBaldo LizBaldo requested a review from rtitle September 6, 2024 18:52
Copy link
Collaborator

@rtitle rtitle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

Copy link
Collaborator

@lucymcnatt lucymcnatt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much cleaner now + clear comments, thanks for doing this work!

@LizBaldo LizBaldo merged commit 10af4b4 into develop Sep 20, 2024
23 checks passed
@LizBaldo LizBaldo deleted the IA-5048-init-script-documentation branch September 20, 2024 18:33
jgainerdewar added a commit that referenced this pull request Oct 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants