Creating-a-Singularity-Container-for-a-Conda-Environment.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c5367b0d",
   "metadata": {},
   "source": [
    "![<CW3E Logo>](https://cw3e.ucsd.edu/images/cw3e_logo_files/wetransfer-b4ff74/CW3E%20Final%20Logo%20Suite/4-Horizontal-Acronym/Digital/PNG/CW3E-Logo-Horizontal-Acronym-FullColor.png \"Center for Western Weather and Water Extremes Logo\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ff1f4bf",
   "metadata": {},
   "source": [
    "# Creating a Singularity Container for a Conda Environment\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c75d6cab",
   "metadata": {},
   "source": [
    "## Overview\n",
    "---\n",
    "\n",
    "Singularity containers are essential for your code as they create complete, uniform software environments applicable for every operating system. The purpose of generating a Singularity container is to allow us to program within a consistent environment. Containerization creates reproducibility in computing workflows.\n",
    "\n",
    "This notebook will go over the following:\n",
    "1. Singularity definition files and their necessary components. \n",
    "2. Building, running, and transferring a Singularity container.\n",
    "3. Utilizing containers for both Bash and Python scripts.\n",
    "4. Using a container to launch a Jupyter notebook server."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af9c8c13",
   "metadata": {},
   "source": [
    "## Prerequisites\n",
    "---\n",
    "\n",
    "| **Concept** | **Importance** | **Notes** |\n",
    "|:-----------:|:--------------:|:---------:|\n",
    "| Bash | Necessary | Understand on an intermediate level |\n",
    "| Python | Necessary | Understand on an intermediate level |\n",
    "| Singularity | Helpful | Understanding the basics will help with this workflow, but not necessary |\n",
    "\n",
    "* **Time to Learn:** 2 hours est."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ffa0285",
   "metadata": {},
   "source": [
    "## Creating a Singularity Definition/Recipe File\n",
    "---\n",
    "\n",
    "Definition files (aka container recipes) are the blueprint for creating a custom Singularity container. These type of files consist of the **header** and the **sections**. These parts will be briefly discussed below. For more information on definition files, refer to the [Singularity Documentation](https://docs.sylabs.io/guides/latest/user-guide/definition_files.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef20562a",
   "metadata": {},
   "source": [
    "### Generating a Definition File\n",
    "\n",
    "---\n",
    "\n",
    "The following line of code can be run to create your own definition file. Make sure to have this definition file in a system that has the capability to build a container (i.e., skyriver or feather)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8f8df09",
   "metadata": {},
   "source": [
    "```\n",
    "vim Singularity.def\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "880f140f",
   "metadata": {},
   "source": [
    "> Usually, the definition file for a Singularity container is just named \"Singularity\". However, if you are making a container for a specific environment, make sure to name it appropriately.\n",
    ">> Ex: `Singularity_NetCDF.def` as a definition file name to build a NetCDF environment container."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c621e9e3",
   "metadata": {},
   "source": [
    "### Header\n",
    "\n",
    "---\n",
    "\n",
    "The header section of a definition file describes the core OS to build and configure within the container. The header can be comprised of several keywords, but for the purposes of our container, we will only need `Bootstrap` and `From`. An example header is shown and explained below."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca653ee7",
   "metadata": {},
   "source": [
    "```\n",
    "Bootstrap: docker\n",
    "From: centos:7.9.2009\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "197fb64d",
   "metadata": {},
   "source": [
    "1. **Bootstrap** - this keyword refers to what kind of base you want to use (in this case, we want to use Docker).\n",
    "2. **From** - this keyword refers to the named container/reference to layers you want to use (in this case, centos:7.9.2009).\n",
    "\n",
    "For more reference to the docker bootstrap agent, refer to the [Singularity appendix.](https://docs.sylabs.io/guides/latest/user-guide/appendix.html#build-docker-module)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3156f84",
   "metadata": {},
   "source": [
    "### Sections\n",
    "\n",
    "---\n",
    "\n",
    "The main content of a definition file is within the sections. Each section has its own function and provides different content and commands. The following sections were used to create the miniconda Singularity container (in this respective order)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae72b4b9",
   "metadata": {},
   "source": [
    "##### %labels\n",
    "\n",
    "This section is used to store metadata within the container. This section is often filled with information about the author and application. An example `%label` section is shown below."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d76cdf2b",
   "metadata": {},
   "source": [
    "```\n",
    "APPLICATION_NAME Miniconda - CentOS 7.9.2009\n",
    "    APPLICATION_URL https://cw3e.ucsd.edu\n",
    "    APPLICATION_VERSION 1.0\n",
    "\n",
    "    AUTHOR_NAME Patrick Mulrooney\n",
    "    AUTHOR_EMAIL pmulrooney@ucsd.edu\n",
    "\n",
    "    CO_AUTHOR_NAME Jozette Conti\n",
    "    CO_AUTHOR_EMAIL jlconti@ucsd.edu\n",
    "\n",
    "    LAST_UPDATED 2023.07.11\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "404f4027",
   "metadata": {},
   "source": [
    "##### %setup\n",
    "\n",
    "Any commands in this section are executed on the host system outside the container after the base OS has been installed. For the purposes of the conda container, leave this blank."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5652f669",
   "metadata": {},
   "source": [
    "```\n",
    "%setup\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "086a4bac",
   "metadata": {},
   "source": [
    "##### %environment\n",
    "\n",
    "This section allows you to define environmental variables set at runtime. If there are any variables needed during the build time, place them in the [`%post` section](#%post). During the building of the container, this section will be written to a file in the container’s metadata folder. This folder is then sourced during the runtime. For the purposes of the miniconda environment, the following environmental variable was set."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb1c1847",
   "metadata": {},
   "source": [
    "```\n",
    "%environment\n",
    "    PATH=/opt/conda/bin:$PATH\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c998c34e",
   "metadata": {},
   "source": [
    "##### %files\n",
    "\n",
    "This section is significant for our container. In this section, you can copy any necessary files into the container. For the purposes of our miniconda container, we need to generate a list of requirements for the conda environment in Comet. This can be achieved by running the following line in the base conda environment within Comet. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d66650d",
   "metadata": {},
   "source": [
    "```\n",
    "conda list --explicit\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9bb6765f",
   "metadata": {},
   "source": [
    "This list can then be copied and pasted into a text file named `requirements.txt`. This file can then be embedded into the container through the definition file, as shown below."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e34dc814",
   "metadata": {},
   "source": [
    "```\n",
    "%files\n",
    "\n",
    "    ./requirements.txt /\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d7daf37",
   "metadata": {},
   "source": [
    "Now the required conda packages in Comet are within our definition file.\n",
    "\n",
    "> If your code is using more than just the base conda environment (i.e. NetCDF or iPython) you will have to generate a new requirements file specific to that environment to implement into your container. (i.e. run `conda list --explicit` in the terminal again when in that specific environment)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c24bcb9",
   "metadata": {},
   "source": [
    "<a id=\"%post\"></a>\n",
    "##### %post\n",
    "\n",
    "The `%post` section is the main section of all the sections. Making directories and installing software/libraries occurs within this section. This section consists of commands that would be run if a root user was directly in the terminal. (i.e., this section are the commands needed to install your program successfully). \n",
    "\n",
    "* The first part of our `%post` section is as follows"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36679e8a",
   "metadata": {},
   "source": [
    "```\n",
    "%post -c /bin/bash\n",
    "\n",
    "    export CONDA_VERSION=py310_23.3.1-0\n",
    "    export SHA256SUM=aef279d6baea7f67940f16aad17ebe5f6aac97487c7c03466ff01f4819e5a651\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3853029",
   "metadata": {},
   "source": [
    "In these first few commands, we define the version and hash of the miniconda we are using. These variable values can be found in the [conda documentation](https://docs.conda.io/en/latest/miniconda_hashes.html).\n",
    "\n",
    "* The second part of the `%post` section is"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bcdcc5d2",
   "metadata": {},
   "source": [
    "```\n",
    "    echo \"=========== \"\n",
    "    yum install -y wget\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdbdc7ca",
   "metadata": {},
   "source": [
    "The purpose of this `echo` command (as well as the ones that follow) is to provide separation between chunks of code and update the user during the building of the container.\n",
    "\n",
    "`yum` is a package installer for RedHat (i.e., CentOS, which we are using). By running `yum install -y wget`, we install a command needed for the next chunk of code.\t \n",
    "> The `-y` flag tells the system to assume yes during the installation of wget\n",
    "\n",
    "* The next part of our `%post` section is"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8bd57d2",
   "metadata": {},
   "source": [
    "```\n",
    "    echo \"=========== wget conda & verify\"\n",
    "    wget --quiet -O miniconda.sh https://repo.anaconda.com/miniconda/Miniconda3-${CONDA_VERSION}-Linux-x86_64.sh && \\\n",
    "      echo \"${SHA256SUM}  miniconda.sh\" > miniconda.sha256 && \\\n",
    "      if ! sha256sum --strict -c miniconda.sha256; then exit 1; fi\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d40c46a",
   "metadata": {},
   "source": [
    "With our now installed wget, we can download the installation script for the conda version previously defined. \n",
    "- `--quiet` bypasses the output of wget\n",
    "- `-O miniconda.sh` saves the file with a desired file name which follows the flag (miniconda.sh) (i.e., changes the output file name)\n",
    "- The `&& \\` at the end of the lines of code lets you do something based on whether the previous command was completed successfully (will complete the following line if the previous one was successful).\n",
    "\n",
    "Once this downloads, the SHA256SUM we defined earlier + the newly installed file (`miniconda.sh`) will be placed into a new file named `miniconda.sha256`\n",
    "\n",
    "Then, using a control statement, we can check if the hash of the downloaded miniconda installer file matches what we have on record. If not, the build will be stopped.\n",
    "\n",
    "* The next `%post` section is"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e8ab1be",
   "metadata": {},
   "source": [
    "```\n",
    "   echo \"=========== install conda\"\n",
    "    mkdir -p /opt && \\\n",
    "      sh miniconda.sh -b -p /opt/conda\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e917d306",
   "metadata": {},
   "source": [
    "This chunk of code creates a directory/subdirectory for miniconda to download into. \n",
    "\n",
    "* The next `%post` section"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a41148c0",
   "metadata": {},
   "source": [
    "```\n",
    "   echo \"=========== add links\"\n",
    "    ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh && \\\n",
    "      echo \". /opt/conda/etc/profile.d/conda.sh\" >> ~/.bashrc && \\\n",
    "      echo \"conda activate base\" >> ~/.bashrc\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "132ae61e",
   "metadata": {},
   "source": [
    "This code block generates a symbolic link and adds commands to the bashrc file. These files will be initiated when the user logs in. \n",
    "\n",
    "> The line `echo \"conda activate base\" >> ~/.bashrc` remains unchanged if you are building a container for a non-base environment (i.e. NetCDF or iPython). \n",
    "\n",
    "* Next `%post` section"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "916f3795",
   "metadata": {},
   "source": [
    "```\n",
    "    echo \"=========== Cleanup \"\n",
    "    rm miniconda.sh miniconda.sha256 && \\\n",
    "      find /opt/conda/ -follow -type f -name '*.a' -delete && \\\n",
    "      find /opt/conda/ -follow -type f -name '*.js.map' -delete && \\\n",
    "      /opt/conda/bin/conda clean -afy\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5df10b66",
   "metadata": {},
   "source": [
    "Now that conda is installed, we can clean up our environment and clear any cache. To do this, we can remove the download files from earlier as well as any files ending in the extension `.a` or `.js.map`.\n",
    "\n",
    "* Next `%post` section"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d14341fe",
   "metadata": {},
   "source": [
    "```\n",
    "    echo \"=========== Init envrionment\"\n",
    "    . /opt/conda/etc/profile.d/conda.sh\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f6e7b60",
   "metadata": {},
   "source": [
    "This block of code initializes the conda environment by running the conda script.\n",
    "\n",
    "* Last `%post` section for our miniconda environment"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b72bd6d3",
   "metadata": {},
   "source": [
    "```\n",
    "    echo \"=========== Install from requirements.txt\"\n",
    "    /opt/conda/bin/conda install -y --file /requirements.txt\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6daced2",
   "metadata": {},
   "source": [
    "As the environment is already initialized, we have to download the requirements for the conda environment from Comet (i.e., the requirements file from earlier). After this, we are done with the `%post` section."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22a060b7",
   "metadata": {},
   "source": [
    "##### %runscript\n",
    "\n",
    "This section comprises of the commands you would use to run the now-installed program from the `%post` section. This section is simple for our definition file."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8494bf30",
   "metadata": {},
   "source": [
    "```\n",
    "%runscript\n",
    "\n",
    "    /bin/bash\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88e18a74",
   "metadata": {},
   "source": [
    "##### %test\n",
    "\n",
    "This is the last section of our definition file! This section is run at the end of the build process. The test section for our definition files is just"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6f8e426",
   "metadata": {},
   "source": [
    "```\n",
    "%test\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec7795ca",
   "metadata": {},
   "source": [
    "#### Finalized Definition File\n",
    "---\n",
    "\n",
    "Now that we have added all the necessary information, our final definition file should look like the following.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e394ed17",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Bootstrap: docker\r\n",
      "From: centos:7.9.2009\r\n",
      "\r\n",
      "%labels\r\n",
      "\r\n",
      "    APPLICATION_NAME Miniconda - CentOS 7.9.2009\r\n",
      "    APPLICATION_URL https://cw3e.ucsd.edu\r\n",
      "    APPLICATION_VERSION 1.0\r\n",
      "\r\n",
      "    AUTHOR_NAME Patrick Mulrooney\r\n",
      "    AUTHOR_EMAIL pmulrooney@ucsd.edu\r\n",
      "\r\n",
      "    CO_AUTHOR_NAME Jozette Conti\r\n",
      "    CO_AUTHOR_EMAIL jlconti@ucsd.edu\r\n",
      "\r\n",
      "    LAST_UPDATED 2023.07.11\r\n",
      "\r\n",
      "%setup\r\n",
      "\r\n",
      "%environment\r\n",
      "    PATH=/opt/conda/bin:$PATH\r\n",
      "\r\n",
      "%files\r\n",
      "\r\n",
      "    ./requirements.txt /\r\n",
      "\r\n",
      "%post -c /bin/bash\r\n",
      "\r\n",
      "    export CONDA_VERSION=py310_23.3.1-0\r\n",
      "    export SHA256SUM=aef279d6baea7f67940f16aad17ebe5f6aac97487c7c03466ff01f4819e5a651\r\n",
      "\r\n",
      "    echo \"=========== \"\r\n",
      "    yum install -y wget\r\n",
      "\r\n",
      "    echo \"=========== wget conda & verify\"\r\n",
      "    wget --quiet -O miniconda.sh https://repo.anaconda.com/miniconda/Miniconda3-${CONDA_VERSION}-Linux-x86_64.sh && \\\r\n",
      "      echo \"${SHA256SUM}  miniconda.sh\" > miniconda.sha256 && \\\r\n",
      "      if ! sha256sum --strict -c miniconda.sha256; then exit 1; fi\r\n",
      "\r\n",
      "    echo \"=========== install conda\"\r\n",
      "    mkdir -p /opt && \\\r\n",
      "      sh miniconda.sh -b -p /opt/conda\r\n",
      "\r\n",
      "    echo \"=========== add links\"\r\n",
      "    ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh && \\\r\n",
      "      echo \". /opt/conda/etc/profile.d/conda.sh\" >> ~/.bashrc && \\\r\n",
      "      echo \"conda activate base\" >> ~/.bashrc\r\n",
      "\r\n",
      "    echo \"=========== Cleanup \"\r\n",
      "    rm miniconda.sh miniconda.sha256 && \\\r\n",
      "      find /opt/conda/ -follow -type f -name '*.a' -delete && \\\r\n",
      "      find /opt/conda/ -follow -type f -name '*.js.map' -delete && \\\r\n",
      "      /opt/conda/bin/conda clean -afy\r\n",
      "\r\n",
      "    echo \"=========== Init envrionment\"\r\n",
      "    . /opt/conda/etc/profile.d/conda.sh\r\n",
      "\r\n",
      "    echo \"=========== Install from requirements.txt\"\r\n",
      "    /opt/conda/bin/conda install -y --file /requirements.txt\r\n",
      "\r\n",
      "%runscript\r\n",
      "\r\n",
      "    /bin/bash\r\n",
      "\r\n",
      "%test\r\n"
     ]
    }
   ],
   "source": [
    "cat Singularity.def"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a11bd54e",
   "metadata": {},
   "source": [
    "## Building a Singularity Container From a Definition File\n",
    "---\n",
    "\n",
    "Now that our definition file is complete, we can build our Singularity environment in feather/skyriver. This can be achieved by using the following syntax in the same directory as our definition file.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92ccf97c",
   "metadata": {},
   "source": [
    "```\n",
    "singularity build ContainerName.sif YourDefineFile.def\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a9da3cd3",
   "metadata": {},
   "source": [
    "If you have the correct permission, running this line of code will generate a Singularity image named `ContainerName.sif`\n",
    "\n",
    "For the miniconda environment I made, I ran the following line of code"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65fe00f0",
   "metadata": {},
   "source": [
    "```\n",
    "singularity build miniconda.sif Singularity.def\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "690ce351",
   "metadata": {},
   "source": [
    "This process will take a little bit to run, but the user will be prompted when the build is complete.\n",
    "\n",
    "> If you are a non-root user in skyriver/feather at this step, you will need to add the flag `--fakeroot`\n",
    ">> Hence, your line of code should be `singularity build --fakeroot ContainerName.sif YourDefineFile.def`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ceba307",
   "metadata": {},
   "source": [
    "## Running/Shelling into a Singularity Container\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a4aa38a",
   "metadata": {},
   "source": [
    "Now that our Singularity container is built, we can do a simple check by running the image or shelling into it.\n",
    "\n",
    "* To run the container, the following prompt was used"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7ac0096",
   "metadata": {},
   "source": [
    "```\n",
    "singularity run miniconda.sif\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f1fe249",
   "metadata": {},
   "source": [
    "* Similarly, the shell into the container, the following prompt was used"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45b016dc",
   "metadata": {},
   "source": [
    "```\n",
    "singularity shell miniconda.sif\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12eff619",
   "metadata": {},
   "source": [
    "> For both of these prompts of code, the username on the command line (i.e., `feather[~]$`) should change to `singularity>`, symbolizing that you are now within the container."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d8a39b8",
   "metadata": {},
   "source": [
    "## Transferring a Singularity Container To Comet\n",
    "\n",
    "---\n",
    "\n",
    "As Singularity containers cannot be built on Comet, we must transfer the container we made on a different system to Comet. We can run a rsync command (preferred over scp) to do this. The following syntax can be used for this command."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6073efb",
   "metadata": {},
   "source": [
    "```\n",
    "rsync -avh --progress /path/to/miniconda.sif <whoami>@comet.sdsc.edu:/cw3e/mead/path/to/where/you/want/miniconda.sif\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "21822259",
   "metadata": {},
   "source": [
    "For example, the line of code I ran for this was, "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f3df3df",
   "metadata": {},
   "source": [
    "```\n",
    "rsync -avh --progress /data/projects/containers/miniconda/miniconda.sif jlconti@comet.sdsc.edu:/cw3e/mead/projects/cwp106/scratch/jlconti/miniconda.sif\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6860246d",
   "metadata": {},
   "source": [
    "> This line of code is to be run in feather/skyriver. It then copies the Singularity image to Comet.\n",
    "\n",
    "* The `--progress` flag will show the user the process of the image transfer. Once completed, make sure to verify the image has, in fact, transferred to the desired directory in Comet."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ce2d578",
   "metadata": {},
   "source": [
    "<a id=\"verify\"></a>\n",
    "## Verifying Your Container\n",
    "\n",
    "---\n",
    "\n",
    "Once the miniconda container has been transferred, run the following line of code</a> with your desired path in Comet."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72898726",
   "metadata": {},
   "source": [
    "```\n",
    "singularity exec -e --bind /cw3e:/cw3e,/scratch:/scratch /cw3e/mead/projects/cwp106/path/to/miniconda.sif conda list\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6a82d36",
   "metadata": {},
   "source": [
    "For example, the line of code I ran was"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1de9505",
   "metadata": {},
   "source": [
    "```\n",
    "singularity exec -e --bind /cw3e:/cw3e,/scratch:/scratch /cw3e/mead/projects/cwp106/scratch/jlconti/miniconda.sif conda list\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f8fb668",
   "metadata": {},
   "source": [
    "The `-e` flag is a longer form of the `-cleanenv` flag. Including this flag helps make sure the container environment is clean of any inherited environment variables from the host system. Hence, for best practices, this flag should be included in any `singularity exec` command."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "04935838",
   "metadata": {},
   "source": [
    "> When using a container, it acts to replace your home operating system. However, there are times when we use a container and still want to reference files/directories on the home system. The `--bind` flag is used to set up user defined bind paths, binding directories from the host system to the directories within the container. The syntax for this binding is `/hostdirectory:/containerdirectory` and multiple bind paths can be written by separating with a comma.\n",
    ">> If running this container on a server other than Comet, these bind paths will need to be adjusted."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b20815b3",
   "metadata": {},
   "source": [
    "Once this command has been successfully run, you should see a list of all the installed packages in your container. Verify this list to ensure the container was built with the correct requirements.\n",
    "\n",
    "> Make sure you run this code line on a compute node in Comet, not the login node. To go onto a compute node, run the following line of code with your specific project ID.\n",
    ">> `srun -A <project_ID> --partition=compute --pty --nodes=1 --wait=0 --export=ALL -t 08:00:00 /bin/bash`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "032b8341",
   "metadata": {},
   "source": [
    "## Implementing the Container into Bash Scripts\n",
    "\n",
    "---\n",
    "\n",
    "Now that our container has been built, transferred, and verified, we can implement it into our scripts. For these next few steps, I will use scripts from the [MET-tools/Grid-Stat repository](https://github.com/CW3E/MET-tools/tree/main). I will work with the [`run_wrfout_cf.sh`](https://github.com/CW3E/MET-tools/blob/main/Grid-Stat/run_wrfout_cf.sh) and [`batch_wrfout_cf.sh`](https://github.com/CW3E/MET-tools/blob/main/Grid-Stat/batch_wrfout_cf.sh) scripts as examples. As these require a NetCDF environment, I built a container (`MET_tools_conda_netcdf.sif`) the same way as above, but now with a different requirements file that is specific to a NetCDF environment. Below are the steps to implement a container into your scripts using `MET_tools_conda_netcdf.sif` as an example."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a0f198a",
   "metadata": {},
   "source": [
    "### Remove any Existing conda Environment \n",
    "\n",
    "First, we will have to remove any reference to conda. This is because our newly built Singularity container replaces the need for a conda environment to be hard-coded within our scripts. For example, in the `batch_wrfout_cf.sh` script, commands such as `conda init bash` or `conda activate netcdf` can be removed."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db2e66c5",
   "metadata": {},
   "source": [
    "### Add Singularity Container into Scripts \n",
    "\n",
    "Now that any reference to conda has been removed, we have to add in our Singularity container for our commands to work. As we are working with a NetCDF environment, we have to reference our container with the following line of code before any calls to NetCDF (i.e., `ncks`, `cdo`, `ncl`)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ed9f593",
   "metadata": {},
   "source": [
    "```\n",
    "singularity exec -e --bind /cw3e:/cw3e,/scratch:/scratch /cw3e/mead/path/to/MET_tools_conda_netcdf.sif\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "301545fe",
   "metadata": {},
   "source": [
    "For example, the line of code I used was"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d11373e",
   "metadata": {},
   "source": [
    "```\n",
    "singularity exec -e --bind /cw3e:/cw3e,/scratch:/scratch /cw3e/mead/projects/cwp106/scratch/jlconti/MET_tools_conda_netcdf.sif\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5ba2b61",
   "metadata": {},
   "source": [
    "This line of code can be added before any NetCDF calls, as shown in the steps below."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c60b106",
   "metadata": {},
   "source": [
    "```\n",
    "cmd=\"ncl 'file_in=\\\"${file_2}\\\"' \"\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea9ce656",
   "metadata": {},
   "source": [
    "This is from [line 203](https://github.com/CW3E/MET-tools/blob/main/Grid-Stat/run_wrfout_cf.sh#L203) in the `run_wrfout_cf.sh` script. To implement the Singularity container, we can add the `singularity exec ...` command right before the call to `ncl`, as shown below. (And similar for any other NetCDF-related command in your scripts)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e9effb4",
   "metadata": {},
   "source": [
    "```\n",
    "cmd=\"singularity exec -e --bind /cw3e:/cw3e,/scratch:/scratch /cw3e/mead/projects/cwp106/scratch/jlconti/MET_tools_conda_netcdf.sif ncl 'file_in=\\\"${file_2}\\\"' \"\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53f4de6f",
   "metadata": {},
   "source": [
    "Check to see if the calls to the Singularity container will successfully run the script. If so, we know everything is working and can move on to the next step."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44400b15",
   "metadata": {},
   "source": [
    "## Using Containers to Run Python Scripts\n",
    "---\n",
    "\n",
    "For Bash scripts, we only had to reference the container for specific NetCDF commands. However, for Python scripts, we will need to use the container itself to run the code. This can be achieved by running a similar `singularity exec -e --bind ...` command from before directly in the terminal of the working directory. The form of the command to use for Python scripts is shown below."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eda24a5c",
   "metadata": {},
   "source": [
    "```\n",
    "singularity exec -e --bind /cw3e:/cw3e,/scratch:/scratch /cw3e/mead/projects/path/to/MET_tools_conda_ipython.sif python /path/to/scripts/python_script.py\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98e0d83b",
   "metadata": {},
   "source": [
    "For example, the line of code that I used to run the `proc_gridstat.py` script from the MET-tools/Grid-Stat directory was "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9882b982",
   "metadata": {},
   "source": [
    "```\n",
    "singularity exec -e --bind /cw3e:/cw3e,/scratch:/scratch /cw3e/mead/projects/cwp106/scratch/jlconti/MET_tools_conda_ipython.sif python /home/jlconti/MET-tools/Grid-Stat/proc_gridstat.py\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "722f7348",
   "metadata": {},
   "source": [
    "This line of code can be used to run any Python script that is dependent on the Singularity container environment."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6017243a",
   "metadata": {},
   "source": [
    "## Using a Singularity Container to Launch a Jupyter Notebook\n",
    "---\n",
    "\n",
    "Another way we can utilize our container is by using it to launch a Juypter notebook server off of Comet. Before we can achieve this, however, we need to check if our container has the required Jupyter packages. To check this, we can run the same line of code we used to [verify our container](#verify). For my specific container, I ran the following line \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9946692e",
   "metadata": {},
   "source": [
    "```\n",
    "singularity exec -e --bind /cw3e:/cw3e,/scratch:/scratch /cw3e/mead/projects/cwp106/scratch/jlconti/miniconda.sif conda list\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7e785a6",
   "metadata": {},
   "source": [
    "If the Juypter notebook packages are listed within the contained environment, then the container can be used to launch a Juypter notebook from the terminal. To do this, we first must make sure we are not in any active conda environment. If we are currently in a conda environment, then we can run the following command repeatedly until we are sure no conda environment is active."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42badb4d",
   "metadata": {},
   "source": [
    "```\n",
    "conda deactivate\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17127fef",
   "metadata": {},
   "source": [
    "Once we are out of any active conda environment, we can run the following line of code in the terminal."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43e71457",
   "metadata": {},
   "source": [
    "```\n",
    "galyleo launch -B  /cw3e:/cw3e,/scratch:/scratch --sif /cw3e/mead/path/to/miniconda.sif -j notebook -A <project_ID> -t 7-00:00:00 \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91865b57",
   "metadata": {},
   "source": [
    "As an example, the line of code I used is below."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3969dcc",
   "metadata": {},
   "source": [
    "```\n",
    "galyleo launch -B  /cw3e:/cw3e,/scratch:/scratch --sif /cw3e/mead/projects/cwp106/scratch/jlconti/miniconda.sif -j notebook -A cwp106 -t 7-00:00:00 \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c45d380",
   "metadata": {},
   "source": [
    "This command uses galyleo to launch a Jupyter notebook from a Singularity container environment for a set amount of time. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3b1605f",
   "metadata": {},
   "source": [
    "## Summary \n",
    "---\n",
    "This notebook explained Singularity containers and how to make one using a definition file. Once built, it was shown that a container replaces the need for any hard-coded conda environments. Commands that once depended on a conda environment now rely on the container made for that respective environment. Lastly, Singularity containers can be used to run Python scripts as well as launch Jupyter notebooks.  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "257365f1",
   "metadata": {},
   "source": [
    "## Next Steps\n",
    "---\n",
    "In the next notebook, [Creating a Configuration File](./Creating-a-Configuration-File.ipynb), we will review how to generate an universal configuration file for both Bash and Python scripts. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}