Skip to content

1. Installation

Jump to bottom
Mark Edward M. Gonzales edited this page Oct 8, 2024 · 280 revisions

This page discusses how to install and run RicePilaf locally, as well as how to deploy RicePilaf to a server.

Basic System Requirements

  • Storage: ~15.6 GB (~1.6 GB for the Docker image and ~14 GB for the accompanying dataset)
  • Memory: ≥ 4 GB
  • Operating System: Linux, macOS, or Windows

1️⃣ Downloading the Dataset

  1. Start by downloading the latest version of the dataset needed to run the app from here.

    💡 If you want to verify the integrity of the downloaded dataset, compute the SHA-512 checksum of the tgz archive file using a hashing utility like certutil in Windows, shasum in Mac, or sha512sum in Linux. You should obtain the following checksum:

    044576b4dc97f5ad78f686f96b2899f9d5f25afc487096309a56d2b9b837a9bdcbc4d97585e95a0b0e880cb3591154e385ce33ef74db682cac56cd519c2677ff

  2. Extract the contents of the downloaded dataset:

    • [For Linux, macOS, and Windows 10 onwards] Launch a terminal from the download location, and run the following command:

      tar -xvzf static.tgz

    • [For Windows versions older than Windows 10] Use an unpacking tool like WinRAR or 7-Zip.

  3. The extraction process should result in a folder named static. Inside this should be a single folder named app_data.

2️⃣ Setting up the Dependencies

⚠️ Important: Depending on your operating system configuration, you may have to use sudo when running the Docker commands.

  1. Download and install Docker, a platform for building and running containerized apps:

    • [For macOS and Windows] Install Docker Desktop.
    • [For Linux] For easier installation, we recommend installing Docker Engine instead of Docker Desktop. Instructions for different Linux distributions can be found here.

  2. Start the Docker daemon:

    • [For macOS and Windows] Open Docker Desktop to start the daemon.
    • [For Linux] Follow the instructions here.

  3. Launch a terminal (from anywhere), and pull the latest RicePilaf Docker image by running:

    docker pull ghcr.io/bioinfodlsu/rice-pilaf/app:latest

  4. Spin up a container from the image by running:

    docker create --name rice-pilaf -v path/to/static/in/local:/app/static -p 8050:8050 ghcr.io/bioinfodlsu/rice-pilaf/app:latest

    Replace path/to/static/in/local with the path to the static folder generated following the steps in the previous section. It may be more convenient to use the absolute path. If you are using Windows, replace the backward slashes (\) in the path with forward slashes (/).

3️⃣ Running the App Locally

⚠️ Important: Depending on your operating system configuration, you may have to use sudo when running the Docker commands.

  1. Launch a terminal (from anywhere), and start the RicePilaf container by running:

    docker start rice-pilaf

  2. To open RicePilaf, visit http://localhost:8050/ on your browser.

    You may have to wait for a few seconds and refresh your browser tab until you are greeted by the following screen:

    RicePilaf Home (Local)

  3. Once you are done using RicePilaf, stop the container by running:

    docker stop rice-pilaf

4️⃣ Deploying the App to a Server

For system administrators: Click here to show/hide the instructions if you want to deploy RicePilaf to a server

Basic System Requirements

  • Storage: ~16 GB (~2 GB for the Docker image and ~14 GB for the accompanying dataset)
  • Memory: ≥ 4 GB
  • Operating System: Linux

Steps

⚠️ Important: Depending on your operating system configuration, you may have to use sudo when running the Docker commands.

  1. Download the dataset following the instructions here. Doing so should result in a folder named static. Inside this should be a single folder named app_data.

  2. Download and install Docker Engine.

  3. Start the Docker daemon following the instructions here.

  4. Create a docker-compose.yml file for pulling the latest version of RicePilaf and for configuring Watchtower to automatically update the deployed app every time a new version is released:

    services:
  rice-pilaf:
    image: ghcr.io/bioinfodlsu/rice-pilaf/deploy:latest
    container_name: rice-pilaf
    ports:
      - 80:80
    labels:
      - "com.centurylinklabs.watchtower.lifecycle.post-update='/app/run-cron.sh'"
    volumes:
      - path/to/static/in/server:/app/static
      - path/to/service/account/key:/app/cron_jobs/credentials/ricepilaf-logger-key.json
    restart: unless-stopped
  watchtower:
    image: containrrr/watchtower
    container_name: watchtower
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    command: rice-pilaf --interval 30 --enable-lifecycle-hooks
    restart: unless-stopped

    Replace path/to/static/in/server with the path to the static folder in the server. It may be more convenient to use the absolute path.

    To help us improve RicePilaf, we provide an option to collect usage logs and automatically send them to us:

    • If you would like to send the usage logs, kindly contact us to request a service account key. Replace path/to/service/account/key with the path to the service account key.
    • If you prefer not to send the usage logs, delete the line - path/to/service/account/key:/app/cron_jobs/credentials/ricepilaf-logger-key.json from the docker-compose.yml file.

  5. Launch a terminal from the folder containing the docker-compose.yml file, and run the following command:

    docker compose up -d

  6. RicePilaf runs the following cron jobs: (i) sending the usage logs and (ii) clearing the directory where results of previously run analyses are cached to prevent repeated computations. Note that the first cron job would successfully execute only if you chose to supply an authorized service key when the container was spun up.

    Activate these cron jobs by running:

    docker exec rice-pilaf cron

  7. To open RicePilaf, visit http://<IP_ADDRESS> on your browser, where <IP_ADDRESS> is the public IP address of the server.

    You may have to wait for a few seconds and refresh your browser tab until you are greeted by the following screen:

    RicePilaf Home (Deployed)

  8. To stop the RicePilaf container, run the following command:

    docker stop rice-pilaf

Configuring App Settings

  1. Open a shell to execute commands in the container:

    docker exec -it rice-pilaf bash

    Doing so should change the working directory to root@<container_id>:/app.

  2. On the shell inside the container, run the script config/generate_config.py to configure app settings. Run the following command to display instructions on how to use this script:

    python3 config/generate_config.py --help

    Common use cases are as follows:

    • To turn off app usage logging, remove the --logging flag, like so:

      python3 config/generate_config.py --deployed --latest-version

    • To turn on app usage logging, include the --logging flag, like so:

      python3 config/generate_config.py --deployed --latest-version --logging

    • To set the maximum storage allocated for the app usage log files (default: 1 GB), use the --max-logging-gb argument, like so:

      python3 config/generate_config.py --deployed --latest-version --logging --max-logging-gb=<allocation_in_GB>

    • To set the maximum storage allocated for the directory where results of previously run analyses are cached to prevent repeated computations (default: 10 GB), use the --max-cache-gb argument, like so:

      python3 config/generate_config.py --deployed --latest-version --logging --max-cache-gb=<allocation_in_GB>

  3. On the shell inside the container, verify that the app settings have been changed by running:

    cat .env

  4. On the shell inside the container, restart the server by running:

    service nginx restart

    This will also stop the entire RicePilaf container, so you have to restart it by running:

    docker start rice-pilaf

5️⃣ Switching from HTTP to HTTPS

For system administrators: Click here to show/hide the instructions if you want to serve RicePilaf over HTTPS. Note that the instructions here are specific to the server of the Bioinformatics Lab, De La Salle University

Following the deployment instructions in the previous section will serve RicePilaf over HTTP. The procedure for switching from HTTP to HTTPS usually depends on the network configuration and policies of your organization.

For the domain name ricepilaf.bioinfodlsu.com (which we maintain at the Bioinformatics Lab, De La Salle University), we use Let's Encrypt for the SSL certificate. The instructions are as follows:

A. Serving the App over Plain HTTP

The app has to be served first over plain HTTP so that Let's Encrypt can issue challenges to verify the lab's control over the domain name. Hence, the deployment steps are different from those in the previous section; the nginx configuration for deploying RicePilaf to the lab's server over plain HTTP has additional directives (e.g., server_name for the domain name) and a location context for ACME challenges.

⚠️ Important #1: If you need to edit the domain name, replace every instance of the domain name in nginx-bioinfodlsu.conf and nginx-bioinfodlsu-https.conf.

⚠️ Important #2: Do not edit /etc/nginx/nginx.conf inside the container. Your changes will be undone when the container is restarted (more details here). If you need to edit the nginx configuration, delete the existing container and its associated image, edit nginx.conf at the root of the cloned repository, build the image, and spin up a new container.

  1. Clone the repository by running:

    git clone https://github.com/bioinfodlsu/rice-pilaf
cd rice-pilaf

  2. Replace nginx.conf with nginx-bioinfodlsu.conf:

    cp -f nginx-bioinfodlsu.conf nginx.conf

  3. Create a docker-compose.yml file inside the root of the cloned repository:

    services:
  rice-pilaf:
    image: rice-pilaf
    build:
      dockerfile: Dockerfile-deploy
    container_name: rice-pilaf
    ports:
      - 80:80
    volumes:
      - path/to/static/in/server:/app/static
      - path/to/service/account/key:/app/cron_jobs/credentials/ricepilaf-logger-key.json
      - /etc/letsencrypt:/etc/letsencrypt:ro
      - /tmp/acme-challenge:/tmp/acme-challenge
    restart: unless-stopped

    Replace path/to/static/in/server and path/to/service/account/key with the path to the static folder and the path to the service account key for logging, respectively. It may be more convenient to use absolute paths.

  4. Launch a terminal from the folder containing the docker-compose.yml file, and run the following command:

    docker compose up -d

  5. Verify that http://ricepilaf.bioinfodlsu.com/ (served over plain HTTP) is live. We recommend testing on an incognito tab to prevent issues related to browser caching.

B. Generating an SSL Certificate

  1. Install certbot, and generate an SSL certificate by running:

    certbot certonly --webroot -w /tmp/acme-challenge/ -d ricepilaf.bioinfodlsu.com --text --agree-tos --email [email protected] --rsa-key-size 4096 --verbose --keep-until-expiring --preferred-challenges=http

    Replace the -d and --email flag values if you need to change the domain name and the email address, respectively.

    Running this command should result in a certificate (fullchain.pem) and a key (privkey.pem) stored at /etc/letsencrypt/live/ricepilaf.bioinfodlsu.com/

C. Serving the App over HTTPS

  1. Stop the RicePilaf container, and delete it and its associated image by running:

    docker stop rice-pilaf
docker rm rice-pilaf
docker rmi rice-pilaf

  2. Delete the previously created docker-compose.yml file, and create a new docker-compose.yml for pulling the latest version of RicePilaf and for configuring Watchtower to automatically update the deployed app every time a new version is released:

    services:
  rice-pilaf:
    image: ghcr.io/bioinfodlsu/rice-pilaf/deploy-bioinfodlsu:latest
    container_name: rice-pilaf
    ports:
      - 80:80
      - 443:443
    volumes:
      - path/to/static/in/server:/app/static
      - path/to/service/account/key:/app/cron_jobs/credentials/ricepilaf-logger-key.json
      - /etc/letsencrypt:/etc/letsencrypt:ro
      - /tmp/acme-challenge:/tmp/acme-challenge
    restart: unless-stopped
  watchtower:
    image: containrrr/watchtower
    container_name: watchtower
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    command: rice-pilaf --interval 30 --enable-lifecycle-hooks
    restart: unless-stopped

    Note that the RicePilaf image pulled here is the deploy-bioinfodlsu image (not the deploy image). The nginx configuration in the deploy-bioinfodlsu image has an additional server block for listening on port 443 (i.e., for serving our app over HTTPS).

  3. Launch a terminal from the folder containing the docker-compose.yml file, and run the following command:

    docker compose up -d

  4. Activate the cron jobs by running:

    docker exec rice-pilaf cron

  5. Verify that https://ricepilaf.bioinfodlsu.com/ (served over HTTPS) is live.

❓ Frequently Asked Questions

How can I build the RicePilaf image locally?

Click here to show/hide the steps

  1. Download Docker, and start the Docker daemon (as in Steps 1 and 2 here).

  2. Clone the RicePilaf repository by running:

    git clone https://github.com/bioinfodlsu/rice-pilaf

  3. Launch a terminal from the root of the cloned repository, and build the Docker image by running:

    docker build -t rice-pilaf -f Dockerfile-app .

  4. Spin up a container from the image by running:

    docker create --name rice-pilaf -v path/to/static/in/local:/app/static -p 8050:8050 rice-pilaf

    Replace path/to/static/in/local with the path to the static folder generated following the steps here. It may be more convenient to use the absolute path. If you are using Windows, replace the backward slashes (\) in the path with forward slashes (/).

  5. Run RicePilaf following the steps here.

How can I run RicePilaf without Docker?

Click here to show/hide the steps (best of luck!)

⚠️ Important: Windows users have to run the commands on Windows Subsystem for Linux (WSL). Using WSL is necessary since one of the dependencies (i.e., pybedtools) requires Unix/Unix-like utilities.

  1. Install Python (recommended version: 3.10.13).

  2. Clone the RicePilaf repository by running:

    git clone https://github.com/bioinfodlsu/rice-pilaf

  3. Transfer the static folder (generated following the steps here) to the root of the cloned repository. It should be in the same level as callbacks, pages, etc.

  4. Launch a terminal from the root of the cloned repository, and install the required Python libraries by running:

    python3 -m pip install -r dependencies/requirements-app.txt

  5. Install MCDP2 (a tool used for the regulatory feature enrichment analysis) by running the following commands:

    git clone https://github.com/fmfi-compbio/mcdp2
cd mcdp2
git reset --hard fd7c69f5e97db8c1052df859cb02d86533287e64
python3 -m pip install .
cd ../

  6. Retain the front-end styling of the app even when used offline by running the following commands:

    wget https://github.com/twbs/icons/releases/download/v1.11.0/bootstrap-icons-1.11.0.zip 
wget https://use.fontawesome.com/releases/v6.5.2/fontawesome-free-6.5.2-web.zip 
unzip bootstrap-icons-1.11.0.zip 
unzip fontawesome-free-6.5.2-web.zip
mv bootstrap-icons-1.11.0 assets
mkdir -p assets/fontawesome-free-6.5.2-web
mv -t assets/fontawesome-free-6.5.2-web fontawesome-free-6.5.2-web/svgs fontawesome-free-6.5.2-web/webfonts
rm -rf bootstrap-* fontawesome-*

  7. Start the server by running:

    python3 app.py

  8. To open RicePilaf, visit http://localhost:8050/ on your browser, as in Step 2 here.

  9. Once you are done using RicePilaf, just close the terminal running the server.

How can I run older versions of RicePilaf?

Click here to show/hide the steps

⚠️ Important: Make sure that the dataset version matches the release version of the code that you want to run. The dataset version consists of the first two numbers in the release version of the code. For example, if the release version of the code is 0.1.x, then the dataset version should be 0.1.

  1. Refer to this spreadsheet for the link to the dataset and its SHA-512 checksum.

  2. Extract the contents of the downloaded dataset. Doing so should result in a folder named static. Inside this should be a single folder named app_data.

  3. Download Docker, and start the Docker daemon (as in Steps 1 and 2 here).

  4. Launch a terminal (from anywhere), and pull the RicePilaf Docker image by running:

    docker pull ghcr.io/bioinfodlsu/rice-pilaf/app:v<RELEASE_VERSION>

    Replace <RELEASE_VERSION> with the release version of the code. A complete list of all the release versions can be found here.

  5. Spin up a container from the image by running:

    docker create --name rice-pilaf -v path/to/static/in/local:/app/static -p 8050:8050 ghcr.io/bioinfodlsu/rice-pilaf/app:<RELEASE_VERSION>

    Replace path/to/static/in/local with the path to the static folder. It may be more convenient to use the absolute path. If you are using Windows, replace the backward slashes (\) in the path with forward slashes (/).

    Replace <RELEASE_VERSION> with the release version of the code.

    Note: If you are intending to run version 0.1.0, change the -p flag value to 8050:80.

  6. Run RicePilaf following the steps here.

Clone this wiki locally