Skip to content

Trollfow2 Installation Tutorial

Jump to bottom
martabalague99 edited this page Dec 18, 2024 · 11 revisions

Welcome to the TROLLFLOW2 Tutorial.

In this wiki you can find all the information about how to set up a Trollflow2 operational environment all through the use of Docker containers.

  1. The first step needed is to create a Pytroll Base Image or also known as building a Docker Image that contains a Pytroll conda environment.

Steps:

a) Clone the Pytroll Docker files using Git

Nordsat option:

git clone https://github.com/nordsat/shared-docker-files.git

b) Edit pytroll_base_image/build_dir/dockerfile_ubuntu to indicate the Ubuntu base that we want to use

FROM ubuntu:latest

c) Build the image in our unit

  • docker build -t pytroll_base_image -f build_dir/Dockerfile_ubuntu build_dir

d) To verify that the image is working correctly we can execute docker image ls and we should get something similar to:

user@machine:~/meteosat/shared-docker-files$ docker image ls

REPOSITORY TAG IMAGE ID CREATED SIZE

pytroll_base_image latest 252160da937e 9 days ago 471MB

  1. The second step is to build the proper Trollflow2 Image

We step into the pytroll_base_image/trollflow2_full_image and we execute the command:

  • docker build -t trollflow2 -f build_dir/Dockerfile_stable build_dir

If we execute docker image ls we should get something similar to:

user@machine:~/shared-docker-files$ docker image ls

REPOSITORY TAG IMAGE ID CREATED SIZE

trollflow2 latest b59dcad1824c 9 days ago 1.85GB

pytroll_base_image latest 252160da937e 9 days ago 471MB

  1. Start the Trollflow2 Docker

Once the container with both images has been created, we can proceed with setting the folder paths and running the image.

Steps:

a) Create the container folder and copy the files

sudo rsync -avP ../../shared-docker-files/trollflow2_full_image/* .

b) Change user access permission

sudo chown -R [usuari]:[grup] *

chmod 700 docker_run_trollflow2.sh

c) Execute the running command

./docker_run_trollflow2.sh

docker run \

--name trollflow2 \

-e LOCAL_USER_ID=id -u $USER \

--mount type=bind,source=pwd/config_seviri_hrit/,target=/mnt/config/ \

--mount type=bind,source=pwd/logs/,target=/mnt/logs/ \

--mount type=bind,source=pwd/input/,target=/mnt/input/ \

--mount type=bind,source=pwd/output/,target=/mnt/output/ \

-p 9001:9001 \

-d \

trollflow2

In order to check if Trollflow2 has been set properly and is running correctly, when doing docker ps we should be getting something like:

user@machine:~/meteosat/shared-docker-files$ docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

5fa6d8cf556b trollflow2 "/usr/local/bin/entr…" 25 hours ago Up 25 hours 0.0.0.0:9001->9001/tcp trollflow2

By following the previous steps we should be able to have the Docker running. We can check the status and runtime of the processes inside the Docker (msg_0deg_segment_gatherer / msg_0deg_trollflow2 / msg_0deg_trollstalker / nameserver) through the use of the Supervisord Website.

imagen

Example of Supervisord website → http://127.0.0.1:9001/ (Your IP may be different to this one) with default credentials being → username & password

If you want to automatize the installation process you can follow this script:

#!/bin/bash

BASE_DIR="$HOME/dockers/pytroll"
PYTROLL_REPO_DIR="$BASE_DIR/repo"
TROLLFLOW2_DEF_DIR="$BASE_DIR/def/trollflow2"
TROLLFLOW2_MNT_DIR="$BASE_DIR/mnt/trollflow2"

LOCAL_USER_ID=$(id -u)

mkdir -p "$PYTROLL_REPO_DIR" "$TROLLFLOW2_DEF_DIR" "$TROLLFLOW2_MNT_DIR"

cd "$PYTROLL_REPO_DIR"

git clone https://github.com/nordsat/shared-docker-files.git

sed -i 's/^FROM vic.smhi.se\/smhi-common\/ubuntu:20.04/#FROM vic.smhi.se\/smhi-common\/ubuntu:20.04/' "$PYTROLL_REPO_DIR/shared-docker-files/pytroll_base_image/build_dir/Dockerfile_ubuntu" && \
sed -i 's/^#FROM ubuntu:latest/FROM ubuntu:latest/' "$PYTROLL_REPO_DIR/shared-docker-files/pytroll_base_image/build_dir/Dockerfile_ubuntu"

docker build -t pytroll_base_image -f "$PYTROLL_REPO_DIR/shared-docker-files/pytroll_base_image/build_dir/Dockerfile_ubuntu" "$PYTROLL_REPO_DIR/shared-docker-files/pytroll_base_image/build_dir"

docker build -t trollflow2 -f "$PYTROLL_REPO_DIR/shared-docker-files/trollflow2_full_image/build_dir/Dockerfile_stable" "$PYTROLL_REPO_DIR/shared-docker-files/trollflow2_full_image/build_dir"

cp -r "$PYTROLL_REPO_DIR/shared-docker-files/trollflow2_full_image/config_seviri_hrit" "$TROLLFLOW2_MNT_DIR/"
cp -r "$PYTROLL_REPO_DIR/shared-docker-files/trollflow2_full_image/logs" "$TROLLFLOW2_MNT_DIR/"
cp -r "$PYTROLL_REPO_DIR/shared-docker-files/trollflow2_full_image/input" "$TROLLFLOW2_MNT_DIR/"
cp -r "$PYTROLL_REPO_DIR/shared-docker-files/trollflow2_full_image/output" "$TROLLFLOW2_MNT_DIR/"

cat <<EOL > "$TROLLFLOW2_DEF_DIR/docker-compose.yaml"
services:
  trollflow2:
    image: trollflow2
    container_name: trollflow2
    ports:
      - "9001:9001"
    volumes:
      - $TROLLFLOW2_MNT_DIR/config_seviri_hrit/:/mnt/config/
      - $TROLLFLOW2_MNT_DIR/logs/:/mnt/logs/
      - $TROLLFLOW2_MNT_DIR/input/:/mnt/input/
      - $TROLLFLOW2_MNT_DIR/output/:/mnt/output/
      # - trollflow2-input:/mnt/input/
      # - trollflow2-output:/mnt/output/
    restart: unless-stopped
    logging:
      driver: json-file
      options:
        max-size: "25m"
        max-file: "3"
    environment:
      - ENTORN=DEV
      - LOCAL_USER_ID=${LOCAL_USER_ID}
EOL

docker compose -f "$TROLLFLOW2_DEF_DIR/docker-compose.yaml" up -d

Tutorial by @martabalague99

Clone this wiki locally