Skip to content

The main web portal for the Pyregence project.

License

Notifications You must be signed in to change notification settings

ijoel/pyregence

 
 

Repository files navigation

Pyregence

Pyregence is a web portal for displaying near-term fire forecast results for the state of California. The fire spread and impact maps are generated by running multiple state-of-the-art simulation models on a continuously updated stream of weather data.

At this time, Pyregence displays results from these open-source fire behavior models:

  1. ELMFIRE
  2. GridFire

Installation Requirements

Application Configuration

On startup, the Pyregence application reads a file called config.edn in the top-level directory of this repository, which contains various user-configurable system parameters. A sample file called config.default.edn is provided with this repository to get you started. To begin, simply rename (or copy) it to config.edn. Then edit it according to the following sections.

Please note that you must replace all values surrounded with angle brackets (e.g., “<api-key>”) with actual values for the application to work correctly. The Mapbox :access-token field requires a Mapbox access token. The :wildfire-camera-api-key field requires an API key for the AlertWildfire API. You should change the :pyr-auth-token field to a unique string that’s at least 16 characters long.

PostgreSQL Database Setup

PostgreSQL needs to be installed on the machine that will be hosting the Pyregence website. Once the PostgreSQL database server is running on your machine, you should navigate to the top-level directory and run the database build command as follows:

clojure -M:build-db build-all [--dev-data]

This will begin by creating a new database and role called pyregence and then add the pgcrypto and postgis extensions to it. Next, the script will populate the database with the schemas, tables, and functions that are necessary for storing and processing Pyregence’s data. Finally, it will load some default data into these tables that is necessary for the website to function properly. You can optionally load dev data with --dev-data. This will initialize the DB with 2 users and a “Development” organization.

Performance Settings

If you want to improve the performance of your PostgreSQL server, one way is to visit the pgtune website and input your system settings. This will generate a set of configuration options that you can add to the postgresql.conf file in your system-specific PostgreSQL data directory.

If you would like to keep these settings separate for your own reference, you can add them to a file called “performance.conf” that you should place in your PostgreSQL data directory. You can then import these settings into postgresql.conf by adding this line to the end of that file:

include = 'performance.conf'

Note that the PostgreSQL server will need to be restarted for these changes to take effect.

Environment Variable Setup

In order for your command-line shell to find the programs installed above, you will need to update your environment variables. Under GNU/Linux, *BSD, or MacOS X, you can set these through your shell’s .*rc or .*profile configuration files in your $HOME directory. In a Windows environment, you will need to set either the USER (for the current user only) or SYSTEM (for any user) environment variables under your system settings menu.

Step 1: Add JAVA_HOME

On a Unix machine using bash (e.g., GNU/Linux or MacOS X):

export JAVA_HOME=<jdk-installation-directory>

On a Unix machine using csh or tcsh (e.g., *BSD):

setenv JAVA_HOME <jdk-installation-directory>

On a Windows machine:

JAVA_HOME = C:\Program Files\Java\jdk-X.X.X

Replace X.X.X with the JDK version installed on your computer.

Step 2: Update PATH

On a Unix machine using bash (e.g., GNU/Linux or MacOS X):

export PATH=$PATH:$JAVA_HOME/bin

On a Unix machine using csh or tcsh (e.g., *BSD):

setenv PATH $PATH\:$JAVA_HOME/bin

On a Windows machine, add these entries to the PATH environment variable:

%JAVA_HOME%\bin
C:\Program Files\PostgreSQL\X\lib
C:\Program Files\PostgreSQL\X\bin

Replace X with the PostgreSQL version installed on your computer.

Development Web Server

Modifying hosts file

Add the following line to your hosts file. This local URL should be used for local developing in place of localhost due to CORS settings on the GeoServer instances used in Pyrecast.

127.0.0.1 local.pyrecast.org

Updating Chrome HSTS Settings

If you are planning on doing your local development on Chrome, you’ll need to update your HSTS (HTTP Strict Transport Security) settings so that trying to hit http://local.pyrecast.org doesn’t automatically redirect to https://local.pyrecast.org. To do so, open chrome://net-internals/#hsts in your Chrome browser and scroll down to the “Delete domain security policies” section. Enter local.pyrecast.org in the box and click “Delete”. You should now be all set to hit http://local.pyrecast.org. If you’re still running into issues, try clearing your Chrome browser cache (visit chrome://settings/privacy and click “Clear browsing data”).

Launching Figwheel

To start the Figwheel server, navigate to the top-level project directory and run:

clojure -M:default-ssl-opts:figwheel

This will start a web server on http://local.pyrecast.org:8080 (also accessible from http://localhost:8080), which serves up the website in dev mode. Any changes to CLJS files will be automatically pushed to the browser when the files are saved. Any changes to CLJ files will be pushed to the running server process. A CLJS browser REPL will also be launched at the terminal for you to interactively develop your client-side code.

Checking for Reflection Warnings

From the top-level project directory run:

clojure -M:check-reflection

This will emit warnings when reflection is needed to resolve Java method calls or field accesses. To resolve any that appear, add type hints to your code. Resolving all reflection warnings can improve system performance.

Debugging Your Dev Environment

If your dev environment isn’t working properly, refer to the following list of things to double-check:

  1. Make sure you have all of the required installation requirements listed above. For example, calling java --version should yield:
openjdk 17.0.3 2022-04-19
OpenJDK Runtime Environment (build 17.0.3+7-Ubuntu-0ubuntu0.20.04.1)
OpenJDK 64-Bit Server VM (build 17.0.3+7-Ubuntu-0ubuntu0.20.04.1, mixed mode, sharing)
  1. Make sure your config.edn file has every field from config.default.edn and you’ve correctly updated the fields that need updating (e.g. the access tokens are correct).
  2. Make sure your local database is working properly. To get a fresh database (with the development data) run:
clojure -M:build-db functions
clojure -M:build-db build-all --dev-data
  1. Make sure your /etc/hosts file has a line for 127.0.0.1 local.pyrecast.org and that you’re hitting http://local.pyrecast.org:8080 (make sure the port matches up to whatever you’ve set in config.edn) in your web browser.

If none of the above fix your problem, you should look at your Figwheel console and web browser dev tools console after running clojure -M:default-ssl-opts:figwheel for more information.

Production Web Server

Installation Requirements

Enabling HTTPS

View the Triangulum HTTPS page for further instructions on enabling HTTPS.

Compiling ClojureScript to JavaScript

To compile the ClojureScript files under src/cljs to JavaScript under `target/public/cljs`, navigate to the top-level project directory and run:

clojure -M:compile-cljs

The main JavaScript entry point file will be written to `target/public/cljs/app.js`. The ClojureScript compiler options are read from the top-level `compile-prod.cljs.edn` file.

Launching the Web Server

To compile and run the web application, navigate to the top-level project directory and run:

clojure -M:build-db functions
clojure -M:default-ssl-opts:server start -m [dev|prod] [-p 8080] [-P 8443] [-r/--repl]

The website will then be available at http://localhost:8080 unless a port is specified. An http port can be specified with -p and an https port can be specified with -P. In dev mode, server-side exceptions will be displayed in the browser and Clojure source files will be reloaded whenever you refresh the page. These features are disabled in prod mode. If -m is unspecified, it will default to prod mode.

If the -r/--repl flag is provided, then a repl server will start on port 5555. This can be connected to for debugging or managing the server.

The server is also configurable using the config.edn file:

;; config.edn
{:server {:http-port  8080
          :https-port 8443
          :mode       "<prod/dev>"
          :log-dir    "logs"
          :repl       <true/false>}}

If you don’t already have an SSL certificate, you cannot specify an https port.

Running the Web Server as a System Service

View the Triangulum Systemd page for further instructions on enabling the app as a system service.

Maintaining Daily Logs

By default the server will log to standard out. If you would like to have the system log to YYYY-DD-MM.log, use the `-o path` option to specify an output path. You can either specify a path relative to the top-level directory of this repository or an absolute path on your filesystem. The logger will keep the 10 most recent logs.

Running https via nginx (optional)

Install required packages

Copy the nginx template and update

Ubuntu example:

sudo cp nginx/nginx-template.conf /etc/nginx/sites-available/pyregence.conf
sudo ln -s /etc/nginx/sites-available/pyregence.conf /etc/nginx/sites-enabled/
sudo nano /etc/nginx/sites-available/pyregence.conf
sudo service nginx restart

Alternative server deployment

When using nginx, run the server in http mode only. If multiple servers are running on the same VM, be sure to assign them different ports. Remember that nginx needs port 8080, which is the default port for the server deployment. Instead deploy with a non default port:

clojure -M:run-server -p 8081

For HTTPS, each server does not need its own deploy hook. Instead nginx needs a single deploy hook:

sudo nano /etc/letsencrypt/renewal-hooks/deploy/nginx.sh

Add systemctl reload nginx to the file and save.

sudo chmod +x /etc/letsencrypt/renewal-hooks/deploy/nginx.sh

Diagnose errors with nginx by looking in the log

Ubuntu example:

sudo less +G /var/log/nginx/error.log

License and Distribution

Copyright © 2020-2024 Spatial Informatics Group, LLC.

Pyregence is distributed by Spatial Informatics Group, LLC. under the terms of the Eclipse Public License version 2.0 (EPLv2). See LICENSE.txt in this directory for more information.

About

The main web portal for the Pyregence project.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Clojure 93.5%
  • PLpgSQL 2.8%
  • JavaScript 2.5%
  • CSS 1.2%