Customize Django settings via environment or settings_local.py (#100)

Enable two new ways to customize the settings: (1) overwrite settings using a `local_settings.py`; (2) use environment variables with prefix `QLEVERUI_`, for example `QLEVERUI_DEBUG` (see the default `.env` file for a complete list). Each environment variable can also be set using a file, for example `QLEVERUI_SECRET_KEY_FILE` (the respective setting then gets the content of that file as value). In particular, this fixes #92. 

Customizing the settings via `settings_secret.py` for `SECRET_KEY` and `ALLOWED_HOSTS` is still supported but deprecated. Instead, `settings_local.py` should now be used. For the migration, all assignments to the default value should be deleted and the file should be renamed from `settings_secret.py` to `settings_local.py`.

.env

@@ -0,0 +1,8 @@
+# This file will be read during the startup of Qlever UI. You can set the values in this file or in environment variables.
+# The available settings are listed below. For further configuration set the values in a `settings_local.py`.
+#QLEVERUI_SECRET_KEY=!!super_secret!!
+#QLEVERUI_DEBUG=True
+# Use comma as a separator between items. For example `https://tf.uni-freiburg.de,https://uni-freiburg.de`
+#QLEVERUI_ALLOWED_HOSTS=*
+#QLEVERUI_CSRF_TRUSTED_ORIGINS=https://*.uni-freiburg.de
+#QLEVERUI_DATABASE_URL=sqlite:////absolute/path/to/db/qleverui.sqlite3

.gitignore

@@ -1,7 +1,6 @@
 *.pyc
 *.tmp
 *.swp
-qlever/settings_secret.py
 backend/static/admin/
 django.log
 .vscode
@@ -11,3 +10,4 @@ db/qleverui.sqlite3.BACKUP*
 backend/static/js/qleverUI.js
 .DS_Store
 staticfiles
+.env

README.md

@@ -12,10 +12,9 @@
     <strong><a href="https://github.com/ad-freiburg/QLever">QLever Engine</a></strong>
 </p>
 <br>
-  <div style="text-align: center;">
+<div style="text-align: center;">
     <img src="docs/screenshot.png" alt="QLever UI">
 </div>
-</p>
 
 # What is QLever UI?
 **QLever UI** is an easy-to-use interactive **user interface** for the **SPARQL search engine QLever** that helps to discover the scopes and information in very large knowledge bases by providing **context-sensitive suggestions** and **auto-completion** while adding helpful information and additional views to the various outputs of the queries.

backend/management/commands/copy.py

@@ -26,7 +26,7 @@ def add_arguments(self, parser):
                             help="Slug of the destination backend (must be new)")
 
     # This defined the actual behavior. See here for how to update the database:
-    # https://docs.djangoproject.com/en/4.1/topics/db/queries/
+    # https://docs.djangoproject.com/en/5.1/topics/db/queries/
     def handle(self, *args, returnLog=False, **options):
         self.log()
         try:

docs/install_qleverui.md

@@ -1,16 +1,17 @@
 # Installing QLever UI
-* [Building the QLever UI Docker Image](#building-the-qlever-ui-docker-container)
+* [Building the QLever UI Docker Image](#building-the-qlever-ui-docker-image)
     * [Setting up the database](#setting-up-the-database)
     * [Running a QLever UI Docker container](#running-a-qlever-ui-docker-container)
 * [Installing QLever UI without docker](#installing-qlever-ui-without-docker)
     * [Setting up the database manually](#setting-up-the-database-manually)
     * [Running QLever UI without docker](#running-qlever-ui-without-docker)
+* [Configuration](#configuration)
 
 <br>
 
-Since QLever UI was built upon Python/Django further information and details on the setup and configuration process (especially in production environments) can be found in the [Django documentation](https://docs.djangoproject.com/en/3.0/).
+Since QLever UI was built upon Python/Django further information and details on the setup and configuration process (especially in production environments) can be found in the [Django documentation](https://docs.djangoproject.com/en/5.1/).
 <br><br>
-We provide a [docker image](#building-the-qlever-ui-docker-container) as well as instructions for a [manual setup](#installing-qlever-ui-without-docker).
+We provide a [docker image](#building-the-qlever-ui-docker-image) as well as instructions for a [manual setup](#installing-qlever-ui-without-docker).
 <br>
 
 # Building the QLever UI Docker Image
@@ -20,10 +21,7 @@ We assume [docker to be installed](https://docs.docker.com/get-docker/) on your
     git clone https://github.com/ad-freiburg/qlever-ui.git qlever-ui
     cd qlever-ui
     ```
-2. Before building the Docker Image, move the [`settings_secret_template.py`](/qlever/settings_secret_template.py) file to `settings_secret.py` and adjust it to [fit your needs](https://docs.djangoproject.com/en/3.0/ref/settings/).
-    ```shell
-    mv qlever/settings_secret_template.py qlever/settings_secret.py
-    ```
+2. Optionally [adjust settings](#configuration)
 3. Finally, build the Docker image by running:
     ```shell
     docker build -t qleverui .
@@ -66,25 +64,22 @@ You should now be able to connect to QLever UI via <http://localhost:7000>. Cont
 
 
 # Installing QLever UI without docker
-When not using docker there are some additional steps to do. QLever UI is built upon a [Python 3](https://www.python.org/downloads/) / [Django 3](https://www.djangoproject.com/) backend so you will need to have Python 3 installed in order to run QLever UI. It's strongly recommended to use [virtual environments](https://docs.python.org/3/library/venv.html) to manage the project's dependencies when not using the docker build. In order to manage the dependencies, we use pip.
+When not using docker there are some additional steps to do. QLever UI is built upon a [Python 3](https://www.python.org/downloads/) / [Django 5](https://www.djangoproject.com/) backend so you will need to have Python 3 installed in order to run QLever UI. It's strongly recommended to use [virtual environments](https://docs.python.org/3/library/venv.html) to manage the project's dependencies when not using the docker build. In order to manage the dependencies, we use pip.
 
 1. If "[pip](https://pypi.org/project/pip/)" is installed on your system / in your virtual environment you can simply use 
     ```shell
     pip install -r requirements.txt
     ```
     inside the project folder to automatically install all dependencies. Otherwise, you can find the list of dependencies in the `requirements.txt` file to install them manually.
 
-2. Next, you will need to adjust your individual settings in `qlever/settings_secret_template.py` and rename it to `qlever/settings_secret.py`. You may want to edit the file to [fit your needs](https://docs.djangoproject.com/en/3.0/ref/settings/).
-    ```shell
-    mv qlever/settings_secret_template.py qlever/settings_secret.py
-    ```
+2. Optionally [adjust settings](#configuration)
 
 ## Setting up the database manually
 1. The QLever UI backend needs a database connection - by default SQLite is used and no further configuration is required. Simply run:
     ```shell
     python manage.py makemigrations —-merge && python manage.py migrate
     ```
-    inside the project folder in order to do so. You will only need to do this once. If you prefer you can also overwrite the database [settings](https://docs.djangoproject.com/en/3.0/ref/settings/) to use some other database management system in your `settings_secret.py`.
+    inside the project folder in order to do so. You will only need to do this once. If you prefer you can also overwrite the database [settings](https://docs.djangoproject.com/en/5.1/ref/settings/) to use some other database management system in your `.env`.
 
 2. For configuring your QLever UI backend you will need an administrative user for the QLever UI administration panel. You can create an admin account by simply running the following command in your project folder: 
     ```shell
@@ -96,7 +91,25 @@ You can either start a development server by simply running
 ```shell
 ./manage.py runserver localhost:7000
 ```
-or prepare a productive environment as described in the [Django documentation](https://docs.djangoproject.com/en/3.0/).
+or prepare a productive environment as described in the [Django documentation](https://docs.djangoproject.com/en/5.1/).
 
 You can start the development instance at any time with this single command and access your instance by opening http://localhost:7000. Feel free to change the port or hostname if needed.
 Read more about configuration in the [next chapter](./configure_qleverui.md).
+
+# Configuration
+
+- You can adjust some basic settings in [`.env`](../.env) or using environment variables. All the available options are listed in [`.env`](../.env).
+- You can generate a secure `SECRET_KEY` and write it into `.env` with:
+  ```shell
+  python3 -c 'from django.core.management.utils import get_random_secret_key; print(f"SECRET_KEY={get_random_secret_key()}")' >> .env
+  ```
+- You can overwrite all settings in [`qlever/settings_local.py`](../qlever/settings_local.py). Just set the setting to the value you want it to have.
+   See the [Django documentation](https://docs.djangoproject.com/en/5.1/ref/settings/) for the available settings.
+
+## Configuration Precedence
+The precedence (the top item has the highest precedence) of the Django settings is:
+1. [`qlever/settings_local.py`](../qlever/settings_local.py)
+2. Environment variables
+3. [`.env`](../.env)
+4. *Deprecated*: `settings_secret.py` (only applied for non-default values)
+5. Default values in [`qlever/settings.py`](../qlever/settings.py)

qlever/settings.py

@@ -4,29 +4,56 @@
 Generated by 'django-admin startproject' using Django 1.11.3.
 
 For more information on this file, see
-https://docs.djangoproject.com/en/1.11/topics/settings/
+https://docs.djangoproject.com/en/5.1/topics/settings/
 
 For the full list of settings and their values, see
-https://docs.djangoproject.com/en/1.11/ref/settings/
+https://docs.djangoproject.com/en/5.1/ref/settings/
 """
 
-from .settings_secret import *
-
 import os
 import subprocess
 import re
+import environ
+try:
+    from .settings_secret import *
+    print("settings_secret.py is deprecated. Please migrate to using settings_local.py. "
+          "To do this rename the file. You can also remove all assignments to the default value.")
+except ImportError:
+    pass
+
+env = environ.FileAwareEnv()
 
 # Build paths inside the project like this: os.path.join(BASE_DIR, ...)
 BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
 
+environ.Env.read_env(os.path.join(BASE_DIR, '.env'))
+
 # Suppress warning in Django 3.2 version.
 DEFAULT_AUTO_FIELD = 'django.db.models.AutoField'
 
 # Quick-start development settings - unsuitable for production
-# See https://docs.djangoproject.com/en/1.11/howto/deployment/checklist/
+# See https://docs.djangoproject.com/en/5.1/howto/deployment/checklist/
 
 # SECURITY WARNING: don't run with debug turned on in production!
-DEBUG = True
+DEBUG = env.bool('QLEVERUI_DEBUG', default=True)
+# https://docs.djangoproject.com/en/5.1/ref/settings/#allowed-hosts
+
+# The values set in settings_secret.py have a higher precedence than the default values.
+# They may or may not be set. If they are set we use them instead of the default value.
+# Otherwise, we get a NameError and use the default value in the except branch
+try:
+    ALLOWED_HOSTS_DEFAULT = ALLOWED_HOSTS
+    print("Using value from settings_secret.py for ALLOWED_HOSTS.")
+except NameError:
+    ALLOWED_HOSTS_DEFAULT = ['*']
+ALLOWED_HOSTS = env.list('QLEVERUI_ALLOWED_HOSTS', default=ALLOWED_HOSTS_DEFAULT)
+
+try:
+    SECRET_KEY_DEFAULT = SECRET_KEY
+    print("Using value from settings_secret.py for SECRET_KEY.")
+except NameError:
+    SECRET_KEY_DEFAULT = '!!super_secret!!'
+SECRET_KEY = env.str('QLEVERUI_SECRET_KEY', default=SECRET_KEY_DEFAULT)
 
 # Application definition
 
@@ -49,7 +76,7 @@
     'django.middleware.clickjacking.XFrameOptionsMiddleware',
 ]
 
-CSRF_TRUSTED_ORIGINS = ['https://*.uni-freiburg.de']
+CSRF_TRUSTED_ORIGINS = env.list('QLEVERUI_CSRF_TRUSTED_ORIGINS', default=['https://*.uni-freiburg.de'])
 
 ROOT_URLCONF = 'qlever.urls'
 
@@ -73,17 +100,14 @@
 WSGI_APPLICATION = 'qlever.wsgi.application'
 
 # Database
-# https://docs.djangoproject.com/en/1.11/ref/settings/#databases
+# https://docs.djangoproject.com/en/5.1/ref/settings/#databases
 
 DATABASES = {
-    'default': {
-        'ENGINE': 'django.db.backends.sqlite3',
-        'NAME': os.path.join(BASE_DIR, 'db', 'qleverui.sqlite3'),
-    }
+    'default': env.db("QLEVERUI_DATABASE_URL", default=f'sqlite:////{os.path.join(BASE_DIR, "db", "qleverui.sqlite3")}')
 }
 
 # Password validation
-# https://docs.djangoproject.com/en/1.11/ref/settings/#auth-password-validators
+# https://docs.djangoproject.com/en/5.1/ref/settings/#auth-password-validators
 
 AUTH_PASSWORD_VALIDATORS = [
     {
@@ -105,7 +129,7 @@
 ]
 
 # Internationalization
-# https://docs.djangoproject.com/en/1.11/topics/i18n/
+# https://docs.djangoproject.com/en/5.1/topics/i18n/
 
 LANGUAGE_CODE = 'en-us'
 
@@ -118,7 +142,7 @@
 USE_TZ = True
 
 # Static files (CSS, JavaScript, Images)
-# https://docs.djangoproject.com/en/1.11/howto/static-files/
+# https://docs.djangoproject.com/en/5.1/howto/static-files/
 
 STATIC_URL = '/static/'
 
@@ -159,3 +183,9 @@
         pass
 
 IMPORT_EXPORT_USE_TRANSACTIONS = True
+
+try:
+    from .settings_local import *
+    print("Loaded settings_local.py")
+except ImportError:
+    pass

qlever/urls.py

@@ -1,7 +1,7 @@
 """qlever URL Configuration
 
 The `urlpatterns` list routes URLs to views. For more information please see:
-    https://docs.djangoproject.com/en/1.11/topics/http/urls/
+    https://docs.djangoproject.com/en/5.1/topics/http/urls/
 Examples:
 Function views
     1. Add an import:  from my_app import views

qlever/wsgi.py

@@ -4,7 +4,7 @@
 It exposes the WSGI callable as a module-level variable named ``application``.
 
 For more information on this file, see
-https://docs.djangoproject.com/en/1.11/howto/deployment/wsgi/
+https://docs.djangoproject.com/en/5.1/howto/deployment/wsgi/
 """
 
 import os

requirements.txt

@@ -3,3 +3,4 @@ requests==2.32.0
 django-import-export==3.3.7
 gunicorn==22.0.0
 whitenoise[brotli]==6.6.0
+django-environ==0.11.2