Merge pull request #84 from redjax/dev

Dev

.vscode/settings.json

@@ -103,5 +103,8 @@
     "workbench.editor.tabActionLocation": "left",
     "[shellscript]": {
         "editor.defaultFormatter": "foxundermoon.shell-format"
-    }
+    },
+    "cSpell.words": [
+        "PUID"
+    ]
 }

README.md

@@ -50,14 +50,16 @@ Using this method, any time you make changes to the [`mkdocs.yml`](./mkdocs.yml)
 
 ### Using Docker/Podman
 
-This site includes a [`Dockerfile`](./Dockerfile) and an accompanying [`dev.docker-compose.yml`](./dev.docker-compose.yml) file. You can run the dev server in Docker without installing any dependencies by running `docker compose -f dev.docker-compose.yml up -d --build` (or for Podman, `podman-compose -f dev.docker-compose.yml pull && podman-compose -f dev.docker-compose.yml up -d --build`). This will install `uv`, build the project in your container, and serve the site on a port specified in your [`.env`](./.env.example) file. You can also run the "production" [`docker-compose.yml`](./docker-compose.yml) file, which will build the `mkdocs` site and serve via `nginx` as a static site. Live reloading does not work here, the rendered docs site HTML will simply be served by Nginx. This enables running the docs site from any machine that supports Docker containers, so the site could be hosted on a VPS or a local machine with port forwarding.
+This site includes a [`Dockerfile`](./containers/Dockerfile) and an accompanying [`dev.docker-compose.yml`](./containers/dev.docker-compose.yml) file. You can run the dev server in Docker without installing any dependencies by running `docker compose -f dev.docker-compose.yml up -d --build` (or for Podman, `podman-compose -f dev.docker-compose.yml pull && podman-compose -f dev.docker-compose.yml up -d --build`). This will install `uv`, build the project in your container, and serve the site on a port specified in your [`.env`](./.env.example) file. You can also run the "production" [`docker-compose.yml`](./containers/docker-compose.yml) file, which will build the `mkdocs` site and serve via `nginx` as a static site. Live reloading does not work here, the rendered docs site HTML will simply be served by Nginx. This enables running the docs site from any machine that supports Docker containers, so the site could be hosted on a VPS or a local machine with port forwarding.
 
-To get started developing with Docker, copy [`./.env.example`](./.env.example) to `.env` and (optionally) edit the values. Remember, for local development you will use the `dev.docker-compose.yml` file, and you must remember to add it to all your commands with `-f dev.docker-compose.yml`, otherwise you will build and run the production Docker stack.
+To get started developing with Docker, copy [`./.env.example`](./containers/.env.example) to `.env` and (optionally) edit the values. Remember, for local development you will use the `dev.docker-compose.yml` file, and you must remember to add it to all your commands with `-f dev.docker-compose.yml`, otherwise you will build and run the production Docker stack.
 
 Pull the container images using `docker compose -f dev.docker-compose.yml pull` or `podman-compose -f dev.docker-compose.yml pull`, then build with `docker compose -f dev.docker-compose.yml build` or `podman-compose -f dev.docker-compose.yml build`. Finally, run the containers with `docker compose -f dev.docker-compose.yml up -d` or `podman-compose -f dev.docker-compose.yml up -d`.
 
 **A note on `podman-compose`**: I use Podman instead of Docker on some machines, specifically on Windows devices (because I like its interface more). The biggest shift in expectations for me is around the `podman-compose up` command. With `docker compose up -d`, if a stack is already running, Docker will handle restarting the existing containers. With `podman-compose up -d`, you either need to pass `--force-recreate` every time, or bring the stack down first with `podman-compose down && podman compose up -d`.
 
+Read more about the containers included in this project in the [containers/](./containers) directory.
+
 ### Writing documentation pages
 
 If you are not familiar with writing docs pages in `mkdocs` format, you should check their [guide on writing docs](https://www.mkdocs.org/user-guide/writing-your-docs/). If you have written any Markdown before, the syntax will be very familiar...because that's what it is!

containers/.env.example

@@ -18,6 +18,8 @@ MKDOCS_HTTP_PORT=
 
 ## Default: (named volume) openvscode-config
 CODE_SERVER_CONF_DIR=
+## Default: 3000
+CODE_HTTP_PORT=
 
 ## Default: 80
 NGINX_HTTP_PORT=

containers/README.md

@@ -0,0 +1,29 @@
+# Containers
+
+The root `docker-compose.yml` file builds the MkDocs site in a containerized environment, then serves it with NGINX in the `serve` layer.
+
+The containers in this directory are for local development only.
+
+**Make sure to copy [`.env.example`](./.env.example) -> `.env`, then edit the environment file to set your ports and container image versions.**
+
+## Dockerfiles
+
+### Dockerfile
+
+This is the "main" Dockerfile, which uses a multi-stage build to create the MkDocs site and serve with either `mkdocs serve` (in the `run` layer), or builds the site and copies it to an NGINX reverse proxy to serve the static HTML.
+
+### vscode.Dockerfile
+
+A customized `openvscode-server` image. Installs the SSH and Git packages, as well as a number of VSCode extensions helpful to the development of this site. Serves a VSCode interface in the browser for quickly editing the docs here.
+
+This image is useful is hosting this site on a remote machine where editing over SSH is impossible or difficult.
+
+## Compose stacks
+
+### dev.docker-compose.yml
+
+A local development stack. Serves the MkDocs site from within a Docker container using `mkdocs serve`. The site mounts the project in the container so any changes made to the files will live-reload the MkDocs site.
+
+### web-edit.docker-compose.yml
+
+Includes the MkDocs site, an `openvscode-server` VSCode server, and an NGINX container with a self-signed SSL certificate. **NOTE**: You must generate your SSL certificates/keys using the [`generate_ssl_certificates.sh`](./generate_ssl_certificates.sh) script.**

docs/programming/VSCode/import-export-extensions.md

@@ -0,0 +1,33 @@
+# Import/Export VSCode Extensions
+
+You can export a list of your VSCode extensions to a text file, then loop that text file to install the extensions. This is how plugins like VSCode's "settings sync" work, but you can do it manually.
+
+This is also useful for Docker containers like `openvscode-server`, where you can add a list of VSXi extensions to install from [[open-vsx.org](https://open-vsx.org)].
+
+## Export VSCode extensions
+
+### Linux
+
+```bash title="Export VSCode extensions to text file" linenums="1"
+code --list-extensions > vscode-extensions.list
+```
+
+### Windows
+
+```powershell title="Export VSCode extensions to text file" linenums="1"
+code --list-extensions > vscode-extensions.list
+```
+
+## Import VSCode extensions
+
+### Linux
+
+```bash title="Import VSCode extensions from text file" linenums="1"
+cat  vscode-extensions.list | xargs -L 1 code --install-extension
+```
+
+### Windows
+
+```powershell title="Import VSCode extensions from text file" linenums="1"
+cat vscode-extensions.list |% { code --install-extension $_ }
+```

docs/programming/python/nox/index.md

@@ -87,7 +87,7 @@ def setup_nox_logging(
     """Configure a logger for the Nox module.
 
     Params:
-        level_name (str): The uppercase string repesenting a logging logLevel.
+        level_name (str): The uppercase string representing a logging logLevel.
         disable_loggers (list[str] | None): A list of logger names to disable, i.e. for 3rd party apps.
             Note: Disabling means setting the logLevel to `WARNING`, so you can still see errors.
 
@@ -325,3 +325,115 @@ def run_initial_setup(session: nox.Session):
 ## Extending the noxfile
 
 Check the [`nox_extra` module](nox_extra-module/index.md) for information on extending `nox` with custom sessions.
+
+## noxfile.py helper functions
+
+### Install UV project
+
+You can install your UV project in the `noxfile.py` with function. Copy/paste this somewhere towards the top of your code, then add to any sessions where you want to install the whole project.
+
+```python title="Install UV in session" linenums="1"
+import nox
+
+from pathlib import Path
+
+
+# this VENV_DIR constant specifies the name of the dir that the `dev`
+# session will create, containing the virtualenv;
+# the `resolve()` makes it portable
+VENV_DIR = Path("./.venv").resolve()
+
+
+def install_uv_project(session: nox.Session, external: bool = False) -> None:
+    """Method to install uv and the current project in a nox session."""
+    log.info("Installing uv in session")
+    session.install("uv")
+    log.info("Syncing uv project")
+    session.run("uv", "sync", external=external)
+    log.info("Installing project")
+    session.run("uv", "pip", "install", ".", external=external)
+
+```
+
+Example session using `install_uv_project()`:
+
+```python title="Session that uses install_uv_project()" linenums="1"
+@nox.session(name="dev-env", tags=["setup"])
+def dev(session: nox.Session) -> None:
+    """Sets up a python development environment for the project.
+
+    Run this on a fresh clone of the repository to automate building the project with uv.
+    """
+    install_uv_project(session, external=True)
+
+```
+
+### @cd context manager
+
+This context manager allows you to change the directory a command is run from.
+
+```python title="@cd context manager" linenums="1"
+import nox
+
+from contextlib import contextmanager
+import os
+import importlib.util
+import typing as t
+
+
+@contextmanager
+def cd(new_dir) -> t.Generator[None, importlib.util.Any, None]: # type: ignore
+    """Context manager to change a directory before executing command."""
+    prev_dir: str = os.getcwd()
+    os.chdir(os.path.expanduser(new_dir))
+    try:
+        yield
+    finally:
+        os.chdir(prev_dir)
+
+```
+
+### find_free_port()
+
+This function can find a free port using the `socket` library. This is useful for sessions that run a service, like `mkdocs-serve`. Instead of hard-coding the port and risking an in-use port, this function can find a free port to pass to the function.
+
+```python title="find_free_port()" linenums="1"
+import nox
+import socket
+
+
+def find_free_port(start_port=8000) -> int:
+    """Find a free port starting from a specific port number."""
+    port = start_port
+    while True:
+        with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
+            try:
+                sock.bind(("0.0.0.0", port))
+                return port
+            except socket.error:
+                log.info(f"Port {port} is in use, trying the next port.")
+                port += 1
+
+```
+
+Example usage:
+
+```python title="Example usage for find_free_port()" linenums="1"
+import nox
+
+
+@nox.session(name="serve-mkdocs", tags=["mkdocs", "serve"])
+def serve_mkdocs(session: nox.Session) -> None:
+    install_uv_project(session)
+
+    free_port = find_free_port(start_port=8000)
+
+    log.info(f"Serving MKDocs site on port {free_port}")
+
+    try:
+        session.run("mkdocs", "serve", "--dev-addr", f"0.0.0.0:{free_port}")
+    except Exception as exc:
+        msg = f"({type(exc)}) Unhandled exception serving MKDocs site. Details: {exc}"
+        log.error(msg)
+
+```

docs/template/docker/databases/prometheus/index.md

@@ -0,0 +1,135 @@
+# Prometheus
+
+Prometheus is equal parts database, logging server, and monitoring/alerting. It's a very useful tool for many use cases, for example as a backend for Grafana.
+
+## Directory Structure
+
+```text title="Container directory structure"
+container_root/
+  ../.env
+  ../.gitignore
+  ../docker-compose.yml
+  ../data/prometheus
+    ../alert.rules
+    ../prometheus.yml
+```
+
+## Container Files
+
+### .env
+
+```text title="prometheus .env" linenums="1"
+## Default: 9090
+PROMETHEUS_WEBUI_PORT=
+## Default: ./data/prometheus/prometheus.yml
+PROMETHEUS_CONFIG_FILE=
+## Default: ./data/prometheus/alert.rules
+PROMETHEUS_ALERTMANAGER_RULES_FILE=
+
+```
+
+### .gitignore
+
+```text title="prometheus .gitignore" linenums="1"
+data/*
+data/prometheus/*
+
+!data/prometheus/
+
+!data/prometheus/example.*
+!data/prometheus/example.*.*
+
+!example.*
+!example.*.*
+!*.example
+!*.example.*
+!.*.example
+!.*.example.*
+
+```
+
+### docker-compose.yml
+
+```text title="prometheus docker-compose.yml" linenums="1"
+---
+services:
+  prometheus:
+    image: prom/prometheus:latest
+    container_name: minio-prometheus
+    volumes:
+      - ${PROMETHEUS_CONFIG_FILE:-./data/prometheus/prometheus.yml}:/etc/prometheus/prometheus.yml
+      - ${PROMETHEUS_ALERTMANAGER_RULES_FILE:-./data/prometheus/alert.rules}:/alertmanager/alert.rules
+    command:
+      - "--config.file=/etc/prometheus/prometheus.yml"
+    ports:
+      - ${PROMETHEUS_WEBUI_PORT:-9090}:9090
+
+```
+
+### data/prometheus/alert.rules
+
+```text title="alert.rules" linenums="1"
+groups:
+- name: example
+  rules:
+
+  ## Alert for any instance unreachable for >5 mins
+  - alert: InstanceDown
+    expr: up == 0
+    for: 5m
+
+```
+
+### data/prometheus/prometheus.yml
+
+```yaml title="prometheus.yml" linenums="1"
+global:
+  scrape_interval: 15s
+
+rule_files:
+- "/alertmanager/alert.rules"
+
+scrape_configs:
+  - job_name: "minio-cluster"
+    ## This can be omitted if MinIO has env variable:
+    #    MINIO_PROMETHEUS_AUTH_TYPE="public"
+    # bearer_token: TOKEN
+    metrics_path: /minio/v2/metrics/cluster
+    scheme: https
+    static_configs:
+    - targets: ["minio:9001"]
+
+  - job_name: "minio-nodes"
+    ## This can be omitted if MinIO has env variable:
+    #    MINIO_PROMETHEUS_AUTH_TYPE="public"
+    # bearer_token: TOKEN
+    metrics_path: /minio/v2/metrics/node
+    scheme: https
+    static_configs:
+    - targets: ["minio:9001"]
+
+  - job_name: "minio-bucket"
+    ## This can be omitted if MinIO has env variable:
+    #    MINIO_PROMETHEUS_AUTH_TYPE="public"
+    # bearer_token: TOKEN
+    metrics_path: /minio/v2/metrics/bucket
+    scheme: https
+    static_configs:
+    - targets: ["minio:9001"]
+
+  - job_name: "minio-resource"
+    ## This can be omitted if MinIO has env variable:
+    #    MINIO_PROMETHEUS_AUTH_TYPE="public"
+    # bearer_token: TOKEN
+    metrics_path: /minio/v2/metrics/resource
+    scheme: https
+    static_configs:
+    - targets: ["minio:9001"]
+
+```
+
+## Notes
+
+- The `data/prometheus/` directory (and the files within, `alert.rules` and `prometheus.yml`) do not exist by default, you must create them before starting the container.
+
+## Links

docs/template/docker/documents/index.md

@@ -0,0 +1,3 @@
+# Documents
+
+Containers related to the creation, management, and sending of documents.