Merge branch 'main' into dev-backup-schedules

README.md

@@ -4,6 +4,18 @@
 
 # Lagoon - the developer-focused application delivery platform for Kubernetes
 
+## Table of Contents
+1. Project Description
+2. Usage
+3. Architecture
+4. Testing
+5. Other Lagoon Components
+6. Contribution
+7. History
+8. Connect
+
+## Project Description
+
 Lagoon solves what developers are dreaming about: A system that allows developers to locally develop their code and their services with Docker and run the exact same system in production. The same container images, the same service configurations and the same code.
 
 > Lagoon is an application delivery **platform**. Its primary focus is as a cloud-native tool for the deployment, management, security and operation of many applications. Lagoon greatly reduces the requirement on developers of those applications to have cloud-native experience or knowledge.
@@ -12,7 +24,8 @@ Lagoon has been designed to handle workloads that have been traditionally more c
 
 Lagoon is fully open-source, built on open-source tools, built collaboratively with our users.
 
-## Installing Lagoon
+## Usage
+### Installation
 
 *Note that is not necessary to install Lagoon on to your local machine if you are looking to maintain websites hosted on Lagoon.*
 
@@ -25,7 +38,7 @@ For more information on developing or contributing to Lagoon, head to https://do
 
 For more information on installing and administering Lagoon, head to https://docs.lagoon.sh/administering-lagoon
 
-## Lagoon architecture
+### Architecture
 
 Lagoon comprises two main components: **Lagoon Core** and **Lagoon Remote**. It's also built on several other third-party services, Operators and Controllers. In a full production setting, we recommend installing Lagoon Core and Remote into different Kubernetes Clusters. A single Lagoon Core installation is capable of serving multiple Remotes, but they can also be installed into the same cluster if preferred.
 
@@ -35,7 +48,7 @@ Lagoon services are mostly built in Node.js. More recent development occurs in G
 
 ### Lagoon Core
 
-All the services that handle the API, authentication and external communication are installed here. Installation is via a [Helm Chart](https://github.com/uselagoon/lagoon-charts/tree/main/charts/lagoon-core)
+All the services that handle the API, authentication and external communication are installed here. Installation is via a [Helm Chart].(https://github.com/uselagoon/lagoon-charts/tree/main/charts/lagoon-core)
 - API
   - [api](https://github.com/uselagoon/lagoon/tree/main/services/api) (the GraphQL API that powers Lagoon)
   - [api-db](https://github.com/uselagoon/lagoon/tree/main/services/api-db) (the MariaDB storage for the API)
@@ -102,7 +115,7 @@ These services are usually installed alongside either Lagoon Core or Lagoon Remo
   - [k8up](https://k8up.io/) (provides a scheduled backup and prune service to environment namespaces)
 
 
-### Testing
+## Testing
 
 Lagoon has a comprehensive [test suite](https://github.com/uselagoon/lagoon/tree/main/tests/tests), designed to cover most end-user scenarios. The testing is automated in Ansible, and runs in Jenkins, but can also be run locally in a self-contained cluster. The testing provisions a standalone Lagoon cluster, running on Kind (Kubernetes in Docker). This cluster is made of Lagoon Core, Lagoon Remote, an image registry and a set of managed databases. It runs test deployments and scenarios for a range of Node.js, Drupal, Python and NGINX projects, all built using the latest Lagoon images.
 
@@ -135,7 +148,7 @@ To add the repository `helm repo add lagoon https://amazeeio.github.io/charts/`
 
 
 
-## Contribute
+## Contribution
 
 Do you want to contribute to Lagoon? Fabulous! [See our Documentation](https://docs.lagoon.sh/contributing/) on how to get started.
 

docs/administering-lagoon/graphql-queries.md

@@ -111,7 +111,7 @@ mutation {
       # This is the actual SSH public key (without the type at the beginning and without the comment at the end, ex. `AAAAB3NzaC1yc2EAAAADAQ...3QjzIOtdQERGZuMsi0p`).
       keyValue: ""
       # TODO: Fill in the keyType field.
-      # Valid values are either SSH_RSA or SSH_ED25519.
+      # Valid values are either SSH_RSA, SSH_ED25519, ECDSA_SHA2_NISTP256/384/521
       keyType: SSH_RSA
       user: {
         # TODO: Fill in the userId field.

docs/contributing-to-lagoon/documentation.md

@@ -8,18 +8,23 @@ We use [mkdocs](https://www.mkdocs.org/) with the excellent [Material](https://s
 
 ## Viewing and updating docs locally
 
-From the root of this repo, just run
+From the root of this repo, just run:
 
-`docker run --rm -it -p 8000:8000 -v ${PWD}:/docs squidfunk/mkdocs-material`
+```bash
+docker run --rm -it -p 127.0.0.1:8000:8000 -v ${PWD}:/docs squidfunk/mkdocs-material
+```
 
-and this will start a development server on your local port 8000, configured to livereload on any updates.  The docker image contains all the necessary extensions.
+<!-- markdown-link-check-disable-next-line -->
+This will start a development server on [http://127.0.0.1:8000](http://127.0.0.1:8000), configured to live-reload on any updates.
+The Docker image contains all the necessary extensions.
 
 ## Editing in the cloud
 
 Each documentation page also has an "edit" pencil in the top right, that will take you to the correct page in the git repository.
 
-Feel free to contribute here too - you can always use the inbuilt [github.dev web-based editor](https://docs.github.com/en/codespaces/the-githubdev-web-based-editor). It's got basic markdown previews, but none of the mkdocs loveliness
+Feel free to contribute here too - you can always use the inbuilt [github.dev web-based editor](https://docs.github.com/en/codespaces/the-githubdev-web-based-editor).
+It's got basic markdown previews, but none of the mkdocs loveliness
 
 ## How we deploy documentation
 
-We use the [Deploy MkDocs](https://github.com/marketplace/actions/deploy-mkdocs) GitHub Action to build all main branch pushes, and trigger a deployment of the gh-pages branch.
+We use the [Deploy MkDocs](https://github.com/marketplace/actions/deploy-mkdocs) GitHub Action to build all main branch pushes, and trigger a deployment of the `gh-pages` branch.

docs/docker-images/php-cli/README.md

@@ -35,15 +35,20 @@ This image is prepared to be used on Lagoon. There are therefore some things alr
 The included cli tools are:
 
 * [`composer` version 1.9.0](https://getcomposer.org/) \(changeable via `COMPOSER_VERSION` and `COMPOSER_HASH_SHA256`\)
-* [`node.js` verison 12](https://nodejs.org/en/) \(as of Jan 2020\)
+* [`node.js` verison 17](https://nodejs.org/en/) \(as of Mar 2022\)
 * [`npm`](https://www.npmjs.com/)
 * [`yarn`](https://yarnpkg.com/lang/en/)
 * `mariadb-client`
 * `postgresql-client`
 
 ### Change Node.js Version
 
-By default this image ships with the current Node.js Version \(v12 as of Jan 2020\). If you need another version you can remove the current version and install the one of your choice.
+By default this image ships with the `nodejs-current` package \(v17 as of Mar 2022\). If you need another version you can remove the current version and install the one of your choice. For example, to install Node.js 16, modify your dockerfile to include:
+
+```
+RUN apk del nodejs-current \
+    && apk add --no-cache nodejs=~16
+```
 
 ## Environment variables
 

docs/docker-images/solr/solr-drupal.md

@@ -14,4 +14,4 @@ For each Solr version, there is a specific `solr-drupal:<version>` Docker image.
 * 6.6 \(available for compatibility, no longer officially supported\)
 * 7.7 [Dockerfile](https://github.com/uselagoon/lagoon-images/blob/main/images/solr-drupal/7.7.Dockerfile) (no longer actively supported upstream) - `uselagoon/solr-7.7-drupal`
 * 7 [Dockerfile](https://github.com/uselagoon/lagoon-images/blob/main/images/solr-drupal/7.Dockerfile) - `uselagoon/solr-7-drupal`
-* 7 [Dockerfile](https://github.com/uselagoon/lagoon-images/blob/main/images/solr-drupal/8.Dockerfile) - `uselagoon/solr-8-drupal`
+* 8 [Dockerfile](https://github.com/uselagoon/lagoon-images/blob/main/images/solr-drupal/8.Dockerfile) - `uselagoon/solr-8-drupal`

docs/installing-lagoon/efs-provisioner.md

@@ -5,21 +5,20 @@
 
 1. Add Helm repository: `helm repo add stable https://charts.helm.sh/stable`
 2. Create `efs-provisioner-values.yml` in your config directory and update the values:
+    ```yaml title="efs-provisioner-values.yml"
+    efsProvisioner:
+      efsFileSystemId: <efsFileSystemId>
+      awsRegion: <awsRegion>
+      path: /
+      provisionerName: example.com/aws-efs
+      storageClass:
+        name: bulk
+        isDefault: false
+        reclaimPolicy: Delete
+        mountOptions: []
+    global:
+      deployEnv: prod
 
-```yaml title="efs-provisioner-values.yml"
-efsProvisioner:
-  efsFileSystemId: <efsFileSystemId>
-  awsRegion: <awsRegion>
-  path: /
-  provisionerName: example.com/aws-efs
-  storageClass:
-    name: bulk
-    isDefault: false
-    reclaimPolicy: Delete
-    mountOptions: []
-global:
-  deployEnv: prod
+    ```
 
-```
-
-3\. Install EFS Provisioner:`helm upgrade --install --create-namespace --namespace efs-provisioner -f efs-provisioner-values.yaml  efs-provisioner stable/efs-provisioner`
+3. Install EFS Provisioner:`helm upgrade --install --create-namespace --namespace efs-provisioner -f efs-provisioner-values.yaml  efs-provisioner stable/efs-provisioner`

docs/installing-lagoon/install-harbor.md

@@ -3,40 +3,39 @@
 1. Add Helm repo: `helm repo add harbor https://helm.goharbor.io`
 2. Create the file `harbor-values.yml` inside of your config directory:
 
-```yaml title="harbor-values.yml"
-expose:
-  ingress:
-    annotations:
-      kubernetes.io/tls-acme: "true"
-    hosts:
-     core: harbor.lagoon.example.com
-  tls:
-    enabled: true
-    certSource: secret
-    secret:
-      secretName: harbor-harbor-ingress
-externalURL: https://harbor.lagoon.example.com
-harborAdminPassword: <your Harbor Admin Password>
-chartmuseum:
-  enabled: false
-clair:
-  enabled: false
-notary:
-  enabled: false
-trivy:
-  enabled: false
-jobservice:
-  jobLogger: stdout
-registry:
-  replicas: 1
+    ```yaml title="harbor-values.yml"
+    expose:
+      ingress:
+        annotations:
+          kubernetes.io/tls-acme: "true"
+        hosts:
+        core: harbor.lagoon.example.com
+      tls:
+        enabled: true
+        certSource: secret
+        secret:
+          secretName: harbor-harbor-ingress
+    externalURL: https://harbor.lagoon.example.com
+    harborAdminPassword: <your Harbor Admin Password>
+    chartmuseum:
+      enabled: false
+    clair:
+      enabled: false
+    notary:
+      enabled: false
+    trivy:
+      enabled: false
+    jobservice:
+      jobLogger: stdout
+    registry:
+      replicas: 1
 
-```
+    ```
 
-1. Install Harbor:`helm upgrade --install --create-namespace --namespace harbor --wait -f harbor-values.yaml --version=1.5.2 harbor harbor/harbor`
-   1. We are currently using Harbor version 1.5.2. A recent update to Harbor breaks the API.
+1. Install Harbor:`helm upgrade --install --create-namespace --namespace harbor --wait -f harbor-values.yaml --version=1.5.6 harbor harbor/harbor`
+   1. We are currently using Harbor version 1.5.6. A recent update to Harbor (Harbor 2.2) breaks the API.
 2. Visit Harbor at the URL you set in `harbor.yml`.
    1. Username: admin
    2. Password:
        `kubectl -n harbor get secret harbor-harbor-core -o jsonpath="{.data.HARBOR_ADMIN_PASSWORD}" | base64 --decode`
-3. Add the above Harbor credentials to the Lagoon Core `values.yml` that you created at the beginning of the process, as well as `harbor-values.yml`.
-4. Upgrade lagoon-core release with the updated `values.yml` file: `helm upgrade --namespace lagoon-core -f values.yaml lagoon-core lagoon/lagoon-core`
+3. You will need to add the above Harbor credentials to the Lagoon Remote `values.yml` in the next step, as well as `harbor-values.yml`.

docs/installing-lagoon/install-lagoon-remote.md

@@ -3,46 +3,48 @@
 Now we will install Lagoon Remote into the Lagoon namespace. The [RabbitMQ](../docker-images/rabbitmq.md) service is the broker.
 
 1. Create `remote-values.yml` in your config directory as you did the previous two files, and update the values.
-   1. rabbitMQPassword: `kubectl -n lagoon-core get secret lagoon-core-broker -o jsonpath="{.data.RABBITMQ_PASSWORD}" | base64 --decode`
-   2. rabbitMQHostname: `lagoon-core-broker.lagoon-core.svc.local`
-   3. taskSSHHost: `kubectl get service lagoon-core-broker-amqp-ext -o custom-columns="NAME:.metadata.name,IP ADDRESS:.status.loadBalancer.ingress[*].ip,HOSTNAME:.status.loadBalancer.ingress[*].hostname"`
-2. Run `helm upgrade --install --create-namespace --namespace lagoon -f remote-values.yaml  lagoon-remote lagoon/lagoon-remote`
+    * **rabbitMQPassword** `kubectl -n lagoon-core get secret lagoon-core-broker -o jsonpath="{.data.RABBITMQ_PASSWORD}" | base64 --decode`
+    * **rabbitMQHostname** `lagoon-core-broker.lagoon-core.svc.local`
+    * **taskSSHHost** `kubectl get service lagoon-core-broker-amqp-ext -o custom-columns="NAME:.metadata.name,IP ADDRESS:.status.loadBalancer.ingress[*].ip,HOSTNAME:.status.loadBalancer.ingress[*].hostname"`
+    * **harbor-password** `kubectl -n harbor get secret harbor-harbor-core -o jsonpath="{.data.HARBOR_ADMIN_PASSWORD}" | base64 --decode`
+2. Add the Harbor configuration from the previous step.
+3. Run `helm upgrade --install --create-namespace --namespace lagoon -f remote-values.yaml  lagoon-remote lagoon/lagoon-remote`
 
-```yaml title="remote-values.yml"
-lagoon-build-deploy:
-  enabled: true
-  extraArgs:
-    - "--enable-harbor=true"
-    - "--harbor-url=https://harbor.lagoon.example.com"
-    - "--harbor-api=https://harbor.lagoon.example.com/api/"
-    - "--harbor-username=admin"
-    - "--harbor-password=<HarborAdminPassword>"
-  rabbitMQUsername: lagoon
-  rabbitMQPassword: <from lagoon-core-broker secret>
-  rabbitMQHostname: lagoon-core-broker.lagoon-core.svc.cluster.local
-  lagoonTargetName: <name of lagoon remote, can be anything>
-  taskSSHHost: <IP of ssh service loadbalancer>
-  taskSSHPort: "22"
-  taskAPIHost: "api.lagoon.example.com"
-dbaas-operator:
-  enabled: true
+    ```yaml title="lagoon-remote-values.yml"
+    lagoon-build-deploy:
+      enabled: true
+      extraArgs:
+        - "--enable-harbor=true"
+        - "--harbor-url=https://harbor.lagoon.example.com"
+        - "--harbor-api=https://harbor.lagoon.example.com/api/"
+        - "--harbor-username=admin"
+        - "--harbor-password=<from harbor-harbor-core secret>"
+      rabbitMQUsername: lagoon
+      rabbitMQPassword: <from lagoon-core-broker secret>
+      rabbitMQHostname: lagoon-core-broker.lagoon-core.svc.cluster.local
+      lagoonTargetName: <name of lagoon remote, can be anything>
+      taskSSHHost: <IP of ssh service loadbalancer>
+      taskSSHPort: "22"
+      taskAPIHost: "api.lagoon.example.com"
+    dbaas-operator:
+      enabled: true
 
-  mariadbProviders:
-    production:
-      environment: production
-      hostname: 172.17.0.1.nip.io
-      readReplicaHostnames:
-      - 172.17.0.1.nip.io
-      password: password
-      port: '3306'
-      user: root
+      mariadbProviders:
+        production:
+          environment: production
+          hostname: 172.17.0.1.nip.io
+          readReplicaHostnames:
+          - 172.17.0.1.nip.io
+          password: password
+          port: '3306'
+          user: root
 
-    development:
-      environment: development
-      hostname: 172.17.0.1.nip.io
-      readReplicaHostnames:
-      - 172.17.0.1.nip.io
-      password: password
-      port: '3306'
-      user: root
-```
+        development:
+          environment: development
+          hostname: 172.17.0.1.nip.io
+          readReplicaHostnames:
+          - 172.17.0.1.nip.io
+          password: password
+          port: '3306'
+          user: root
+    ```

docs/installing-lagoon/lagoon-backups.md

@@ -12,6 +12,7 @@ Lagoon uses the k8up backup operator: [https://k8up.io](https://k8up.io). Lagoon
     `kubectl apply -f https://github.com/vshn/k8up/releases/download/v1.1.0/k8up-crd.yaml`
 
     `helm upgrade --install --create-namespace --namespace k8up -f k8up-values.yaml k8up appuio/k8up`
+
 4. Update `lagoon-core-values.yaml`:
 
       ```yaml title="lagoon-core-values.yaml"

docs/installing-lagoon/requirements.md

@@ -11,8 +11,32 @@
 !!! Note "Note:"
     We acknowledge that this is a lot of steps, and our roadmap for the immediate future includes reducing the number of steps in this process.
 
+## Specific requirements (as of March 2022)
+
+### Kubernetes
+Lagoon supports Kubernetes versions 1.19, 1.20 and 1.21. Support for 1.22 is underway, and mostly complete. There are a number of relevant API deprecations in 1.22 that Lagoon utilized across a number of dependencies.
+
+### ingress-nginx
+Lagoon is currently only for a single ingress-nginx controller, and therefore defining an IngressClass has not been necessary.
+
+This means that Lagoon currently works best with version 3 of the ingress-nginx Helm chart - latest release [3.40.0](https://github.com/kubernetes/ingress-nginx/releases/tag/helm-chart-3.40.0).
+
+In order to use a version of the Helm chart (>=4) that supports Ingress v1 (i.e for Kubernetes 1.22), the following configuration should be used, as per [the ingress-nginx docs](https://kubernetes.github.io/ingress-nginx/#what-is-an-ingressclass-and-why-is-it-important-for-users-of-ingress-nginx-controller-now).
+
+- nginx-ingress should be configured as the default controller - set `.controller.ingressClassResource.default: true` in Helm values
+- nginx-ingress should be configured to watch ingresses without IngressClass set - set `.controller.watchIngressWithoutClass: true` in Helm values
+
+This will configure the controller to create any new ingresses with itself as the IngressClass, and also to handle any existing ingresses without an IngressClass set.
+
+Other configurations may be possible, but have not been tested.
+
+### Harbor
+Only Harbor <2.2 is currently supported - the method of retrieving robot accounts was changed in 2.2, and we are working on a fix.
+
+This means you should install Harbor [2.1.6](https://github.com/goharbor/harbor/releases/tag/v2.1.6) with Helm chart [1.5.6](https://github.com/goharbor/harbor-helm/releases/tag/1.5.6).
+
 ## How much Kubernetes experience/knowledge is required?
 
-Lagoon uses some very involved Kubernetes and Cloud Native concepts, and whilst full familiarity may not be necessary to install and configure Lagoon, diagnosing issues and contributing may prove difficult without a good level of familiarity.
+Lagoon uses some very involved Kubernetes and Cloud Native concepts, and while full familiarity may not be necessary to install and configure Lagoon, diagnosing issues and contributing may prove difficult without a good level of familiarity.
 
 As an indicator, comfort with the curriculum for the [Certified Kubernetes Administrator](https://www.cncf.io/certification/cka/) would be suggested as a minimum.