v11.0.0 (2020-12-09)

- 💥[Improvement] Upgrade Open edX to Koa - 💥 Setting changes: - The ``ACTIVATE_HTTPS`` setting was renamed to ``ENABLE_HTTPS``. - Other ``ACTIVATE_*`` variables were all renamed to ``RUN_*``. - The ``WEB_PROXY`` setting was removed and ``RUN_CADDY`` was added. - The ``NGINX_HTTPS_PORT`` setting is deprecated. - Architectural changes: - Use Caddy as a web proxy for automated SSL/TLS certificate generation: - Nginx no longer listens to port 443 for https traffic - The Caddy configuration file comes with a new ``caddyfile`` patch for much simpler SSL/TLS management. - Configuration files for web proxies are no longer provided. - Kubernetes deployment no longer requires setting up a custom Ingress resource or custom manager. - Gunicorn and Whitenoise are replaced by uwsgi: this increases boostrap performance and makes it no longer necessary to mount media folders in the Nginx container. - Replace memcached and rabbitmq by redis. - Additional features: - Make it possible to disable all plugins at once with ``plugins disable all``. - Add ``tutor k8s wait`` command to wait for a pod to become ready - Faster, more reliable static assets with local memory caching - Deprecation: proxy files for Apache and Nginx are no longer provided out of the box. - Removed plugin `{{ patch (...) }}` statements: - "https-create", "k8s-ingress-rules", "k8s-ingress-tls-hosts": these are no longer necessary. Instead, declare your app in the "caddyfile" patch. - "local-docker-compose-nginx-volumes": this patch was primarily used to serve media assets. The recommended is now to serve assets with uwsgi.
overhangio · Dec 10, 2020 · 728ef96 · 728ef96
1 parent d3c842c
commit 728ef96
Show file tree

Hide file tree

Showing 61 changed files with 659 additions and 755 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -4,6 +4,31 @@ Note: Breaking changes between versions are indicated by "💥".
 
 ## Unreleased
 
+## v11.0.0 (2020-12-09)
+
+- 💥[Improvement] Upgrade Open edX to Koa
+    - 💥 Setting changes:
+        - The ``ACTIVATE_HTTPS`` setting was renamed to ``ENABLE_HTTPS``.
+        - Other ``ACTIVATE_*`` variables were all renamed to ``RUN_*``.
+        - The ``WEB_PROXY`` setting was removed and ``RUN_CADDY`` was added.
+        - The ``NGINX_HTTPS_PORT`` setting is deprecated.
+    - Architectural changes:
+        - Use Caddy as a web proxy for automated SSL/TLS certificate generation:
+            - Nginx no longer listens to port 443 for https traffic
+            - The Caddy configuration file comes with a new ``caddyfile`` patch for much simpler SSL/TLS management.
+            - Configuration files for web proxies are no longer provided.
+            - Kubernetes deployment no longer requires setting up a custom Ingress resource or custom manager.
+        - Gunicorn and Whitenoise are replaced by uwsgi: this increases boostrap performance and makes it no longer necessary to mount media folders in the Nginx container.
+        - Replace memcached and rabbitmq by redis.
+    - Additional features:
+        - Make it possible to disable all plugins at once with ``plugins disable all``.
+        - Add ``tutor k8s wait`` command to wait for a pod to become ready
+        - Faster, more reliable static assets with local memory caching
+    - Deprecation: proxy files for Apache and Nginx are no longer provided out of the box.
+    - Removed plugin `{{ patch (...) }}` statements:
+        - "https-create", "k8s-ingress-rules", "k8s-ingress-tls-hosts": these are no longer necessary. Instead, declare your app in the "caddyfile" patch.
+        - "local-docker-compose-nginx-volumes": this patch was primarily used to serve media assets. The recommended is now to serve assets with uwsgi.
+
 ## v10.5.3 (2020-12-09)
 
 - [Security] Apply upstream edx-platform [security patch](https://github.com/edx/edx-platform/pull/25782)

diff --git a/Makefile b/Makefile
@@ -89,10 +89,10 @@ ci-test-bundle: ## Run basic tests on bundle
 	yes "" | ./dist/tutor config save --interactive
 	./dist/tutor config save
 	./dist/tutor plugins list
-	# ./dist/tutor plugins enable discovery ecommerce figures lts minio notes xqueue
-	./dist/tutor plugins enable discovery ecommerce lts minio notes xqueue
+	# ./dist/tutor plugins enable discovery ecommerce figures license minio notes xqueue
+	./dist/tutor plugins enable discovery ecommerce license minio notes xqueue
 	./dist/tutor plugins list
-	./dist/tutor lts --help
+	./dist/tutor license --help
 
 ./releases/github-release: ## Download github-release binary
 	mkdir -p releases/

diff --git a/README.rst b/README.rst
@@ -34,7 +34,7 @@ Tutor: the docker-based Open edX distribution designed for peace of mind
 
 **Tutor** is a docker-based `Open edX <https://openedx.org>`_ distribution, both for production and local development. The goal of Tutor is to make it easy to deploy, customize, upgrade and scale Open edX. Tutor is reliable, fast, extensible, and it is already used by dozens of Open edX platforms around the world.
 
-Do you need professional assistance setting up or managing your Open edX platform? Overhang.IO provides online support as part of its `Long Term Support (LTS) offering <https://overhang.io/tutor/lts>`__.
+Do you need professional assistance setting up or managing your Open edX platform? Overhang.IO provides online support as part of its `Long Term Support (LTS) offering <https://overhang.io/tutor/pricing>`__.
 
 Features
 --------
@@ -46,7 +46,7 @@ Features
 * Extensible architecture with `plugins <https://docs.tutor.overhang.io/plugins.html>`__
 * Works with `Kubernetes <https://docs.tutor.overhang.io/k8s.html>`__
 * No technical skill required with the `1-click Tutor AWS image <https://docs.tutor.overhang.io/install.html#cloud-deployment>`__
-* Professional support and premium plugins available with `Tutor Long Term Support (LTS) <https://overhang.io/tutor/lts>`__
+* Amazing plugins available with `Tutor Wizard Edition <https://overhang.io/tutor>`__
 
 .. _readme_intro_end:
 
@@ -71,7 +71,7 @@ Extensive documentation is available online: https://docs.tutor.overhang.io/
 Support
 -------
 
-To get community support, go to the official discussion forums: https://discuss.overhang.io. For official support, please subscribe to a Long Term Support (LTS) license at https://overhang.io/tutor/lts.
+To get community support, go to the official discussion forums: https://discuss.overhang.io. For official support, please subscribe to a Long Term Support (LTS) license at https://overhang.io/tutor/pricing.
 
 .. _readme_support_end:
 

diff --git a/bin/main.py b/bin/main.py
@@ -6,7 +6,7 @@
     "discovery",
     "ecommerce",
     # "figures",
-    "lts",
+    "license",
     "minio",
     "notes",
     "xqueue",

diff --git a/docs/configuration.rst b/docs/configuration.rst
@@ -40,16 +40,15 @@ With an up-to-date environment, Tutor is ready to launch an Open edX platform an
 Individual service activation
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-- ``ACTIVATE_LMS`` (default: ``true``)
-- ``ACTIVATE_CMS`` (default: ``true``)
-- ``ACTIVATE_FORUM`` (default: ``true``)
-- ``ACTIVATE_ELASTICSEARCH`` (default: ``true``)
-- ``ACTIVATE_MEMCACHED`` (default: ``true``)
-- ``ACTIVATE_MONGODB`` (default: ``true``)
-- ``ACTIVATE_MYSQL`` (default: ``true``)
-- ``ACTIVATE_RABBITMQ`` (default: ``true``)
-- ``ACTIVATE_SMTP`` (default: ``true``)
-- ``ACTIVATE_HTTPS`` (default: ``false``)
+- ``RUN_LMS`` (default: ``true``)
+- ``RUN_CMS`` (default: ``true``)
+- ``RUN_FORUM`` (default: ``true``)
+- ``RUN_ELASTICSEARCH`` (default: ``true``)
+- ``RUN_MONGODB`` (default: ``true``)
+- ``RUN_MYSQL`` (default: ``true``)
+- ``RUN_REDIS`` (default: ``true``)
+- ``RUN_SMTP`` (default: ``true``)
+- ``ENABLE_HTTPS`` (default: ``false``)
 
 Every single Open edX service may be (de)activated at will by these configuration parameters. This is useful if you want, for instance, to distribute the various Open edX services on different servers.
 
@@ -81,40 +80,45 @@ You may want to pull/push images from/to a custom docker registry. For instance,
 Open edX customisation
 ~~~~~~~~~~~~~~~~~~~~~~
 
-- ``OPENEDX_COMMON_VERSION`` (default: ``"open-release/juniper.3"``)
+- ``OPENEDX_COMMON_VERSION`` (default: ``"open-release/koa.1"``)
 
 This defines the default version that will be pulled from all Open edX git repositories.
 
-- ``OPENEDX_CMS_GUNICORN_WORKERS`` (default: ``2``)
-- ``OPENEDX_LMS_GUNICORN_WORKERS`` (default: ``2``)
+- ``OPENEDX_CMS_UWSGI_WORKERS`` (default: ``2``)
+- ``OPENEDX_LMS_UWSGI_WORKERS`` (default: ``2``)
 
-By default there are 2 `gunicorn worker processes <https://docs.gunicorn.org/en/stable/settings.html#worker-processes>`__ to serve requests for the LMS and the CMS. However, each workers requires upwards of 500 Mb of RAM. You should reduce this value to 1 if your computer/server does not have enough memory.
+By default there are 2 `uwsgi worker processes <https://uwsgi-docs.readthedocs.io/en/latest/Options.html#processes>`__ to serve requests for the LMS and the CMS. However, each workers requires upwards of 500 Mb of RAM. You should reduce this value to 1 if your computer/server does not have enough memory.
 
 
 Vendor services
 ~~~~~~~~~~~~~~~
 
+Caddy
+*****
+
+- ``RUN_CADDY`` (default: ``true``)
+
+`Caddy <https://caddyserver.com>`__ is a web server used in Tutor as a web proxy for the generation of SSL/TLS certificates at runtime. If ``RUN_CADDY`` is set to ``false`` then we assume that SSL termination does not occur in the Caddy container, and thus the ``caddy`` container is not started.
+
 Nginx
 *****
 
 - ``NGINX_HTTP_PORT`` (default: ``80``)
-- ``NGINX_HTTPS_PORT`` (default: ``443``)
-- ``WEB_PROXY`` (default: ``false``)
 
-Nginx is used to route web traffic to the various applications and to serve static assets. In case there is another web server in front of the Nginx container (for instance, a web server running on the host or an Ingress controller on Kubernetes), the container exposed ports can be modified. If ``WEB_PROXY`` is set to ``true`` then we assume that SSL termination does not occur in the Nginx container.
+Nginx is used to route web traffic to the various applications and to serve static assets. When ``RUN_CADDY`` is false, the ``NGINX_HTTP_PORT`` is exposed on the host.
 
 MySQL
 *****
 
-- ``ACTIVATE_MYSQL`` (default: ``true``)
+- ``RUN_MYSQL`` (default: ``true``)
 - ``MYSQL_HOST`` (default: ``"mysql"``)
 - ``MYSQL_PORT`` (default: ``3306``)
 - ``MYSQL_ROOT_USERNAME`` (default: ``"root"``)
 - ``MYSQL_ROOT_PASSWORD`` (default: randomly generated) Note that you are responsible for creating the root user if you are using a managed database.
 
 By default, a running Open edX platform deployed with Tutor includes all necessary 3rd-party services, such as MySQL, MongoDb, etc. But it's also possible to store data on a separate database, such as `Amazon RDS <https://aws.amazon.com/rds/>`_. For instance, to store data on an external MySQL database, set the following configuration::
 
-    ACTIVATE_MYSQL: false
+    RUN_MYSQL: false
     MYSQL_HOST: yourhost
     MYSQL_ROOT_USERNAME: <root user name>
     MYSQL_ROOT_PASSWORD: <root user password>
@@ -127,34 +131,31 @@ Elasticsearch
 - ``ELASTICSEARCH_PORT`` (default: ``9200``)
 - ``ELASTICSEARCH_HEAP_SIZE`` (default: ``"1g"``)
 
-Memcached
-*********
-
-- ``MEMCACHED_HOST`` (default: ``"memcached"``)
-- ``MEMCACHED_PORT`` (default: ``11211``)
-
 Mongodb
 *******
 
-- ``ACTIVATE_MONGODB`` (default: ``true``)
+- ``RUN_MONGODB`` (default: ``true``)
 - ``MONGODB_HOST`` (default: ``"mongodb"``)
 - ``MONGODB_DATABASE`` (default: ``"openedx"``)
 - ``MONGODB_PORT`` (default: ``27017``)
 - ``MONGODB_USERNAME`` (default: ``""``)
 - ``MONGODB_PASSWORD`` (default: ``""``)
 
-Rabbitmq
-********
+Redis
+*****
+
+- ``RUN_REDIS`` (default: ``true``)
+- ``REDIS_HOST`` (default: ``"redis"``)
+- ``REDIS_HOST`` (default: ``6379``)
+- ``REDIS_USERNAME`` (default: ``""``)
+- ``REDIS_PASSWORD`` (default: ``""``)
 
-- ``ACTIVATE_RABBITMQ`` (default: ``true``)
-- ``RABBITMQ_HOST`` (default: ``"rabbitmq"``)
-- ``RABBITMQ_USERNAME`` (default: ``""``)
-- ``RABBITMQ_PASSWORD`` (default: ``""``)
+Note that Redis has replaced Rabbitmq as the Celery message broker since Tutor v11.0.0.
 
 SMTP
 ****
 
-- ``ACTIVATE_SMTP`` (default: ``true``)
+- ``RUN_SMTP`` (default: ``true``)
 - ``SMTP_HOST`` (default: ``"smtp"``)
 - ``SMTP_PORT`` (default: ``25``)
 - ``SMTP_USERNAME`` (default: ``""``)
@@ -167,7 +168,7 @@ Note that the SMTP server shipped with Tutor by default does not implement TLS.
 SSL/TLS certificates for HTTPS access
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-- ``ACTIVATE_HTTPS`` (default: ``false``)
+- ``ENABLE_HTTPS`` (default: ``false``)
 
 By activating this feature, a free SSL/TLS certificate from the `Let's Encrypt <https://letsencrypt.org/>`_ certificate authority will be created for your platform. With this feature, **your platform will no longer be accessible in HTTP**. Calls to http urls will be redirected to https url.
 
@@ -179,15 +180,7 @@ The following DNS records must exist and point to your server::
 
 Thus, **this feature will (probably) not work in development** because the DNS records will (probably) not point to your development machine.
 
-To create the certificate manually, run::
-
-    tutor local https create
-
-To renew the certificate, run this command once per month::
-
-    tutor local stop nginx
-    tutor local https renew
-    tutor local start -d
+The SSL/TLS certificates will automatically be generated and updated by the Caddy proxy server container at runtime. Thus, as of v11.0.0 you no longer have to generate the certificates manually.
 
 .. _customise:
 
@@ -214,7 +207,7 @@ openedx Docker Image build arguments
 When building the "openedx" Docker image, it is possible to specify a few `arguments <https://docs.docker.com/engine/reference/builder/#arg>`__:
 
 - ``EDX_PLATFORM_REPOSITORY`` (default: ``"https://github.com/edx/edx-platform.git"``)
-- ``EDX_PLATFORM_VERSION`` (default: ``"open-release/juniper.3"``)
+- ``EDX_PLATFORM_VERSION`` (default: ``"open-release/koa.1"``)
 - ``EDX_PLATFORM_VERSION_DATE`` (default: ``"20200227"``)
 - ``NPM_REGISTRY`` (default: ``"https://registry.npmjs.org/"``)
 
@@ -286,16 +279,16 @@ Note that your edx-platform version must be a fork of the latest release **tag**
 
 If you don't create your fork from this tag, you *will* have important compatibility issues with other services. In particular:
 
-- Do not try to run a fork from an older (pre-Juniper) version of edx-platform: this will simply not work.
+- Do not try to run a fork from an older (pre-Koa) version of edx-platform: this will simply not work.
 - Do not try to run a fork from the edx-platform master branch: there is a 99% probability that it will fail.
-- Do not try to run a fork from the open-release/juniper.master branch: Tutor will attempt to apply security and bug fix patches that might already be included in the open-release/juniper.master but which were not yet applied to the latest release tag. Patch application will thus fail if you base your fork from the open-release/juniper.master branch.
+- Do not try to run a fork from the open-release/koa.master branch: Tutor will attempt to apply security and bug fix patches that might already be included in the open-release/koa.master but which were not yet applied to the latest release tag. Patch application will thus fail if you base your fork from the open-release/koa.master branch.
 
 .. _i18n:
 
 Adding custom translations
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-If you are not running Open edX in English, chances are that some strings will not be properly translated. In most cases, this is because not enough contributors have helped translate Open edX in your language. It happens! With Tutor, available translated languages include those that come bundled with `edx-platform <https://github.com/edx/edx-platform/tree/open-release/juniper.3/conf/locale>`__ as well as those from `openedx-i18n <https://github.com/openedx/openedx-i18n/tree/master/edx-platform/locale>`__.
+If you are not running Open edX in English, chances are that some strings will not be properly translated. In most cases, this is because not enough contributors have helped translate Open edX in your language. It happens! With Tutor, available translated languages include those that come bundled with `edx-platform <https://github.com/edx/edx-platform/tree/open-release/koa.1/conf/locale>`__ as well as those from `openedx-i18n <https://github.com/openedx/openedx-i18n/tree/master/edx-platform/locale>`__.
 
 Tutor offers a relatively simple mechanism to add custom translations to the openedx Docker image. You should create a folder that corresponds to your language code in the "build/openedx/locale" folder of the Tutor environment. This folder should contain a "LC_MESSAGES" folder. For instance::
 
@@ -308,9 +301,9 @@ Then, add a "django.po" file there that will contain your custom translations::
     msgid "String to translate"
     msgstr "你翻译的东西 la traduction de votre bidule"
 
-The "String to translate" part should match *exactly* the string that you would like to translate. You cannot make it up! The best way to find this string is to copy-paste it from the `upstream django.po file for the English language <https://github.com/edx/edx-platform/blob/open-release/juniper.3/conf/locale/en/LC_MESSAGES/django.po>`__.
+The "String to translate" part should match *exactly* the string that you would like to translate. You cannot make it up! The best way to find this string is to copy-paste it from the `upstream django.po file for the English language <https://github.com/edx/edx-platform/blob/open-release/koa.1/conf/locale/en/LC_MESSAGES/django.po>`__.
 
-If you cannot find the string to translate in this file, then it means that you are trying to translate a string that is used in some piece of javascript code. Those strings are stored in a different file named "djangojs.po". You can check it out `in the edx-platform repo as well <https://github.com/edx/edx-platform/blob/open-release/juniper.3/conf/locale/en/LC_MESSAGES/djangojs.po>`__. Your custom javascript strings should also be stored in a "djangojs.po" file that should be placed in the same directory.
+If you cannot find the string to translate in this file, then it means that you are trying to translate a string that is used in some piece of javascript code. Those strings are stored in a different file named "djangojs.po". You can check it out `in the edx-platform repo as well <https://github.com/edx/edx-platform/blob/open-release/koa.1/conf/locale/en/LC_MESSAGES/djangojs.po>`__. Your custom javascript strings should also be stored in a "djangojs.po" file that should be placed in the same directory.
 
 To recap, here is an example. To translate a few strings in French, both from django.po and djangojs.po, we would have the following file hierarchy::
 

diff --git a/docs/dev.rst b/docs/dev.rst
@@ -25,7 +25,7 @@ This ``openedx-dev`` development image differs from the ``openedx`` production i
 
 - The user that runs inside the container has the same UID as the user on the host, in order to avoid permission problems inside mounted volumes (and in particular in the edx-platform repository).
 - Additional python and system requirements are installed for convenient debugging: `ipython <https://ipython.org/>`__, `ipdb <https://pypi.org/project/ipdb/>`__, vim, telnet.
-- The edx-platform `development requirements <https://github.com/edx/edx-platform/blob/open-release/juniper.3/requirements/edx/development.in>`__ are installed.
+- The edx-platform `development requirements <https://github.com/edx/edx-platform/blob/open-release/koa.1/requirements/edx/development.in>`__ are installed.
 
 Since the ``openedx-dev`` is based upon the ``openedx`` docker image, it should be re-built every time the ``openedx`` docker image is modified.
 
@@ -68,11 +68,11 @@ If you don't want to rewrite this option every time, you can define a command al
     alias tutor-dev-run-lms="tutor dev run -v /path/to/edx-platform:/openedx/edx-platform lms"
 
 For technical reasons, the ``-v`` option is only supported for the ``run`` and ``runserver`` commands. With these commands, only one service is started. But there are cases where you may want to launch and debug a complete Open edX platform with ``tutor dev start`` and mount a custom edx-platform fork. For instance, this might be needed when testing the interaction between multiple services. To do so, you should create a ``docker-compose.override.yml`` file that will specify a custom volume to be used with all ``dev`` commands::
-    
+
     vim "$(tutor config printroot)/env/dev/docker-compose.override.yml"
 
 Then, add the following content::
-    
+
     version: "3.7"
     services:
       lms:
@@ -90,7 +90,7 @@ Then, add the following content::
 
 This override file will be loaded when running any ``tutor dev ..`` command. The edx-platform repo mounted at the specified path will be automaticall mounted inside all LMS and CMS containers. With this file, you should no longer specify the ``-v`` option from the command line with the ``run`` or ``runserver`` commands.
 
-**Note:** containers are built on the Juniper release. If you are working on a different version of Open edX, you will have to rebuild the ``openedx`` docker images with the version. See the :ref:`fork edx-platform section <edx_platform_fork>`.
+**Note:** containers are built on the Koa release. If you are working on a different version of Open edX, you will have to rebuild the ``openedx`` docker images with the version. See the :ref:`fork edx-platform section <edx_platform_fork>`.
 
 Prepare the edx-platform repo
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~