Tutor v3 complete rewrite

Replace all make commands by a single "tutor" binary. Environment and
data are all moved to ~/.tutor/local/share/tutor. We take the
opportunity to add a web UI and revamp the documentation.

This is a complete rewrite.

Close #121.
Close #147.

.github/ISSUE_TEMPLATE.md

@@ -6,8 +6,8 @@ Include the exact command that you are running and that is causing an error. In
 
 ## Unexpected behavior
 
-Include the full, exact output from the command that is causing your issue. Also include relevant error logs; for instance, to debug the LMS take a look at the files in `data/lms/logs`.
+Include the full, exact output from the command that is causing your issue. Also include relevant error logs; for instance, to debug the LMS provide the output of `tutor local logs lms --tail=100`
 
 ## Additional info (IMPORTANT)
 
-Include the output of the `make info` command.
+Provide the output of `tutor --version`.

.gitignore

@@ -1,5 +1,7 @@
 .*.swp
 !.gitignore
-/config.json
-/data*/
 TODO
+
+/build/tutor
+/dist/
+/tutor_openedx.egg-info/

.travis.yml

@@ -1,11 +1,72 @@
-language: minimal
-services:
-  - docker
+language: python
+matrix:
+  include:
+    - os: linux
+      # We need an older version of python in order to have compatibility with
+      # older versions of libc
+      dist: trusty
+      python: 3.6
+      services:
+        - docker
+      env:
+        BUILD_BINARY: "true"
+    - os: linux
+      dist: xenial
+      python: 3.6
+      services:
+        - docker
+      env:
+        BUILD_DOCKER: "true"
+    - os: linux
+      dist: xenial
+      python: 3.6
+      services:
+        - docker
+      env:
+        BUILD_PYPI: "true"
+    - os: osx
+      language: generic
+      env:
+        BUILD_BINARY: "true"
 script:
-  - make travis
+  - python3 --version
+  - pip3 --version
+  - pip3 install -U setuptools
+  - pip3 install -r requirements/dev.txt
+  - make bundle
+  - ./dist/tutor config noninteractive
+  - ./dist/tutor images env
+  - ./dist/tutor local env
+
+before_deploy:
+  - cp ./dist/tutor ./dist/tutor-$TRAVIS_OS_NAME
+
 deploy:
-  provider: script
-  script: cd build/ && docker login -u "$DOCKER_USERNAME" -p "$DOCKER_PASSWORD" && make push
-  on:
-    all_branches: true
-    condition: $TRAVIS_BRANCH =~ ^master|release\/.*$
+  # Push tutor binary to github releases
+  - provider: releases
+    api_key:
+      secure: ffrhGvgxHf3WVnKsKiJZTUoEf3+ZBtIydPHQoTDmHiYMyPnaO6RJWxq5PnoDRr47b1vnvbCE0lhNsGr3sgnruQSN3xHYfT4wz1F6Lz7Y5Jt7Uq1W0/UfSRlB3BwYcFpkUb5fSXLSku+QrV8OTk/yItlY9tppnvXA9h4CZjQqgJYHYc1DRKWQVYnVs/dCttUpiPV3eyIFeQoPKFsHwZXKcm9cVzom5r+lkjwpw3AIuAutPd71jyj/Va/By6B/43yAIqw3sA/thBeL76vqd8C4cMeWE00of1EWnH+sx6pKz32Fqe/o6dVop5PZPCiI+TVIDxR8oLevHtPTCfIwRDg60y23DdH3bMGSw1bAyjeWTnhRGcdYQJi1NKq3hJ0ldy0lOFlUiMkKPdQBtU7i0xIpoXTIhnro0YUUtjbh/QEtyRn8nbMVeenId42Bymah5P8srhE8S2z0x0mirIqNlM/tgLKEOJXdSL4sPKCMwRLcTUIc0fCiwLeHJnHIrMNEfnWsqO1odzv/PS5nLsdiGHBmC63d+xLNllzincIV1djyi5SEzMZxcqmjX1n/G3nKYVXhaIQE0wWQSHqNwLlKluY0kPXCNtU1TPr5SJHGnomQKizVaEsDrdAjylBL+rPwVCAv9z8FCtyOg7uMx2WdD0pvky2Iy4rbl2RBLUHmUZoAjPY=
+    file: dist/tutor-$TRAVIS_OS_NAME
+    skip_cleanup: true
+    on:
+      tags: true
+      condition: $BUILD_BINARY = true
+
+  # Push docker images to docker hub
+  - provider: script
+    script: ./dist/tutor images build all && \
+      ./dist/tutor local databases && \
+      docker login -u "$DOCKER_USERNAME" -p "$DOCKER_PASSWORD" && \
+      ./dist/tutor images push all
+    skip_cleanup: true
+    on:
+      tags: true
+      condition: $BUILD_DOCKER = true
+
+  # Push to pypi
+  - provider: script
+    script: pip install twine && python setup.py sdist && twine upload dist/*.tar.gz
+    skip_cleanup: true
+    on:
+      tags: true
+      condition: $BUILD_PYPI = true

CHANGELOG.md

@@ -1,5 +1,14 @@
 # Changelog
 
+## 3.0.0 (2019-06-09)
+
+- [Improvement] Complete rewrite of Tutor: switch from a make-based project to a single binary which runs all commands.
+- [Feature] An web user interface can be created with `tutor webui start`
+- [Bugfix] Add missing elasticsearch to Kubernetes deployment (#147)
+- [Improvement] Upload `tutor-openedx` to pypi
+
+## Older changes
+
 - 2019-01-27 [Bugfix] Fix video transcript/srt upload and download of user-uploaded files. Thanks @dannielariola!
 - 2019-01-20 [Improvement] Make it easy to load custom settings for the local production install
 - 2019-01-16 [Improvement] Switch license from MIT to AGPL

Makefile

@@ -1,66 +1,17 @@
-.PHONY: android build
 .DEFAULT_GOAL := help
 
-PWD = $$(pwd)
-USERID ?= $$(id -u)
+compile-requirements: ## Compile requirements files
+	pip-compile -o requirements/base.txt requirements/base.in
+	pip-compile -o requirements/dev.txt requirements/dev.in
 
-build: ## Build all docker images
-	cd build/ && make build
+bundle: ## Bundle the tutor package in a single "dist/tutor" executable
+	pyinstaller --onefile --name=tutor --add-data=./tutor/templates:./tutor/templates ./bin/main
 
-config.json: ## Generate config.json configuration file interactively
-	@$(MAKE) -s upgrade-to-tutor
-	@$(MAKE) -s -C build/ build-configurator 1> /dev/null
-	@docker run --rm -it \
-		--volume="$(PWD):/openedx/config/" \
-		-e USERID=$(USERID) -e SILENT=$(SILENT) \
-		regis/openedx-configurator:hawthorn \
-		configurator interactive
-
-substitute: config.json
-	@docker run --rm -it \
-		--volume="$(PWD)/config.json:/openedx/config/config.json" \
-		--volume="$(TEMPLATES):/openedx/templates" \
-		--volume="$(OUTPUT):/openedx/output" \
-		-e USERID=$(USERID) -e SILENT=$(SILENT) $(CONFIGURE_OPTS) \
-		regis/openedx-configurator:hawthorn \
-		configurator substitute /openedx/templates/ /openedx/output/
-
-local: ## Configure and run a ready-to-go Open edX platform
-	$(MAKE) -C deploy/local all
-
-stop: ## Stop all single server services
-	$(MAKE) -C deploy/local stop
-
-android: ## Configure and build a development Android app
-	cd android/ && make all
-
-travis:
-	cd build && make build
-	cd deploy/local \
-		&& make configure SILENT=1 CONFIGURE_OPTS="-e SETTING_ACTIVATE_NOTES=1 -e SETTING_ACTIVATE_XQUEUE=1" \
-		&& make databases
-
-upgrade-to-tutor: ## Upgrade from earlier versions of tutor
-	@(stat config/config.json > /dev/null 2>&1 && (\
-		echo "You are running an older version of Tutor. Now migrating to the latest version" \
-		&& echo "Moving config/config.json to ./config.json" && mv config/config.json config.json \
-		&& echo "Moving config/ to deploy/env/" && mv config/ deploy/env/ \
-		&& ((ls openedx/themes/* > /dev/null 2>&1 && echo "Moving openedx/themes/* to build/openedx/themes/" && mv openedx/themes/* build/openedx/themes/) || true) \
-		&& (mv .env deploy/local/ > /dev/null 2>&1 || true)\
-		&& echo "Done migrating to tutor. This command will not be run again."\
-	)) || true
-
-info: ## Print some information about the current install, for debugging
-	uname -a
-	@echo "-------------------------"
-	git rev-parse HEAD
-	@echo "-------------------------"
-	docker version
-	@echo "-------------------------"
-	docker-compose --version
-	@echo "-------------------------"
-	echo $$EDX_PLATFORM_PATH
-	echo $$EDX_PLATFORM_SETTINGS
+travis: bundle ## Run tests on travis-ci
+	./dist/tutor config noninteractive
+	./dist/tutor images env
+	./dist/tutor images build all
+	./dist/tutor local databases
 
 ESCAPE = 
 help: ## Print this help

README.rst

@@ -13,6 +13,10 @@ Tutor 🎓 Open edX 1-click install for everyone
     :alt: GitHub closed issues
     :target: https://github.com/regisb/tutor/issues?q=is%3Aclosed
 
+.. image:: https://img.shields.io/github/license/regisb/tutor.svg
+    :alt: AGPL License
+    :target: https://www.gnu.org/licenses/agpl-3.0.en.html
+
 **Tutor** is a one-click install of `Open edX <https://openedx.org>`_, both for production and local development, inside docker containers. Tutor is easy to run, fast, full of cool features, and it is already used by dozens of Open edX platforms in the world.
 
 .. image:: https://asciinema.org/a/octNfEnvIA6jNohCBmODBKizE.png
@@ -22,18 +26,16 @@ Tutor 🎓 Open edX 1-click install for everyone
 Quickstart
 ----------
 
-::
-
-    git clone https://github.com/regisb/tutor
-    cd tutor/deploy/local
-    make all
+1. `Download <https://github.com/regisb/tutor/releases>`_ the latest stable release of Tutor, uncompress the file and place the ``tutor`` executable in your path.
+2. Run ``tutor local quickstart``
+3. You're done!
 
 Documentation
 -------------
 
 Extensive documentation is available online: http://docs.tutor.overhang.io/
 
-How to contribute
------------------
+Contributing
+------------
 
-We go to great lengths to make it as easy as possible for people to run Open edX inside Docker containers. If you have an improvement idea, feel free to `open an issue on Github <https://github.com/regisb/tutor/issues/new>`_ so that we can discuss it. `Pull requests <https://github.com/regisb/tutor/pulls>`_ will be happily examined, too! However, we should be careful to keep the project lean and simple: both to use and to modify. Optional features should not make the user experience more complex. Instead, documentation on how to add the feature is preferred.
+We go to great lengths to make it as easy as possible for people to run Open edX inside Docker containers. If you have an improvement idea, feel free to `open an issue on Github <https://github.com/regisb/tutor/issues/new>`_ so that we can discuss it. `Pull requests <https://github.com/regisb/tutor/pulls>`_ will be happily examined, too!

bin/main

@@ -0,0 +1,4 @@
+#!/usr/bin/env python3
+
+from tutor.cli import main
+main()