Skip to content

edX Managing the Production Stack

John Jarvis edited this page Apr 30, 2014 · 70 revisions

NOTE: These are instructions for the Production Stack configuration, not the Developer Stack.

Managing the server

manage.py commands

login to the server and change to the /edx/app/edxapp/edx-platform directory to use manage.py

  • list all manage.py commands - sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws help
  • create a final grade and generate a certificate for a single user - sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws regenerate_user -u [email protected] -c edX/Open_DemoX/edx_demo_course
  • create a new user - sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws create_user -e [email protected]
  • set or change password - sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws changepassword user
  • set user to staff - sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws set_staff [email protected]
  • launch the django shell - sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws shell
  • importing a course from github Example using the demo course
    cd /var/tmp
    git clone https://github.com/edx/edx-demo-course.git
    cd /edx/app/edxapp/edx-platform
    sudo -u www-data /edx/bin/python.edxapp ./manage.py cms --settings=aws
    

import /edx/var/edxapp/data /var/tmp/edx-demo-course

* **create superuser** - 

sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws create_user -e [email protected] sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws changepassword user sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws shell

from django.contrib.auth.models import User me = User.objects.get(username="user") me.is_superuser = True me.is_staff = True me.save()

* **run migrations** - `sudo -u www-data /edx/bin/python.edxapp ./manage.py lms syncdb --migrate --settings aws`
* **delete course** - `sudo -u www-data /edx/bin/python.edxapp ./manage.py cms --settings aws delete_course Organization/CourseNumber/CourseNumber commit`

See `help` and `help <cmd>` for more information!


## See what services are running

`sudo /edx/bin/supervisorctl -c /edx/etc/supervisord.conf status`

you should see something like the following:

certs RUNNING pid 19862, uptime 0:00:04 discern RUNNING pid 19926, uptime 0:00:02 discern_celery RUNNING pid 19857, uptime 0:00:04 edxapp:cms RUNNING pid 19884, uptime 0:00:03 edxapp:lms RUNNING pid 19879, uptime 0:00:03 edxapp_worker:cms_default_4 RUNNING pid 19915, uptime 0:00:03 edxapp_worker:cms_high_1 RUNNING pid 19901, uptime 0:00:03 edxapp_worker:cms_low_3 RUNNING pid 19890, uptime 0:00:03 edxapp_worker:lms_default_3 RUNNING pid 19897, uptime 0:00:03 edxapp_worker:lms_high_4 RUNNING pid 19922, uptime 0:00:03 edxapp_worker:lms_high_mem_2 RUNNING pid 19894, uptime 0:00:03 edxapp_worker:lms_low_1 RUNNING pid 19908, uptime 0:00:03 forum RUNNING pid 19855, uptime 0:00:04 ora RUNNING pid 19875, uptime 0:00:04 ora_celery RUNNING pid 19866, uptime 0:00:04 xqueue RUNNING pid 19853, uptime 0:00:04 xqueue_consumer RUNNING pid 19860, uptime 0:00:04 xserver RUNNING pid 19854, uptime 0:00:04


## Restarting Services

* LMS/CMS - `sudo /edx/bin/supervisorctl -c /edx/etc/supervisord.conf restart edxapp:`
* Workers - `sudo /edx/bin/supervisorctl -c /edx/etc/supervisord.conf restart edxapp_worker:`

(*Note:* the colon character `:` at the end of the line is mandatory!)


## Updating Versions using edX repos

Use the `/edx/bin/update` to update repo versions on the server.

Allowed `repo` names [cs_comments_service,ease,discern,xqueue,xserver,edx-platform,configuration,edx-ora]

Examples:
* `sudo /edx/bin/update edx-platform master` - change edx-platform to use the latest on the master branch
* `sudo /edx/bin/update cs_comments_service master` - change the forums service to use the latest on the master branch

## Generate certificates after a course ends

*Note: the [email protected] and [email protected] users have been pre-added to the certificate whitelist in the vagrant image so you can generate a certificate even if they have not passed the course*

* Certificates should only be generated after the course ends. The reason for this is because we store a final grade in the certificate table.  To ensure your course has ended login to studio and confirm the end date
![end date](https://raw.githubusercontent.com/edx/configuration/master/.wiki-images/set-end-date-studio.png)
* Once the end date has passed for a course the users will see that 'final course details' are being wrapped up. 
![final course details](https://raw.githubusercontent.com/edx/configuration/master/.wiki-images/wrapping-up-dashboard.png)
* Next you will can either run the ungenerated_certs django admin command or the regenerate_user django admin command to generate certificates for an entire course or for a single user.  This command does the following:
 * Grade the student
 * Update the generated certificate table with the students score and certificate status
 * Send a certificate request to the certificate process
 * The certificate server will generate a pdf, sign it and copy it to the certificate publish directory
 * The student's dashboard will update to show the link to the pdf and the validation page.

![cert download](https://raw.githubusercontent.com/edx/configuration/master/.wiki-images/download-certificate.png)

These are the django commands to generate certificates for an entire course or a single student.
Note the *--insecure* flag which is so that http is used for the lms callback url. It is not necessary if https is supported on the lms.

Generate or regenerate a certificate for a singel user in a course

sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws regenerate_user -u [email protected] -c edX/Open_DemoX/edx_demo_course --insecure

Grade and generate certs for all users in a course

sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws ungenerated_certs -c edX/Open_DemoX/edx_demo_course --insecure


## Using an edx-platform fork

* `sudo rm -rf /edx/app/edxapp/edx-platform` - to change the remote you need to remove the existing edx-platform checkout
* Add the following line to `/edx/app/edx_ansible/server-vars.yml` - `edx_platform_repo: "https://github.com/<user>/edx-platform.git"`
* `sudo /edx/bin/update edx-platform master` - update edx-platform using the forked repo

## Compile assets manually

To compile javascript and css outside of the update script run the following commands:

* `sudo -u edxapp bash`
* `source /edx/app/edxapp/edxapp_env`
* `cd /edx/app/edxapp/edx-platform`
* `SERVICE_VARIANT=lms rake lms:gather_assets:aws`
* `SERVICE_VARIANT=cms rake cms:gather_assets:aws`

# Troubleshooting
## Unable to connect to the LMS

Check to see if nginx is running:

* `sudo service nginx status`
* `sudo service nginx start`

NGINX logs are located in `/edx/var/log/nginx`
NGINX configurations are located in `/edx/app/nginx/sites-enabled`


Check to see if the lms/cms is running

* `sudo /edx/bin/supervisorctl -c /edx/etc/supervisord.conf status edxapp:`


If you are still unable to connect to the LMS run the following commands:

* `sudo service nginx stop`
* `sudo service supervisor stop`
* `sudo service supervisor.devpi stop`
* `sudo pkill -u www-data`
* `sudo service nginx start`
* `sudo service supervisor start`
* `sudo service supervisor.devpi start`
* stop everything except the LMS and CMS

sudo /edx/bin/supervisorctl -c /edx/etc/supervisord.conf stop discern discern_celery
forum ora ora_celery xqueue xqueue_consumer xserver edxapp_worker:

* `sudo tail -f /edx/var/log/{lms,cms,nginx}/*log`

_While tailing the log files try to connect your browser to the LMS or Studio_

## Log files

* The production stack sends all application logs to rsyslog which writes config files to the `/edx/var/log` directory.
* Supervisor logs stderr and stdout, its logs are located in `/edx/var/log/supervisor`