02456 Deep Learning (with PyTorch)

This repository contains exercises for the DTU course 02456 Deep Learning. All exercises are written in the Python programming language and formatted into Jupyter Notebooks. If you are unfamiliar with notebooks, it can be a good idea to familiarize yourself with them in advance.

This repository borrows heavily from previous works, in particular:

• 2015 DTU Summerschool in Deep Learning. A PhD summerschool that was held at DTU in 2015. Exercises both in numpy and Theano.

• 02456-deep-learning. Previous version of the course material for this course, but using TensorFlow for the exercises.

• Pytorch Tutorial. A remix popular deep learning materials, including material from 02456, collected in one coherent package using PyTorch, with a focus on natural language processing (NLP)

• pytorch/tutorials. Official tutorials from the PyTorch repo.

Setup

The recommended (and by far the easiest) way to get started with the exercises is by using Google Colab. It allows you to work with Jupyter Notebooks in the cloud with all dependencies pre-installed, and Colab offers GPU utility for free which allows you to run the exercises considerably faster.

If you prefer to work locally, we here provide installation guides for Linux and Mac using Docker.

• NB: If you have an Nvidia GPU you should follow the GPU instructions below, since working on a GPU rather than CPU will accelerate things substantially.
• NB: If you are using Windows working locally is a bit more difficult. We will help you the best we can, but be prepared to do some research on your own.

CPU (Linux & mac)

First you need to install Docker.

Setup Docker by typing this in the command line in the exercise folder (this one)

• docker build -t munkai/pytorch:cpu -f Dockerfile.cpu .

Start Docker container

• docker run -it -p 8888:8888 -v `pwd`:/work munkai/pytorch:cpu ./jupyter_run.sh

Go to your browser, and type in http://localhost:8888. It will ask for a password (token) that you can copy from your terminal.

GPU (Linux & mac)

Setting up your GPU may take a while, and you might need to consult your favorite search engine. You need Nvidia and nvidia-docker installed for this.

Make sure you have Nvidia's drivers installed for your system. The following instructions will install CUDA and Nvida drivers on Ubuntu 16.04. Adjust as appropriate.

DISTRO=ubuntu
VERSION=1604
ARCH=x86_64
sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/${DISTRO}${VERSION}/${ARCH}/7fa2af80.pub
sudo sh -c 'echo "deb http://developer.download.nvidia.com/compute/cuda/repos/${DISTRO}${VERSION}/${ARCH}/" > /etc/apt/sources.list.d/cuda.list'
sudo apt-get update && sudo apt-get install -y --no-install-recommends cuda-drivers

Install nvidia-docker: See https://github.com/NVIDIA/nvidia-docker on how to install nvidia-docker.

Setup Docker with GPU by typing this in the command line in the exercise folder (this one)

• docker build -t munkai/pytorch:gpu -f Dockerfile.gpu .

Running docker with a CUDA-enabled machine

• nvidia-docker run -it -p 8888:8888 -v `pwd`:/work munkai/pytorch:gpu ./jupyter_run.sh

Go to your browser, and type in http://localhost:8888. It will ask for a password (token) that you can copy from your terminal.

CPU (Windows)

The following instructions will help you setup Docker on Windows.

1. Install install Docker
2. Make drive shareable - complete steps 1-3 in this guide
3. Setup Docker by typing this in the command line in the exercise folder (this one)
4. docker build -t munkai/pytorch:cpu -f Dockerfile.cpu .
5. Make sure that jupyter_run.sh has Unix style line endings (git has probably made it Windows style when you downloaded. Text editors like Sublime can change that).
6. Run Docker (change command to match your setup)
7. docker run -v c:/PATH/TO/EXERCISES/02456-deep-learning-with-PyTorch:/work -it --rm -p 8888:8888 munkai/pytorch:cpu ./jupyter_run.sh
8. Go to your browser, and type in http://localhost:8888. It will ask for a password (token) that you can copy from your terminal.

And you are done! Once setup is complete you only need to perform the last 2 steps to get up and running.

Debugging

If you are having issues we have made a list of problems and solutions to help you. Please help us extend this list and help people in the future by letting us know about you issues and the solutions you found.

• Jupyter starts, but you don't see the exercises.

  • The drive isn't shared properly. Take a look at this guide again, or this Stack Overflow question.

• standard_init_linux.go:190: exec user process caused "no such file or directory"

  • docker/labs#215

GPU (Windows)

We haven't tested this, but it should be easy to combine the GPU (Linux & mac) and CPU (Windows) guides above. Let us know if this works/doesn't work for you.

7. Additional content

If you are interested in some PyTorch code bases check out the following links (reinforcement learning, GANs, ResNet, etc).

• Train neural nets to play video games
• Train a state-of-the-art ResNet network on imagenet
• Train a face generator using Generative Adversarial Networks
• Train a word-level language model using Recurrent LSTM networks
• Spinning up: Deep Reinforcement Learning
• More examples
• More tutorials
• Discuss PyTorch on the Forums
• Chat with other users on Slack