A Docker image for Livy, the REST Spark Server.
The image found in dockerhub can be run with (not local customization unless you rebuild it, see next section).
docker run -p 8998:8998 -d tobilg/livy
which will expose the port 8998 on the Docker host for this image.
- build the dockerfile locally and tag it, e.g.
docker build -t local/livy-spark:latest . - can expose volumes from which to customize configuration (example belows assumes overlay driver, e.g. centos host config here)
docker run --rm -it \ --ulimit nofile=9999:9999 \ -v "$(pwd)/conf/livy.conf":/apps/livy-server-0.3.0-SNAPSHOT/conf/livy.conf \ -v "$(pwd)/docker-repos/docker-livy/upload":/apps/livy-server-0.3.0-SNAPSHOT/upload \ -p 8998:8998 \ --name sparkly local/livy-spark
Have a look at the official docs to see how to use the Livy REST API.