diff --git a/README.md b/README.md index 9d6347c..0ea5c8c 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ it needs to be included while running them. 1. Set the home environment variable - ``` + ```bash $ export EMISSION_SERVER_HOME= ``` @@ -21,26 +21,26 @@ it needs to be included while running them. 1. If you haven't setup before, set up the evaluation system - ``` + ```bash $ source setup.sh ``` 1. If you have, activate - ``` + ```bash $ source activate.sh ``` 1. Access the visualizations of interest and copy the config over. The `` mentioned below can be any folder containing notebooks and/or .py files for visualisation or other purposes. E.g. : `TRB_label_assist` is one such folder. -``` +```bash $ cd $ cp -r ../conf . ``` 1. Start the notebook server -``` +```bash $ ../bin/em-jupyter-notebook.sh ``` @@ -50,6 +50,16 @@ $ ../bin/em-jupyter-notebook.sh the top of the notebook, and request the data for research purposes using https://github.com/e-mission/e-mission-server/wiki/Requesting-data-as-a-collaborator +- Assuming that your data is in the "mongodump" format, this repository has a helper script to load the data directly into the database. + - Navigate to the `e-mission-eval-private-data/` directory and start the docker environment + ```bash + $ docker-compose -f docker-compose.dev.yml up + ``` + - In another terminal, again navigate to the repository. Using the script provided, load the mongodump into docker + ```bash + $ bash bin/load_mongodump.sh + ``` + - Depending on the size of the mongodump, the loading step may take quite a long time (up to _several hours_). For more details on how to speed up this process, please refer to the data request documentation [here](https://github.com/e-mission/e-mission-server/wiki/Requesting-data-as-a-collaborator). ### Cleaning up After completing analysis, tear down diff --git a/bin/load_mongodump.sh b/bin/load_mongodump.sh new file mode 100644 index 0000000..d131cf1 --- /dev/null +++ b/bin/load_mongodump.sh @@ -0,0 +1,9 @@ +MONGODUMP_FILE=$1 + +echo "Copying file to docker container" +docker cp $MONGODUMP_FILE e-mission-eval-private-data-db-1:/tmp + +FILE_NAME=`basename $MONGODUMP_FILE` + +echo "Restoring the dump from $FILE_NAME" +docker exec -e MONGODUMP_FILE=$FILE_NAME e-mission-eval-private-data-db-1 bash -c 'cd /tmp && tar xvf $MONGODUMP_FILE && mongorestore' diff --git a/docker-compose.dev.yml b/docker-compose.dev.yml new file mode 100644 index 0000000..476524d --- /dev/null +++ b/docker-compose.dev.yml @@ -0,0 +1,17 @@ +version: "3" +services: + db: + image: mongo:4.4.0 + volumes: + - mongo-data:/data/db + networks: + - emission + ports: + # + - "27017:27017" + +networks: + emission: + +volumes: + mongo-data: