Skip to content

qubole/bootstrap-functions

Folders and files

NameName
Last commit message
Last commit date

Latest commit

1ad7a0b · Jul 29, 2020

History

61 Commits
Mar 18, 2020
Mar 18, 2020
Jul 29, 2020
Jul 3, 2018
Mar 18, 2020
Jul 29, 2020
Jun 10, 2020
May 18, 2020
Mar 18, 2020
Jul 21, 2020
Mar 7, 2018
May 18, 2020
Mar 18, 2020
Mar 12, 2018
Mar 18, 2020

Repository files navigation

bootstrap-functions

This repository holds common functions that can be used in Qubole node bootstraps

How to use

Source the required script in your bootstrap script. For example, to mount an EFS volume with the bootstrap, you may do the following:

source /usr/lib/qubole/bootstrap-functions/misc/mount_nfs.sh

mount_nfs fs-7abd2444.efs.us-east-1.amazonaws.com:/ /mnt/efs

Available functions

The following set of functions are available at present:

Contributing

Please raise a pull request for any modifications or additions you would like to make. There may be a delay between when you want to start using a method and when it might be available via Qubole's AMI. To work around this, it is recommended to put a placeholder source line in your bootstrap script. For example

function mysparkfunction() {
  # ... do some stuff
}

source /usr/lib/qubole/bootstrap-functions/spark/mysparkfunction.sh

mysparkfunction arg1 arg2 ...

This way, when the function makes it to the AMI, you will automatically use the copy in the bootstrap-functions library.