-
Notifications
You must be signed in to change notification settings - Fork 6
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
8ed0ea4
commit 4296b57
Showing
2 changed files
with
85 additions
and
83 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,35 +1,45 @@ | ||
| # misc/mount_nfs.sh | ||
| # misc/util.sh | ||
|
|
||
| Provides function to mount a NFS volume | ||
| Provides miscellaneous utility functions | ||
|
|
||
| * [mount_nfs_volume()](#mountnfsvolume) | ||
| * [set_timezone()](#settimezone) | ||
| * [add_to_authorized_keys()](#addtoauthorizedkeys) | ||
|
|
||
|
|
||
| ## mount_nfs_volume() | ||
| ## set_timezone() | ||
|
|
||
| Mounts an NFS volume on master and worker nodes | ||
| Set the timezone | ||
|
|
||
| Instructions for AWS EFS mount: | ||
| 1. After creating the EFS file system, create a security group | ||
| 2. Create an inbound traffic rule for this security group that allows traffic on | ||
| port 2049 (NFS) from this security group as described here: | ||
| https://docs.aws.amazon.com/efs/latest/ug/accessing-fs-create-security-groups.html | ||
| 3. Add this security group as a persistent security group for the cluster from which | ||
| you want to mount the EFS store, as described here: | ||
| http://docs.qubole.com/en/latest/admin-guide/how-to-topics/persistent-security-group.html | ||
| This function sets the timezone on the cluster node. | ||
| The timezone to set is a mandatory parameter and must be present in /usr/share/zoneinfo | ||
| Eg: "US/Mountain", "America/Los_Angeles" etc. | ||
|
|
||
| TODO: add instructions for Azure file share | ||
| After setting the timezone, it is advised to restart engine daemons on the master and worker nodes | ||
|
|
||
| ### Example | ||
|
|
||
| ```bash | ||
| mount_nfs_volume "example.nfs.share:/" /mnt/efs | ||
| set_timezone "America/Los_Angeles" | ||
| ``` | ||
|
|
||
| ### Arguments | ||
|
|
||
| * **$1** (string): Path to NFS share | ||
| * **$2** (string): Mount point to use | ||
| * **$1** (string): Timezone to set | ||
|
|
||
| ## add_to_authorized_keys() | ||
|
|
||
| Add a public key to authorized_keys | ||
|
|
||
| ### Example | ||
|
|
||
| ```bash | ||
| add_to_authorized_keys "ssh-rsa xyzxyzxyzxyz...xyzxyz [email protected]" ec2-user | ||
| ``` | ||
|
|
||
| ### Arguments | ||
|
|
||
| * **$1** (string): Public key to add to authorized_keys file | ||
| * **$2** (string): User for which the public key is added. Defaults to `ec2-user` | ||
|
|
||
| # misc/python_venv.sh | ||
|
|
||
|
|
@@ -67,77 +77,67 @@ install_python_env 3.6 /path/to/virtualenv/py36 | |
| * **0**: Python virtualenv was created and activated | ||
| * **1**: Python executable for virtualenv couldn't be found or installed | ||
|
|
||
| # misc/awscli.sh | ||
| # misc/mount_nfs.sh | ||
|
|
||
| Provides function to configure AWS CLI | ||
| Provides function to mount a NFS volume | ||
|
|
||
| * [configure_awscli()](#configureawscli) | ||
| * [mount_nfs_volume()](#mountnfsvolume) | ||
|
|
||
|
|
||
| ## configure_awscli() | ||
| ## mount_nfs_volume() | ||
|
|
||
| Configure AWS CLI | ||
| Mounts an NFS volume on master and worker nodes | ||
|
|
||
| A credentials file containing the AWS Access Key and the AWS Secret Key | ||
| separated by a space, comma, tab or newline must be provided | ||
| Instructions for AWS EFS mount: | ||
| 1. After creating the EFS file system, create a security group | ||
| 2. Create an inbound traffic rule for this security group that allows traffic on | ||
| port 2049 (NFS) from this security group as described here: | ||
| https://docs.aws.amazon.com/efs/latest/ug/accessing-fs-create-security-groups.html | ||
| 3. Add this security group as a persistent security group for the cluster from which | ||
| you want to mount the EFS store, as described here: | ||
| http://docs.qubole.com/en/latest/admin-guide/how-to-topics/persistent-security-group.html | ||
|
|
||
| TODO: add instructions for Azure file share | ||
|
|
||
| ### Example | ||
|
|
||
| ```bash | ||
| configure_awscli -p exampleprofile -r us-east-1 -c /path/to/credentials/file | ||
| mount_nfs_volume "example.nfs.share:/" /mnt/efs | ||
| ``` | ||
|
|
||
| ### Arguments | ||
|
|
||
| * -p string Name of the profile. Defaults to `default` | ||
| * -r string AWS region. Defaults to `us-east-1` | ||
| * -c string Path to credentials file | ||
|
|
||
| ### Exit codes | ||
|
|
||
| * **0**: AWS CLI is configured | ||
| * **1**: AWS CLI or credentials file not found | ||
|
|
||
| # misc/util.sh | ||
| * **$1** (string): Path to NFS share | ||
| * **$2** (string): Mount point to use | ||
|
|
||
| Provides miscellaneous utility functions | ||
| # misc/awscli.sh | ||
|
|
||
| * [set_timezone()](#settimezone) | ||
| * [add_to_authorized_keys()](#addtoauthorizedkeys) | ||
| Provides function to configure AWS CLI | ||
|
|
||
| * [configure_awscli()](#configureawscli) | ||
|
|
||
| ## set_timezone() | ||
|
|
||
| Set the timezone | ||
| ## configure_awscli() | ||
|
|
||
| This function sets the timezone on the cluster node. | ||
| The timezone to set is a mandatory parameter and must be present in /usr/share/zoneinfo | ||
| Eg: "US/Mountain", "America/Los_Angeles" etc. | ||
| Configure AWS CLI | ||
|
|
||
| After setting the timezone, it is advised to restart engine daemons on the master and worker nodes | ||
| A credentials file containing the AWS Access Key and the AWS Secret Key | ||
| separated by a space, comma, tab or newline must be provided | ||
|
|
||
| ### Example | ||
|
|
||
| ```bash | ||
| set_timezone "America/Los_Angeles" | ||
| configure_awscli -p exampleprofile -r us-east-1 -c /path/to/credentials/file | ||
| ``` | ||
|
|
||
| ### Arguments | ||
|
|
||
| * **$1** (string): Timezone to set | ||
|
|
||
| ## add_to_authorized_keys() | ||
|
|
||
| Add a public key to authorized_keys | ||
|
|
||
| ### Example | ||
|
|
||
| ```bash | ||
| add_to_authorized_keys "ssh-rsa xyzxyzxyzxyz...xyzxyz [email protected]" ec2-user | ||
| ``` | ||
| * -p string Name of the profile. Defaults to `default` | ||
| * -r string AWS region. Defaults to `us-east-1` | ||
| * -c string Path to credentials file | ||
|
|
||
| ### Arguments | ||
| ### Exit codes | ||
|
|
||
| * **$1** (string): Public key to add to authorized_keys file | ||
| * **$2** (string): User for which the public key is added. Defaults to `ec2-user` | ||
| * **0**: AWS CLI is configured | ||
| * **1**: AWS CLI or credentials file not found | ||
|
|
||