Refactor TopmedPipeline.py Cluster classes

[amstilp]

These changes could make it easier and more straightforward to subclass Cluster for your own specific cluster instance. I'm also not sure that it would work with all of the existing cluster subclasses.

• Cluster: Add methods to process the arguments passed to submitJob. Any class subclassing Cluster would be expected to override these methods. The method in the Cluster class should return a raise a NotImplementedError.
• Cluster: Add a submitJob method to the Cluster class. This method would call the methods defined in the previous step and then finally submit the generated command to the cluster. This also defines the keywords that are allowed to be passed to submitJob (which I think are only defined in the subclass methods)?
• Subclasses of cluster: implement the methods from the first step, specific for the cluster type.

Users would then only have to override the get_job_name, get_submit_command etc methods in their subclass, rather than overriding the entire submitJob method.

Here is a brief toy example:

class Cluster:
    def __init__(self):
        # intended to hold a list of arguments to pass to the submit command (e.g. "-N <job_name>" for SGE and "--job-name=<job_name>" for Slurm.
        self.submit_arguments = []

    def set_job_name(self):
        raise NotImplementedError("implement get_job_name")

    def get_submit_command(self):
        raise NotImplementedError("implement get_submit_command")

    def construct_command(self, submit_command):
         return self.get_submit_command + " " + " ".join(self.submit_arguments)

    def submitJob(self, job_name):
        self.set_job_name(self, job_name):
        cmd = self.construct_command()
        (jobid, status) = port_popen.popen(cmd)
        return jobid

class SGECluster(Cluster):
    def get_job_name(self, job_name):
        self.submit_arguments = "-N " + job_name

    def get_submit_command(self):
        return "qsub"

class SlurmCluster(Cluster):
    def get_job_name(self, job_name):
        self.submit_arguments = "--job-name " + job_name

    def get_submit_command(self):
        return "sbatch"

Obviously we would need to think about the exact implementation, what would be defined where, and what methods would be needed, but in the long run it would probably make it easier for a user to create a new subclass.