The configurations of a Project.
No Additional PropertiesThe name of the project
The base directory containing the project related files. Default is a folder with the project name inside the projects folder
Folder where remote files are copied. Default a 'tmp' folder in base_dir
Folder containing all the logs. Default a 'log' folder in base_dir
Folder containing daemon related files. Default to a 'daemon' folder in base_dir
The level set for logging
The options for the Runner
No Additional PropertiesDelay between subsequent execution of the checkout from database (seconds)
Delay between subsequent execution of the checking the status of jobs that are submitted to the scheduler (seconds)
Delay between subsequent advancement of the job's remote state (seconds)
Delay between subsequent refresh from the DB of the number of submitted and running jobs (seconds). Only use if a worker with max_jobs is present
Delay between subsequent refresh from the DB of the number of submitted and running jobs (seconds). Only use if a worker with max_jobs is present
Time to consider the lock on a document expired and can be overridden (seconds)
Whether to delete the local temporary folder after a job has completed
Maximum number of attempt performed before failing an advancement of a remote state
List of increasing delay between subsequent attempts when the advancement of a remote step fails
No Additional ItemsA dictionary with the worker name as keys and the worker configuration as values
Each additional property must conform to the following schema
Worker representing the local host.
Executes command directly.
No Additional PropertiesThe discriminator field to determine the worker type
Specific value:"local" Type of the scheduler. Depending on the values supported by QToolKit
Absolute path of the directory of the worker where subfolders for executing the calculation will be created
A dictionary defining the default resources requested to the scheduler. Used to fill in the QToolKit template
String with commands that will be executed before the execution of the Job
String with commands that will be executed after the execution of the Job
Timeout for the execution of the commands in the worker (e.g. submitting a job)
The maximum number of jobs that can be submitted to the queue.
Value must be greater or equal to 0
Options for batch execution. If define the worker will be considered a batch worker
Worker representing a remote host reached through an SSH connection.
Uses a Fabric Connection. Check Fabric documentation for more datails on the
options defininf a Connection.
The discriminator field to determine the worker type
Specific value:"remote" Type of the scheduler. Depending on the values supported by QToolKit
Absolute path of the directory of the worker where subfolders for executing the calculation will be created
A dictionary defining the default resources requested to the scheduler. Used to fill in the QToolKit template
String with commands that will be executed before the execution of the Job
String with commands that will be executed after the execution of the Job
Timeout for the execution of the commands in the worker (e.g. submitting a job)
The maximum number of jobs that can be submitted to the queue.
Value must be greater or equal to 0
Options for batch execution. If define the worker will be considered a batch worker
The host to which to connect
Login username
Port number
Login password
The filename, or list of filenames, of optional private key(s) and/or certs to try for authentication
Passphrase used for decrypting private keys
A shell command string to use as a proxy or gateway
Whether to enable SSH agent forwarding
Connection timeout, in seconds
Other keyword arguments passed to paramiko.client.SSHClient.connect
Whether to send environment variables 'inline' as prefixes in front of command strings
Keepalive value in seconds passed to paramiko's transport
The shell command used to execute the command remotely. If None the command is executed directly
Whether to use a login shell when executing the command
Dictionary describing a maggma Store used for the queue data. Can contain the monty serialized dictionary or a dictionary with a 'type' specifying the Store subclass. Should be subclass of a MongoStore, as it requires to perform MongoDB actions.
A dictionary with the ExecutionConfig name as keys and the ExecutionConfig configuration as values
Each additional property must conform to the following schema
Type: objectConfiguration to be set before and after the execution of a Job.
No Additional Propertieslist of modules to be loaded
dictionary with variable to be exported
Other commands to be executed before the execution of a job
Commands to be executed after the execution of a job
The JobStore used for the input. Can contain the monty serialized dictionary or the Store int the Jobflow format
A dictionary with metadata associated to the project
The configurations of a Project.
No Additional PropertiesThe name of the project
The base directory containing the project related files. Default is a folder with the project name inside the projects folder
Folder where remote files are copied. Default a 'tmp' folder in base_dir
Folder containing all the logs. Default a 'log' folder in base_dir
Folder containing daemon related files. Default to a 'daemon' folder in base_dir
The level set for logging
The options for the Runner
No Additional PropertiesDelay between subsequent execution of the checkout from database (seconds)
Delay between subsequent execution of the checking the status of jobs that are submitted to the scheduler (seconds)
Delay between subsequent advancement of the job's remote state (seconds)
Delay between subsequent refresh from the DB of the number of submitted and running jobs (seconds). Only used if a worker with max_jobs is present
Delay between subsequent refresh from the DB of the number of submitted and running jobs (seconds). Only used if a batch worker is present
Time to consider the lock on a document expired and can be overridden (seconds)
Whether to delete the local temporary folder after a job has completed
Maximum number of attempt performed before failing an advancement of a remote state
List of increasing delay between subsequent attempts when the advancement of a remote step fails
No Additional ItemsA dictionary with the worker name as keys and the worker configuration as values
Each additional property must conform to the following schema
Worker representing the local host.
Executes command directly.
No Additional PropertiesThe discriminator field to determine the worker type
Specific value:"local" Type of the scheduler. Depending on the values supported by QToolKit
Absolute path of the directory of the worker where subfolders for executing the calculation will be created
A dictionary defining the default resources requested to the scheduler. Used to fill in the QToolKit template
String with commands that will be executed before the execution of the Job
String with commands that will be executed after the execution of the Job
Timeout for the execution of the commands in the worker (e.g. submitting a job)
The maximum number of jobs that can be submitted to the queue.
Value must be greater or equal to 0
Options for batch execution. If define the worker will be considered a batch worker
Configuration for execution of batch jobs.
Allows to execute multiple Jobs in a single process executed on the worker (e.g. SLURM job).
Same definition as BatchConfigWorker representing a remote host reached through an SSH connection.
Uses a Fabric Connection. Check Fabric documentation for more details on the
options defining a Connection.
The discriminator field to determine the worker type
Specific value:"remote" Type of the scheduler. Depending on the values supported by QToolKit
Absolute path of the directory of the worker where subfolders for executing the calculation will be created
A dictionary defining the default resources requested to the scheduler. Used to fill in the QToolKit template
String with commands that will be executed before the execution of the Job
String with commands that will be executed after the execution of the Job
Timeout for the execution of the commands in the worker (e.g. submitting a job)
The maximum number of jobs that can be submitted to the queue.
Value must be greater or equal to 0
Options for batch execution. If define the worker will be considered a batch worker
Configuration for execution of batch jobs.
Allows to execute multiple Jobs in a single process executed on the worker (e.g. SLURM job).
Same definition as BatchConfigThe host to which to connect
Login username
Port number
Login password
The filename, or list of filenames, of optional private key(s) and/or certs to try for authentication
Passphrase used for decrypting private keys
A shell command string to use as a proxy or gateway
The representation of a fabric connection.
Mainly used in case of nested gateways.
Whether to enable SSH agent forwarding
Connection timeout, in seconds
Other keyword arguments passed to paramiko.client.SSHClient.connect
Whether to send environment variables 'inline' as prefixes in front of command strings
Keepalive value in seconds passed to paramiko's transport
The shell command used to execute the command remotely. If None the command is executed directly
Whether to use a login shell when executing the command
Whether the authentication to the host should be interactive
The configuration of the Store used to store the states of the Jobs and the Flows
No Additional PropertiesDictionary describing a maggma Store used for the queue data. Can contain the monty serialized dictionary or a dictionary with a 'type' specifying the Store subclass. Should be subclass of a MongoStore, as it requires to perform MongoDB actions. The collection is used to store the jobs
The name of the collection containing information about the flows. Taken from the same database as the one defined in the store
The name of the collection containing auxiliary information. Taken from the same database as the one defined in the store
a string defining the prefix added to the integer ID associated to each Job in the database
A dictionary with the ExecutionConfig name as keys and the ExecutionConfig configuration as values
Each additional property must conform to the following schema
Type: objectConfiguration to be set before and after the execution of a Job.
No Additional Propertieslist of modules to be loaded
dictionary with variable to be exported
Other commands to be executed before the execution of a job
Commands to be executed after the execution of a job
The JobStore used for the input. Can contain the monty serialized dictionary or the Store int the Jobflow format
A dictionary with metadata associated to the project