Note
This page is a reference documentation. It only explains the class signature, and not how to use it. Please refer to the user guide for the big picture.
hopla.Executor¶
- class hopla.Executor(cluster, folder, queue, image, name='hopla', memory=2, walltime=72, n_cpus=1, n_gpus=0, n_multi_cpus=1, modules=None, project_id=None)[source]¶
Bases:
objectBase job executor.
- Parameters:
- cluster: str
the type of cluster: ‘slurm’, ‘ccc’, ‘pbs’.
- folder: Path/str
folder for storing job submission/output and logs.
- queue: str
the name of the queue where the jobs will be submitted.
- image: str
path to a docker ‘.tar’ image or apptainer ‘.simg’ image or name of an existing image.
- name: str, default ‘hopla’
the name of the submitted jobs.
- memory: float , default 2
the memory allocated to each job (in GB).
- walltime: int default 72
the walltime used for each job (in hours).
- n_cpus: int, default 1
the number of cores allocated for each job.
- n_gpus: int, default 0
the number of GPUs allocated for each job.
- n_multi_cpus: int, default 1
the number of cores reserved fir each multi-tasks job.
- modules: list of str, default None
the environment modules to be loaded.
- project_id: str, default None
the project ID where you have computing hours.
- Raises:
- ValueError
If the cluster type is not supported.
Examples
>>> import hopla >>> executor = hopla.Executor( ... cluster="slurm", ... folder="/tmp/hopla", ... queue="Nspin_long", ... image="/tmp/hopla/my-apptainer-img.simg", ... ) >>> jobs = [executor.submit("sleep", k) for k in range(1, 11)] >>> executor(max_jobs=2) >>> print(executor.report)
- __init__(cluster, folder, queue, image, name='hopla', memory=2, walltime=72, n_cpus=1, n_gpus=0, n_multi_cpus=1, modules=None, project_id=None)[source]¶
- property n_done_jobs¶
Get the number of finished jobs.
- property n_jobs¶
Get the number of stacked jobs.
- property n_running_jobs¶
Get the number of running jobs.
- property n_waiting_jobs¶
Get the number of waiting jobs.
- property report¶
Generate a general report for all jobs.
- property status¶
Display current status.
- submit(script, *args, execution_parameters=None, **kwargs)[source]¶
Create a delayed job.
- Parameters:
- script: Path/str or list of DelayedSubmission
script(s) to execute.
- *args: any positional argument of the script.
- execution_parameters: str or list of str
parameters passed to the container during execution.
- **kwargs: any named argument of the script.
- Returns:
- job: DelayedJob
a job instance.
- Raises:
- RuntimeError
If the job class is not DelayedCCCJob for multi-tasks submission.
Examples using hopla.Executor¶
Basic example on how to use the CCC cluster using multi-tasks
