Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding SLURM-compatibility to Benchexec #995

Merged
merged 62 commits into from
Feb 20, 2024
Merged
Changes from 1 commit
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
49f52ed
Added slurm executor
leventeBajczi Feb 17, 2024
687cdb4
Fixed memory calculation
leventeBajczi Feb 17, 2024
a53f6d6
Fixed memory calculation #2
leventeBajczi Feb 17, 2024
7b11e4b
Implemented slurm executor
leventeBajczi Feb 17, 2024
9577b1c
Fixed timelimit
leventeBajczi Feb 17, 2024
2496c19
Fixed subprocess
leventeBajczi Feb 17, 2024
be275ea
Fixed stdout
leventeBajczi Feb 17, 2024
3f72aa8
Fixed stdout
leventeBajczi Feb 17, 2024
f2f385b
Added logging
leventeBajczi Feb 17, 2024
754ab98
Fixed memory limit
leventeBajczi Feb 17, 2024
4f0f006
Adding 6 lines of metadata to beginning of file
leventeBajczi Feb 17, 2024
ea9b778
Reformatted file
leventeBajczi Feb 17, 2024
4d5af97
Cleared up commands
leventeBajczi Feb 18, 2024
e7cdee6
Formatted file
leventeBajczi Feb 18, 2024
6a7dbfb
Moved --slurm out of main benchexec code
leventeBajczi Feb 18, 2024
00ed378
Fixed path
leventeBajczi Feb 18, 2024
4e837d3
Fixed formatting command
leventeBajczi Feb 18, 2024
f78ff8f
Using --no-home instead of specifying -B
leventeBajczi Feb 18, 2024
a0c662d
Added --contain
leventeBajczi Feb 18, 2024
394a7c4
Added -B $PWD:$HOME
leventeBajczi Feb 18, 2024
2ea64b9
Added log
leventeBajczi Feb 18, 2024
82d960a
ntasks=1
leventeBajczi Feb 18, 2024
26d4472
removed unused import
leventeBajczi Feb 18, 2024
18ffb45
Added fusemount options
leventeBajczi Feb 18, 2024
c88e5dc
Added temp files
leventeBajczi Feb 18, 2024
1fbd3e4
Reformatted
leventeBajczi Feb 18, 2024
0d7246b
Updated copyright, added readme
leventeBajczi Feb 19, 2024
9f62e8e
Updated copyright
leventeBajczi Feb 19, 2024
26d1933
Added myself to the list of contributors
leventeBajczi Feb 19, 2024
501e43b
Added contact info to top of README
leventeBajczi Feb 19, 2024
9b2f445
Updated README regarding contact
leventeBajczi Feb 20, 2024
493ce92
Update README with better description on the workflow.
leventeBajczi Feb 20, 2024
6a5d5e6
Moved contributor entry to correct place in alphabetical order
leventeBajczi Feb 20, 2024
a9f1c7b
Added links in preliminaries
leventeBajczi Feb 20, 2024
67dcd61
Updated requirement description
leventeBajczi Feb 20, 2024
3f28a45
Moved disclaimer to limitations instead
leventeBajczi Feb 20, 2024
17f02ef
REmoved confusing documentation comment
leventeBajczi Feb 20, 2024
6b6cd65
Modified help text
leventeBajczi Feb 20, 2024
2a2ffe0
Reworked param passing
leventeBajczi Feb 20, 2024
6425071
Removed starttime
leventeBajczi Feb 20, 2024
a7501c2
Moved to factory method for ProcessExitCode
leventeBajczi Feb 20, 2024
61f7fe7
Replaced bash pipes with python code, moved regexes
leventeBajczi Feb 20, 2024
de4e23c
Removed unused import
leventeBajczi Feb 20, 2024
7307acc
Fixed missing str()
leventeBajczi Feb 20, 2024
8ef0505
Added extra logging
leventeBajczi Feb 20, 2024
28da4ac
Fixed bug in exit code parsing
leventeBajczi Feb 20, 2024
86574f2
Fixed quoting issue
leventeBajczi Feb 20, 2024
46247d7
Minor fixes: typo, logging
leventeBajczi Feb 20, 2024
e6b108e
Formatting fix
leventeBajczi Feb 20, 2024
f110bcc
Updated README and requering --no-hyperthreading
leventeBajczi Feb 20, 2024
2e6e2d9
Added exception to when exit code is not parsed
leventeBajczi Feb 20, 2024
bb9f5bf
Added new line to limitations
leventeBajczi Feb 20, 2024
9b86b5f
Formatting fix
leventeBajczi Feb 20, 2024
d742e30
Minor fixes from feedback
leventeBajczi Feb 20, 2024
ae7ebcc
Formatted file
leventeBajczi Feb 20, 2024
255eb6d
Not using shell anymore
leventeBajczi Feb 20, 2024
4368b34
str() wrap around ints
leventeBajczi Feb 20, 2024
e07ce23
Added comma
leventeBajczi Feb 20, 2024
24dd37e
str() wrap around ints
leventeBajczi Feb 20, 2024
1563694
Implemented some fixes
leventeBajczi Feb 20, 2024
d9fba5d
Formatted file
leventeBajczi Feb 20, 2024
e6e9b92
better handling of bad scratchdir
leventeBajczi Feb 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 35 additions & 22 deletions contrib/slurm/slurmexecutor.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ def execute_benchmark(benchmark, output_handler):

if benchmark.config.use_hyperthreading:
sys.exit(
"SLURM can only work properly without hyperthreading enabled. See README.md for details."
"SLURM can only work properly without hyperthreading enabled, by passing the --no-hyperthreading option. See README.md for details."
)

for runSet in benchmark.run_sets:
Expand Down Expand Up @@ -215,28 +215,41 @@ def run_slurm(benchmark, args, log_file):
os.makedirs(os.path.join(tempdir, "upper"))
os.makedirs(os.path.join(tempdir, "work"))

tool_command = " ".join(args)
singularity_command = (
f"singularity exec "
f'-B "$PWD":/lower --no-home '
f'-B "{tempdir}":/overlay '
f'--fusemount "container:fuse-overlayfs -o lowerdir=/lower -o upperdir=/overlay/upper -o workdir=/overlay/work ""$HOME""" '
f"{benchmark.config.singularity} {tool_command}"
(
[
"singularity",
"exec",
"-B",
"./:/lower",
"--no-home",
"-B",
f"{tempdir}:/overlay",
"--fusemount",
f"container:fuse-overlayfs -o lowerdir=/lower -o upperdir=/overlay/upper -o workdir=/overlay/work /home/{os.getlogin()}",
PhilippWendler marked this conversation as resolved.
Show resolved Hide resolved
benchmark.config.singularity,
]
+ args
)
if benchmark.config.singularity
else tool_command
)
srun_command = (
f"srun "
f"-t {srun_timelimit} "
f"-c {cpus} "
f"-o {log_file} "
f"--mem-per-cpu {mem_per_cpu} "
f"--threads-per-core=1 " # --use_hyperthreading=False is always given here
f"--ntasks=1 "
f"{singularity_command}"
else args
)
srun_command = [
"srun",
"-t",
srun_timelimit,
"-c",
cpus,
"-o",
log_file,
"--mem-per-cpu",
mem_per_cpu,
"--threads-per-core=1" # --use_hyperthreading=False is always given here
"--ntasks=1",
] + singularity_command
logging.debug("Command to run: %s", " ".join(srun_command))
leventeBajczi marked this conversation as resolved.
Show resolved Hide resolved
srun_result = subprocess.run(
["bash", "-c", srun_command],
srun_command,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
)
Expand All @@ -253,10 +266,10 @@ def run_slurm(benchmark, args, log_file):
stop()
return -1

seff_command = f"seff {jobid}"
logging.debug("Command to run: %s", seff_command)
seff_command = ["seff", jobid]
logging.debug("Command to run: %s", " ".join(seff_command))
result = subprocess.run(
["bash", "-c", seff_command],
seff_command,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
)
Expand Down
Loading