Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use HTCondor python bindings #304

Open
JannisNe opened this issue Aug 24, 2023 · 2 comments
Open

Use HTCondor python bindings #304

JannisNe opened this issue Aug 24, 2023 · 2 comments
Labels
enhancement New feature or request

Comments

@JannisNe
Copy link
Collaborator

There are actually python bindings for HTCondor clusters: link.
That means a lot of the stuff in the HTCondorSubmitter class could use that, which would make everything more robust, probably.
Just leaving this here as an idea!

@JannisNe JannisNe added the enhancement New feature or request label Aug 24, 2023
@mlincett
Copy link
Collaborator

mlincett commented Sep 1, 2023

I have seen this used in some other projects but I have mixed feelings about it.

I like having a static and declarative job description in the form of a plaintext submit file, that gives you an outline of the job(s) at a glance.

Replacing this with something procedural could make the whole thing a bit opaque and harder to debug, but at the same time allows for more sanity checks (i.e. does $INPUT_FILE exist or not) at runtime.

Still, I am in favour of it :)

@JannisNe
Copy link
Collaborator Author

JannisNe commented Sep 6, 2023

In general I think flarestack's behaviour of submitting jobs automatically is not optimal. I think the submission to the cluster should always happen manually, i.e. flarestack should only create the submission file. That would also remove the need for the flarestack process to hang around until all jobs have completed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants