Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add preprocessing training data #344

Merged
merged 6 commits into from
Nov 14, 2024
Merged

Conversation

ElliottKasoar
Copy link
Member

Resolves #37

This probably should should probably be moved to training after #341 is merged.

This also currently relies on us passing each config option. I'm hoping we should be able to update this (see MACE PR) to allow the config to be passed straight through, as we do for training.

@ElliottKasoar ElliottKasoar added the enhancement New/improved feature or request label Oct 31, 2024
@ElliottKasoar ElliottKasoar force-pushed the add-preprocess branch 3 times, most recently from 903ea88 to 7351264 Compare November 1, 2024 16:38
Copy link
Collaborator

@oerc0122 oerc0122 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

Suggested a couple of possible extensions/simplifications which may not be fore this PR.

janus_core/cli/preprocess.py Show resolved Hide resolved
janus_core/helpers/utils.py Show resolved Hide resolved
janus_core/helpers/utils.py Outdated Show resolved Hide resolved
janus_core/training/preprocess.py Outdated Show resolved Hide resolved
janus_core/training/preprocess.py Outdated Show resolved Hide resolved
janus_core/training/preprocess.py Show resolved Hide resolved
janus_core/training/preprocess.py Show resolved Hide resolved
janus_core/training/train.py Outdated Show resolved Hide resolved
oerc0122
oerc0122 previously approved these changes Nov 8, 2024
@ElliottKasoar
Copy link
Member Author

(Rebased and updated use of none_to_dict in preprocess as well)

@alinelena alinelena self-requested a review November 14, 2024 15:52
@alinelena alinelena merged commit ca1d799 into stfc:main Nov 14, 2024
8 checks passed
@ElliottKasoar ElliottKasoar deleted the add-preprocess branch November 14, 2024 21:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New/improved feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

add data reduction to hdf5
3 participants