Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Errors when try to save checkpoints during the full fine-tuning #1764

Open
lemon-awa opened this issue Oct 1, 2024 · 0 comments
Open

Errors when try to save checkpoints during the full fine-tuning #1764

lemon-awa opened this issue Oct 1, 2024 · 0 comments
Labels
question Further information is requested

Comments

@lemon-awa
Copy link

When I try to save checkpoints during full fine-tune, errors happen when running save_hyperparameters(setup, check_file.parent) this line. The detailed errors are shown below:
[rank0]: save_hyperparameters(setup, checkpoint_file.parent)
[rank0]: File "litgpt/litgpt/utils.py", line 487, in save_hyperparameters
[rank0]: parser = capture_parser(lambda: CLI(function))
[rank0]: File ".conda/envs/gpt/lib/python3.10/site-packages/jsonargparse/_util.py", line 109, in capture_parser
[rank0]: function(*args, **kwargs)
[rank0]: File "litgpt/litgpt/utils.py", line 487, in
[rank0]: parser = capture_parser(lambda: CLI(function))
[rank0]: File "litgpt/litgpt/utils.py", line 449, in CLI
[rank0]: return CLI(*args, **kwargs)
[rank0]: File ".conda/envs/gpt/lib/python3.10/site-packages/jsonargparse/_cli.py", line 88, in CLI
[rank0]: _add_component_to_parser(components, parser, as_positional, fail_untyped, config_help)
[rank0]: File ".conda/envs/gpt/lib/python3.10/site-packages/jsonargparse/_cli.py", line 187, in _add_component_to_parser
[rank0]: added_args = parser.add_function_arguments(component, as_group=False, **kwargs)
[rank0]: File ".conda/envs/gpt/lib/python3.10/site-packages/jsonargparse/_signatures.py", line 209, in add_function_arguments
[rank0]: return self._add_signature_arguments(
[rank0]: File ".conda/envs/gpt/lib/python3.10/site-packages/jsonargparse/_signatures.py", line 289, in _add_signature_arguments
[rank0]: self._add_signature_parameter(
[rank0]: File ".conda/envs/gpt/lib/python3.10/site-packages/jsonargparse/_signatures.py", line 418, in _add_signature_parameter
[rank0]: raise ValueError(
[rank0]: ValueError: With fail_untyped=True, all mandatory parameters must have a supported type. Parameter 'checkpoint_dir' from 'main.setup' does not specify a type.

@lemon-awa lemon-awa added the question Further information is requested label Oct 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

1 participant