Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Data Resource Manager crashes when non-JSON file is encountered #44

Open
gregmundy opened this issue Jan 7, 2020 · 1 comment
Open

Data Resource Manager crashes when non-JSON file is encountered #44

gregmundy opened this issue Jan 7, 2020 · 1 comment
Assignees
@loganripplinger

Comments

@gregmundy
Copy link
Contributor

If a non-JSON file is encountered in the location where the data resource API retrieves files it will crash. This sometimes happens when configurations are pulled from a repository that may have additional files (such as README). The Data Resource Manager should gracefully ignore this invalid file and continue to process valid files. Ideally, this issue should be seen in logs.
@loganripplinger
Copy link
Contributor

loganripplinger commented Jan 7, 2020 •
edited
Loading

Probably just need to wrap the json.loads() commands in try/except and catch JSONDecodeError per https://docs.python.org/3/library/json.html

https://docs.python.org/3/library/json.html#json.JSONDecodeError

data-resource-api/data_resource_api/app/data_resource_manager.py

Line 228 in 8c30dc3

schema_dict = json.load(fh)

data-resource-api/data_resource_api/app/data_model_manager.py

Line 307 in 8c30dc3

schema_dict = json.load(fh)

@loganripplinger loganripplinger mentioned this issue Mar 2, 2020
Merged
loganripplinger added a commit that referenced this issue Mar 2, 2020
@loganripplinger
Fix DRM / Refactor schema loader (#55)
63f74df 
* Adds a set of Descriptor classes.
* A DescriptorGetter class allows DMM/DRM to load both a list of directories and a list of descriptor dictionaries.
* Descriptor helper classes act as interfaces that encompass the relevant handling of the data type
* Dependency inject sqlalchemy base to enable testing
* Loads only json files - #44
* Add logging to conftest.
* Fixes DRM error.
loganripplinger pushed a commit that referenced this issue Mar 2, 2020
Fix DRM / Refactor schema loader (#55)
f606cfe 
* Adds a set of Descriptor classes.
* A DescriptorGetter class allows DMM/DRM to load both a list of directories and a list of descriptor dictionaries.
* Descriptor helper classes act as interfaces that encompass the relevant handling of the data type
* Dependency inject sqlalchemy base to enable testing
* Loads only json files - #44
* Add logging to conftest.
* Fixes DRM error.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@loganripplinger loganripplinger

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@gregmundy @loganripplinger