Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add read_nwb_method for local paths in both hdf5 and zarr #1994

Open
wants to merge 10 commits into
base: dev
Choose a base branch
from

Conversation

h-mayorquin
Copy link
Contributor

@h-mayorquin h-mayorquin commented Nov 16, 2024

Motivation

See #1974

How to test the behavior?

I added tests.

Checklist

  • Did you update CHANGELOG.md with your changes?
  • Have you checked our Contributing document?
  • Have you ensured the PR clearly describes the problem and the solution?
  • Is your contribution compliant with our coding style? This can be checked running ruff check . && codespell from the source directory.
  • Have you checked to ensure that there aren't other open Pull Requests for the same change?
  • Have you included the relevant issue number using "Fix #XXX" notation where XXX is the issue number? By including "Fix #XXX" you allow GitHub to close issue #XXX when the PR is merged.

Copy link

codecov bot commented Nov 16, 2024

Codecov Report

Attention: Patch coverage is 18.18182% with 9 lines in your changes missing coverage. Please review.

Project coverage is 91.39%. Comparing base (2c2bc85) to head (33847a6).

Files with missing lines Patch % Lines
src/pynwb/__init__.py 18.18% 9 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##              dev    #1994      +/-   ##
==========================================
- Coverage   91.69%   91.39%   -0.30%     
==========================================
  Files          27       27              
  Lines        2708     2719      +11     
  Branches      707      710       +3     
==========================================
+ Hits         2483     2485       +2     
- Misses        149      158       +9     
  Partials       76       76              
Flag Coverage Δ
integration 72.60% <18.18%> (-0.23%) ⬇️
unit 82.38% <18.18%> (-0.27%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@h-mayorquin h-mayorquin marked this pull request as ready for review November 16, 2024 04:31
@rly rly requested a review from stephprince December 10, 2024 00:41
Copy link
Contributor

@stephprince stephprince left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay @h-mayorquin! Added my review comments

Comment on lines +594 to +599
from hdmf_zarr import NWBZarrIO
backend_is_zarr = NWBZarrIO.can_read(path=path)
if backend_is_zarr:
return NWBZarrIO.read_nwb(path=path)
else:
raise ValueError(f"Unsupported backend for file: {path}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
from hdmf_zarr import NWBZarrIO
backend_is_zarr = NWBZarrIO.can_read(path=path)
if backend_is_zarr:
return NWBZarrIO.read_nwb(path=path)
else:
raise ValueError(f"Unsupported backend for file: {path}")
try:
from hdmf_zarr import NWBZarrIO
backend_is_zarr = NWBZarrIO.can_read(path=path)
if backend_is_zarr:
return NWBZarrIO.read_nwb(path=path)
else:
raise ValueError(f"Unsupported backend for file: {path}")
except ImportError:
raise ValueError(f"Unsupported backend for file: '{path}'. If you are trying to read a Zarr file, make sure you have hdmf-zarr installed.")

I believe read_nwb will currently return an ImportError if the user provides an invalid path but does not have hdmf_zarr installed. Can you support this case?

self.assertContainerEqual(read_nwbfile, self.nwbfile)
read_nwbfile.get_read_io().close()

@unittest.skipIf(not HAVE_NWBZarrIO, "NWBZarrIO library not available")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add tests for

  1. the user providing a path that does not exist
  2. if the user does not have hdmf_zarr installed and provides an unsupported file path

This function uses the following defaults:
* Always opens in read-only mode
* Automatically loads namespaces
* Detects file format based on extension
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Detects file format based on extension
* Detects file format based on whether supported backend IO classes can read the file

I think in the current implementation other extensions will work since can_read and IO.read_nwb methods just check whether the IO can successfully open the file?

CHANGELOG.md Outdated
Comment on lines 8 to 12
- Removed `SpatialSeries.bounds` field that was not functional. This will be fixed in a future release. @rly [#1907](https://github.com/NeurodataWithoutBorders/pynwb/pull/1907), [#1996](https://github.com/NeurodataWithoutBorders/pynwb/pull/1996)
- Added support for `NWBFile.was_generated_by` field. @stephprince [#1924](https://github.com/NeurodataWithoutBorders/pynwb/pull/1924)
- Added support for `model_number`, `model_name`, and `serial_number` fields to `Device`. @stephprince [#1997](https://github.com/NeurodataWithoutBorders/pynwb/pull/1997)
- Deprecated `EventWaveform` neurodata type. @rly [#1940](https://github.com/NeurodataWithoutBorders/pynwb/pull/1940)
- Deprecated `ImageMaskSeries` neurodata type. @rly [#1941](https://github.com/NeurodataWithoutBorders/pynwb/pull/1941)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Removed `SpatialSeries.bounds` field that was not functional. This will be fixed in a future release. @rly [#1907](https://github.com/NeurodataWithoutBorders/pynwb/pull/1907), [#1996](https://github.com/NeurodataWithoutBorders/pynwb/pull/1996)
- Added support for `NWBFile.was_generated_by` field. @stephprince [#1924](https://github.com/NeurodataWithoutBorders/pynwb/pull/1924)
- Added support for `model_number`, `model_name`, and `serial_number` fields to `Device`. @stephprince [#1997](https://github.com/NeurodataWithoutBorders/pynwb/pull/1997)
- Deprecated `EventWaveform` neurodata type. @rly [#1940](https://github.com/NeurodataWithoutBorders/pynwb/pull/1940)
- Deprecated `ImageMaskSeries` neurodata type. @rly [#1941](https://github.com/NeurodataWithoutBorders/pynwb/pull/1941)
- Removed `SpatialSeries.bounds` field that was not functional. This will be fixed in a future release. @rly [#1907](https://github.com/NeurodataWithoutBorders/pynwb/pull/1907), [#1996](https://github.com/NeurodataWithoutBorders/pynwb/pull/1996)
- Added support for `NWBFile.was_generated_by` field. @stephprince [#1924](https://github.com/NeurodataWithoutBorders/pynwb/pull/1924)
- Added support for `model_number`, `model_name`, and `serial_number` fields to `Device`. @stephprince [#1997](https://github.com/NeurodataWithoutBorders/pynwb/pull/1997)
- Deprecated `EventWaveform` neurodata type. @rly [#1940](https://github.com/NeurodataWithoutBorders/pynwb/pull/1940)
- Deprecated `ImageMaskSeries` neurodata type. @rly [#1941](https://github.com/NeurodataWithoutBorders/pynwb/pull/1941)

These updates were indented to indicate schema 2.8.0 related changes

Comment on lines +105 to +107
from pynwb import read_nwb

nwbfile = read_nwb(filepath)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think once the simplified read_nwb with streaming is supported we may want to go through the other tutorials that use io.read() in read only mode and update those as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants