testing nwb file compatibility with nwbe

[anhknguyen96]

With the current version of test_nwbe_compatibility function in the nwb_table_readme.py script (dev branch), a nwb file compatibility with NWBE will be tested and classified as:

• LIKELY: file can be opened using pynwb (hence the NWBReader class in NWBE functions) and metadata is displayed on the webapp, with the possibility of its time series data being visualized.
• NC - 0: Not compatible level 0 - file can't be opened using pynwb
• NC - 1: Not compatible level 1 - a geppetto model for the file can't be generated after 60s (timeout exception raised in the function), most probably getting stuck somewhere here
• NI: No information - file is not tested, most probably because its size exceeds the 1GB limit in the script.

That said, the test does not cover instances where a nwb file is able to be opened and displayed in NWBE, but its time series data can't be visualized, or other data types available in nwb format.

[anhknguyen96]

@pgleeson @sanjayankur31 : what do you think about the classification and how the testing is being done? Do we want to cover instances of data being unable to be visualized?

[pgleeson]

That looks like a good classification. One addition might be at the "top" level, that it can be opened and there are datatypes present in the file which extend TimeSeries, i.e. there is something interesting to plot in there. This won't be the case (yet) for datatsets which just contain unit recording data. So maybe 2 categories LIKELY VIEWABLE, LIKELY PLOTTABLE.

[anhknguyen96]

LIKELY VIEWABLE and LIKELY PLOTTABLE classes added, classified based on type hierarchy in the file

[pgleeson]

Thanks @anhknguyen96. Could you add some info on the Readme about the sizes of the 2 chosen NWB files?

Also can the BIDS dandisets appear in the main part too, without validation info, but with keywords, etc.?

[anhknguyen96]

BIDS dandisets are already included in the main part in the latest commit, with information such as number of dandisets, median file sizes and median file numbers. Anything further than that I would need some time to look at them

[pgleeson]

Thanks. Yes, a brief bit of info on each of the BIDS dandisets (title/keywords/source paper) would be great if possible.

[anhknguyen96]

linking this issue as an instance of nwbe compatibility not covered by this classification

[pgleeson]

Core features implemented and live!