Ragged dtype=object BaseSignal saving .hspy should give better error

[magnunor]

When trying to save a ragged BaseSignal with dtype=object, the error message is confusing for users. Example:

import numpy as np
import hyperspy.api as hs

data = np.empty((3, 5), dtype=object)
for ix in range(data.shape[-1]):
    for iy in range(data.shape[-2]):
        data[iy, ix] = np.random.randint(1, 100, size=(5, 2))

s = hs.signals.BaseSignal(data, ragged=True)
s.save("test.hspy", overwrite=True)

Gives the error message TypeError: Object dtype dtype('O') has no native HDF5 equivalent. This is due to h5py not supporting dtype=object.

However, the error message could be improved to something like: Saving ragged signals with dtype=object is not supported, try saving it as the .zspy format instead.

[CSSFrancis]

@magnunor That is a good point! I ran into this the other day as well.

That being said I think we need to improve the performance of the read for ragged arrays. I don't know if this is a regression from zarr but currently the time to load a ragged arrays seems to increase exponentially with the number of navigation positions. This causes it to become fairly unusable when you have even 100x100 navigation positions.

I'll see if I can look more into this...

[ericpre]

The issue is rather that hspy should be able to save this type of data. A workaround may be to unfold the signal when writing (write with navigation dimension 1) and fold it to the correct navigation shape when reading - see h5py/h5py#876.

[ericpre]

Fixed in #164.