Masked arrays

[jdroenner]

Numpy supports masked arrays which is very useful for many use-cases. Are there any plans how a similar feature could be implemented for rust-ndarray?

My current idea would be to build a decorator which wraps an array and contains an additional mask object: struct MaskedArray<A, M>{ array: A, mask: M, }

The mask itself should be a trait (and/or enum?) to handle the following cases:

• empty (no elements are masked)
• hard mask: a bool array with the same shape as the data array (masking by coordinate)
• no-data value (masking by value)
• predicate (masking by value and/or coordinate)

The problem with this approach is that there is currently no array trait which is needed for the decorator pattern.

[bluss]

Not much plans. It's needed for sure. We need an implementation that is easy to optimize well, at least for the common cases.

One question that arises for example, in the API is it the arrays or the operations on them that should be masked?

Also mentioned in #120

[jdroenner]

This is a good question. I guess it is more explicit if the operations are masked. This way it is always clear that masked cells are not really empty/non-existent. Additionally it may lead to more generic code e.g. write a histogram generator for IntoIterator<Item=T> instead of Array<T>.

Masked arrays are more intuitive then masked operations. The result of a get(...) wouldn't need an additional check if it is masked. There could be additional functionality like a is_masked(...) function. Also slicing a single masked array is more intuitive than slicing a data array and a mask array...

I think i like the idea of masked operations and they should work well for my use-cases (raster data). But there may be other opinions and use-cases.

[andrei-papou]

Hi @bluss . Is this feature still relevant? If yes, then I can work on POC of either of both approaches (masked arrays or masked operations).

[GeoffChurch]

+1 I think masked indexing (along with arrays-as-indices and/or where/nonzero) is an important step toward being able to transparently port readable scalar code to use ndarrays.

[hdamron17]

What is the status of masked arrays in nadarray? I am interested in taking a stab at it.

[andrei-papou]

@bluss @jturner314 @LukeMathWalker I've created a draft of the potential masked array implementation #849 . Could you please take a look and see if it makes any sense and if I should continue moving in this direction? Thanks a lot.