Determining unset or empty []string attribute

[egoldschmidt]

We are adding a new attribute that contains a filterable array of strings, but we want backwards compatibility with old data that does not have this set yet. So my thinking was to leverage a filter of the form [x, 'Eq', null] or [x, 'Eq' []]. Alas neither seems to work:

filter error in key x: type mismatch, Eq expects array filter operator, but got null

or

filter error in key x: type mismatch, Eq expects array filter operator, but got []

Please let me know if there is a better way to approach this, thanks!

[pushrax]

Can you clarify the use case? I'm not following how "we want backwards compatibility with old data" would lead to querying for null or [].

The error message you're seeing seems like it needs to be improved, it should read something like Eq cannot be used on an array attribute

[egoldschmidt]

I was intending to have a query filter that looked for records with the attribute set OR the attribute entirely unset (implying it was legacy data, which would be backfilled over time, and for which it was OK to preserve existing behavior)

E.g., ['Or', [['attribute', 'Eq', null], ['attribute', 'In', ['value1', 'value2']]]]

But my use case aside: is there any way to check for an unset or empty array?

[pushrax]

At the moment we don't have an index that can serve a query that's filtering by array equality. The index we build to accelerate In / NotIn uses the individual array elements as keys, so an empty array doesn't appear in this index at all.

This is definitely a reasonable feature request.

Here's a possible workaround you can use: include a schema generation number on each row, and filter with that.

['Or', [['generation', 'Lt', 1], ['attribute', 'In', ['value1', 'value2']]]]

(note that null < 1)