function .isin() rasie Exception: pyarrow.lib import ArrowInvalid: Cannot append scalar of type string to builder for type large_string

[myloe00]

I found out by debugging that while vaex.expression.Expression.to_arrow return a dataset more than one chunk,
like this:

<pyarrow.lib.ChunkedArray object at 0x000002109B68A270>
[
  [
    "B-XXXXX",
   ],
   [ 
     "C1-XXXXX"
   ]
]

this exception will be raised.
So I try to change function vaex.expression.Expression.__arrow_array__ to

def __arrow_array__(self, type=None):
    values = self.to_arrow()
    try:
        res = pa.array(values, type=type)
    except ArrowInvalid as e:
        values = values.combine_chunks()
        res = pa.array(values, type=type)
    return res

And the exception resolved.

So, can i do it like this . Or other method to resovle this problem?

[ddelange]

hi @myloe00 👋

please fill out the bug template including a minimal reproducible example and a full stack trace