feat - Fixed Array Size for collections (support duckdb VSS)

[rapatel0]

Is your feature request related to a problem?

I would like to define a fixed array type. This is important for UDFs and usage of the VSS extension of duckdb. Also likely introduces performance advantages for many embeddings used in machine learning.

Current workaround is raw_sql queries and manual registration of function using con.con in duckdb.

What is the motivation behind your request?

Duckdb VSS extension requires fixed array sizes for computing using array_similarity and array_cosine_similarity. using these with native ibis is not possible because there is no way to express fixed collection types in the ibis signature

Example Error:

---------------------------------------------------------------------------
BinderException                           Traceback (most recent call last)
array_cosine_similarity(ibis.array(first_embeddings), dd.embeddings.cast('array<float>')).execute()
....
BinderException: Binder Error: No function matches the given name and argument types 'array_cosine_similarity(DOUBLE[], DOUBLE[])'. You might need to add explicit type casts.
	Candidate functions:
	array_cosine_similarity(FLOAT[ANY], FLOAT[ANY]) -> FLOAT
	array_cosine_similarity(DOUBLE[ANY], DOUBLE[ANY]) -> DOUBLE
	

Tried multiple workarounds to try to mitigate this but nothing obvious.

Describe the solution you'd like

Fixed size array type:
.cast('array<float32, 64>')

or or pass through casting to underlying datatype
cast('FLOAT[64]')

What version of ibis are you running?

9.3.0

What backend(s) are you using, if any?

local and remote parquet on s3 via duckdb and HTTPFS filesystem

Code of Conduct

• I agree to follow this project's Code of Conduct