feat: Ergonomic casts from Categorical to Enum

[alexander-beedie]

ref: #19868 (comment).

Allow for stable ergonomic casts from Categorical to Enum.

If no categories are defined, we read the Categorical rev_map and normalise to a lexically sorted Enum (which ensures that if the same categories appear in a different order they will return the same Enum).

Example

import polars as pl

df = pl.DataFrame({
  "id": [0, 1, 2, 3],
  "foo": pl.Series(["aaa", "bbb", "aaa", "bbb"], dtype=pl.Categorical),
  "bar": pl.Series(["zzz", "xxx", "xxx", "yyy"], dtype=pl.Categorical),
  "baz": pl.Series(["xxx", "yyy", "zzz", "xxx"], dtype=pl.Categorical),
})

Before:
(note that we have to sort the categories to ensure a stable Enum)

df.cast({
  "foo": pl.Enum(sorted(df["foo"].cat.get_categories())),
  "bar": pl.Enum(sorted(df["bar"].cat.get_categories())),
  "baz": pl.Enum(sorted(df["baz"].cat.get_categories())),
})

After:
(no boilerplate, and the resulting enums will be stable/sorted)

df.cast({pl.Categorical: pl.Enum})

[ritchie46]

I am not sure about this. The fields in an enum belong to the data type. In a categorical they are inferred at runtime.

[codecov]

Codecov Report

Attention: Patch coverage is 93.75000% with 1 line in your changes missing coverage. Please review.

Project coverage is 79.01%. Comparing base (6cd9988) to head (90521e2).

Files with missing lines	Patch %	Lines
...-core/src/chunked_array/logical/categorical/mod.rs	92.85%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #20645   +/-   ##
=======================================
  Coverage   79.01%   79.01%           
=======================================
  Files        1557     1557           
  Lines      220552   220560    +8     
  Branches     2514     2514           
=======================================
+ Hits       174274   174284   +10     
+ Misses      45705    45703    -2     
  Partials      573      573           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[alexander-beedie]

I am not sure about this. The fields in an enum belong to the data type. In a categorical they are inferred at runtime.

Yup; it's an ergonomic improvement to something that may not actually be a good idea, heh. I'll park in Draft so you can have a think about it; not a big PR - feel free to close if you prefer not to merge ;)

[coastalwhite]

I don't think "lexically sorted enum"s exist. That will panic in many places.

[mcrumiller]

FYI there's a little relevant discussion in #14952.

[alexander-beedie]

I don't think "lexically sorted enum"s exist. That will panic in many places.

As in, the Enum gets created with the values having been sorted (as the same Categorical values coming from several sources could have a very different physical ordering, but -if we were to merge this- we'd want the Enum equivalent to resolve consistently). I could have worded that better ;)

[s-banach]

I just wanted to freeze an existing categorical encoding, e.g. the output of cut(). Definitely didn’t imagine it would change the ordering.