[QST] Prepare a triton inference request with a is_list, is_ragged column

[BowlingX]

❓ Questions & Help

Details

Hi. I have a request schema like

request_schema = Schema(
        [
            ColumnSchema("my_column_a", dtype=object, is_list=True, is_ragged=True),
            ColumnSchema("my_column_b", dtype=object, is_list=True, is_ragged=True)
           # ... other columns
        ]
    )

And a pandas DataFrame I would like to use for my inference request e.g.

myData = {
  "my_column_a": ["my", "values"],
  "my_column_b": ["my", "other", "values"]
}

df = pandas.DataFrame(data=myData)

To send this request, I'm currently using the convert_df_to_triton_input method from the package merlin.systems.triton.

However, It seems like the conversion is unsupported currently, because I receive the following exception (see

systems/merlin/systems/triton/__init__.py

Line 168 in 92340ba

raise ValueError("this function doesn't support CPU list values yet")

)

Is there another way to prepare the request? Is it just not implemented?

I'm using Multi-Hot encoding for some columns and the export of the Ensemble worked well. It's just unclear what is currently expected for those inputs:

input {
  name: "my_column_a__values"
  data_type: TYPE_STRING
  dims: -1
}
input {
  name: "my_column_a__offsets"
  data_type: TYPE_INT32
  dims: -1
}
....

Thank you very much!

[BowlingX]

Ok, it seems it is only implemented on the GPU with a cuda df. Is there a way to make it run on the CPU?

[BowlingX]

I adjusted the method to support panda dataframes and it worked properly. Will create a PR for the missing implementation. It's already implemented here: https://github.com/NVIDIA-Merlin/systems/blob/main/merlin/systems/dag/ops/feast.py#L247 and can be adapted.