Slot-aligned tensors

[billschereriii]

Description

Allow users to pick the slots in which tensors are stored. This should open up an opportunity for advanced users to get improved performance with scripts and models.

Longer explanation

In RedisAI, the AI.SCRIPTRUN/EXECUTE and AI.MODELRUN/EXECUTE commands require that all input and output tensors be aligned to the same slot as the script or model itself. If this is not the case, execution is aborted and a CROSSSLOTS (cross slots) error is issued. In order to ensure this prerequisite, the SmartRedis library currently copies input tensors to the slot with the model/script and then copies output to the target destination. Moving computation to the data instead of data to the computation is a well known performance optimization.

Justification

Advanced users will be able to get better performance from their models and scripts using this approach.

Implementation Strategy

We would add parallel API points for clustered Redis for the following:

• AlignTensor(): move a tensor to a known slot
• StoreTensor(): write a tensor a known slot
• SetModelAligned(): Store a model to a known slot
• SetScriptAligned(): Store a script to a known slot
• RunModelAligned(): Execute a model with aligned tensors. No copy of tensor data is performed and CROSSSLOTS errors are returned to users if they occur
• RunScriptAligned(): Execute a script with aligned tensors. No copy of tensor data is performed and CROSSSLOTS errors are returned to users if they occur

Documentation updates, including a tutorial, would be very helpful to make users understand how to use this functionality.