Selection of BLOCK_SIZE in create_block_mask

[tsrikris]

In the create_block_mask function, the BLOCK_SIZE parameter seems to be control the size of the sub-blocks that leverage block sparsity. Do you have any recommendations on tuning this parameter? I can imagine this is a function of the document size in Document Masking.

[Chillee]

Generally, I would recommend sticking with BLOCK_SIZE=128 or above, unless you're constrained by memory. We're limited in that the tile sizes chosen within the kernel itself must divide into BLOCK_SIZE of BlockMask.