Problem with demo

[HARISKHAN-1729]

Hi,

I am trying to run the model, but whenever I try to execute the demo.app, the following error appears:

RuntimeError: batch size must be positive

This error seems to be related to a batch size issue within the model’s execution flow. I have attached a screenshot of the full traceback for reference. It looks like the error is being triggered during the forward pass, possibly involving flash_attn_varlen_func and batch size handling.

I would appreciate any guidance on how to resolve this issue. Thank you!