Max number of rows for table QA

[tiwari93]

Hi,

First of all i want to say thank you for awesome job on table QA part using haystack.
I am trying to use own custom table for QA.
I am able to store and able to get the answer but answer is correct till row no 16 in table.
I tried many tables find same issue.
So just wanted to check do we have limitations here for number of rows in a table and how can we resolve this?

[bglearning]

Hi @tiwari93,
What is your current pipeline? (i.e. what retriever/reader are you using?)

I am able to store and able to get the answer but answer is correct till row no 16 in table.

Do you mean if the answer is within row 16 the pipeline returns the answer, else it does not?

[tiwari93]

Currently I have uploaded table with 21 columns and 30 rows. Which contains various details including sales data for various regions.
So when I ask a question like total sales for west region?

It takes the sum of values till 16th rows and neglect values after this row even though value exits.

I tried this with different different large tables having different number of rows and columns but issue exist. The maximum number of rows selected for aggregation_operator varies but it count till certain rows only and leave others.

I also read that TAPAS model have maximum cell limit of 500. So I guess this is the reason for that.
So just wanted to check how can we perform table qa for big table?

[bglearning]

Hi @tiwari93
Ya, I think it's because of the TAPAS model limit. Might be worth splitting up the tables and trying it that way but not sure. @bogdankostic what do you think?

If using the embedding retriever, it could also be due to this issue: #3240 but I think here it's on the reader(?)

[tiwari93]

@bglearning Thanks for the update. It seems to be some tricky part, trying some filtering options and building some logic for that. lets see.

Thank for your super work.

[bogdankostic]

Yes, TAPAS models have a token limit (as it is the case for all transformer models). Unfortunately, we currently don't support splitting larger tables into smaller subtables and aggregating results after doing the predictions on the subtables.

[tiwari93]

@bogdankostic Thanks for the update