Add row limit to lookup table uploads

[gherceg]

Product Description

Technical Summary

https://dimagi.atlassian.net/browse/SAAS-16223

As discussed in the ticket, there aren't any existing lookup tables with more than 500k rows, so that seems like a very safe limit to start with. The risky part of this PR is that it adds an explicit limit to the WorkbookJSONReader class, currently set to 1 million. This limit doesn't have to be set to anything necessarily, but I figured in the spirit of implementing limits, putting a limit like this in code seems ideal. On the flip side, it could be viewed as a lazy way of adding a limit to any code that uses this class under the hood (like scheduling).

Feature Flag

Safety Assurance

Safety story

Automated test coverage

QA Plan

No

Rollback instructions

• This PR can be reverted after deploy with no further considerations

Labels & Review

• Risk label is set correctly
• The set of people pinged as reviewers is appropriate for the level of risk of the change

[gherceg]

I think I actually want to replace the implementation of _max_row here to use dimensions instead.

[gherceg]

Just kidding. After reading some source code I realized the dimensions weren't going to work the way I had hoped, though there is some room for a deeper optimization. My understanding is that dimensions are unreliably set depending on the application that created the file in the first place. So we could rework our code to check for the total size of the workbook/worksheet when first loading it, using the dimensions if they are set, otherwise manually iterating over rows, whereas we currently just iterate over all rows to determine table sizes.

Update for clarity: For now, just using the existing max_row call available on the worksheet is sufficient.

Missed Graham's most recent comment. Sounds like this will not work and is incomlete.

[millerdev]

Just kidding. 😄 Apparently this is complete.

[sentry-io]

Suspect Issues

This pull request was deployed and Sentry observed the following issues:

• ‼️ WorkbookTooManyRows /a/{domain}/settings/users/commcare/upload/ View Issue
• ‼️ FixtureUploadError: ['Lookup tables can contain a maximum of 500000 rows. The uploaded file contains 564182 rows.'] corehq.apps.fixtures.tasks.fixture_upload_async View Issue

_{Did you find this useful? React with a 👍 or 👎}