You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I encountered a bug where querying all documents from collections with lots of documents.
I identified that the problem was how mongodb_filte.py handled filtering. Currently filtering
is done using multiple aggregate queries to mongodb using document ID's that matched
previous filters, this can lead to a situation where a database contains lots of documents
the hard limit of 16 megabyte queries is exceeded and mongodb cannot process the filter. My
changes make it so that instead of multiple filters and queries a single filter is generated
based on the request made and that is passed to mongodb. This way the 16 megabyte query
size limit should never be triggered, and mongodb can process the filter internally.
The text was updated successfully, but these errors were encountered:
To reproduce this bug you would need a mongodb containing documents, which IDs size alone when combined
into a single filter would amount to larger than a 16 megabyte query. This happens due to generating the filter based
on IDs received from previous queries.
I encountered a bug where querying all documents from collections with lots of documents.
I identified that the problem was how mongodb_filte.py handled filtering. Currently filtering
is done using multiple aggregate queries to mongodb using document ID's that matched
previous filters, this can lead to a situation where a database contains lots of documents
the hard limit of 16 megabyte queries is exceeded and mongodb cannot process the filter. My
changes make it so that instead of multiple filters and queries a single filter is generated
based on the request made and that is passed to mongodb. This way the 16 megabyte query
size limit should never be triggered, and mongodb can process the filter internally.
The text was updated successfully, but these errors were encountered: