Optimize bins filtering by merging bins #11248
Conversation
fuzhaoyuan
force-pushed
the
demo-rfc80-poc-bin-range-optimization
branch
2 times, most recently
from
1f0e84c
to
8f87492
Compare
fuzhaoyuan
force-pushed
the
demo-rfc80-poc-bin-range-optimization
branch
from
5fbbefd
to
9e32067
Compare
fuzhaoyuan
force-pushed
the
demo-rfc80-poc-bin-range-optimization
branch
from
9e32067
to
ad021d6
Compare
fuzhaoyuan
force-pushed
the
demo-rfc80-poc-bin-range-optimization
branch
from
ad021d6
to
abf0718
Compare
|
@fuzhaoyuan Looks good. Just fix the one sonar issue and we are good |
Fixed. Thank you! |
|
src/test/java/org/cbioportal/service/util/StudyViewColumnarServiceUtilTest.java
Show resolved
Hide resolved
| /** | ||
| * Merge the range of numerical bins in DataFilters to reduce the number of scans that runs on the database when filtering. | ||
| */ | ||
| private static <T extends DataFilter> List<T> mergeDataFilters(List<T> filters) { |
There was a problem hiding this comment.
We should probably make this public and add a test for this one as well. I noticed that we only have negative values in the tests, may be good to have positive values as well. A mix of positive and negative values would be even better I guess.
There was a problem hiding this comment.
By public you mean we could probably use it somewhere else right? That makes sense. I'm still trying to decide where it's the best place to put it. The test part is easy done
src/main/java/org/cbioportal/service/util/StudyViewColumnarServiceUtil.java
Show resolved
Hide resolved
| List<DataFilterValue> mergedValues = new ArrayList<>(); | ||
|
|
||
| BigDecimal mergedStart = null, mergedEnd = null; | ||
| for (DataFilterValue dataFilterValue : filter.getValues()) { |
There was a problem hiding this comment.
this logic only works if the bins are sorted properly, right? are we sure that bins are always sorted the way we want?
There was a problem hiding this comment.
I had 2 assumptions: 1) they come in sorted and 2) they're continuous. I handled the discontinuous case by adjacent merging but if sorting is needed we'll need an extra part and way more combinations of possible cases. What would you suggest?
Describe changes proposed in this pull request: