Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes Numeric exact match queries to use range queries internally #11209

Merged
merged 18 commits into from
Jan 4, 2024

Conversation

harshavamsi
Copy link
Contributor

@harshavamsi harshavamsi commented Nov 15, 2023

Description

Updates term and terms queries for numeric fields to use IndexOrDocValues where appropriate.

Related Issues

Resolves #11097

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)
  • Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions github-actions bot added bug Something isn't working Search:Performance v2.12.0 Issues and PRs related to version 2.12.0 labels Nov 15, 2023
Copy link
Contributor

❌ Gradle check result for 6a324a4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

github-actions bot commented Nov 15, 2023

Compatibility status:

Checks if related components are compatible with change 339a681

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/performance-analyzer.git]

Copy link
Contributor

❌ Gradle check result for 443e205: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 36708f8: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for d97865e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for bbd4e4e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 1289b3e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 27f4e47: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❕ Gradle check result for c5acd4c: UNSTABLE

  • TEST FAILURES:
      1 org.opensearch.index.shard.RemoteIndexShardTests.testRepicaCleansUpOldCommitsWhenReceivingNew

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Copy link

codecov bot commented Nov 16, 2023

Codecov Report

Attention: 166 lines in your changes are missing coverage. Please review.

Comparison is base (8440468) 71.44% compared to head (339a681) 70.90%.
Report is 3 commits behind head on main.

Files Patch % Lines
...org/opensearch/index/mapper/NumberFieldMapper.java 40.26% 68 Missing and 21 partials ⚠️
.../document/SortedUnsignedLongDocValuesSetQuery.java 15.09% 43 Missing and 2 partials ⚠️
.../main/java/org/apache/lucene/util/LongHashSet.java 39.62% 21 Missing and 11 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #11209      +/-   ##
============================================
- Coverage     71.44%   70.90%   -0.54%     
+ Complexity    59267    58864     -403     
============================================
  Files          4909     4911       +2     
  Lines        278427   278651     +224     
  Branches      40460    40524      +64     
============================================
- Hits         198910   197585    -1325     
- Misses        63004    64661    +1657     
+ Partials      16513    16405     -108     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@harshavamsi
Copy link
Contributor Author

Gradle failure is flaky test -- #5329

Copy link
Contributor

github-actions bot commented Jan 3, 2024

❌ Gradle check result for 430310a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>
Copy link
Contributor

github-actions bot commented Jan 3, 2024

❌ Gradle check result for b54a539: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@reta
Copy link
Collaborator

reta commented Jan 3, 2024

@harshavamsi please rebase against latest main, thank you

Copy link
Contributor

github-actions bot commented Jan 3, 2024

❕ Gradle check result for ebba45f: UNSTABLE

  • TEST FAILURES:
      2 org.opensearch.remotestore.RemoteIndexPrimaryRelocationIT.testPrimaryRelocationWhileIndexing
      1 org.opensearch.index.shard.RemoteIndexShardTests.testNRTReplicaWithRemoteStorePromotedAsPrimaryCommitRefresh

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

@reta reta self-requested a review January 3, 2024 20:37
@reta
Copy link
Collaborator

reta commented Jan 3, 2024

@msfroh could you please take a look? (waiting for your signoff) thank you

CHANGELOG.md Outdated Show resolved Hide resolved
@msfroh
Copy link
Collaborator

msfroh commented Jan 3, 2024

Just a residual merge conflict on the changelog, where you picked up a couple of other changes. Otherwise, it looks good to me 👍

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>
@harshavamsi
Copy link
Contributor Author

Just a residual merge conflict on the changelog, where you picked up a couple of other changes. Otherwise, it looks good to me 👍

Thanks @msfroh, fixed the changelog.

Copy link
Contributor

github-actions bot commented Jan 3, 2024

❌ Gradle check result for 339a681: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

github-actions bot commented Jan 4, 2024

❕ Gradle check result for 339a681: UNSTABLE

  • TEST FAILURES:
      2 org.opensearch.index.shard.RemoteStoreRefreshListenerTests.classMethod
      1 org.opensearch.search.SearchTimeoutIT.testSimpleTimeout {p0={"search.concurrent_segment_search.enabled":"true"}}
      1 org.opensearch.remotestore.multipart.RemoteStoreMultipartIT.testOverriddenBufferInterval
      1 org.opensearch.index.shard.RemoteStoreRefreshListenerTests.testRefreshAfterCommit

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

@reta reta added the v3.0.0 Issues and PRs related to version 3.0.0 label Jan 4, 2024
@reta reta merged commit 178a7a0 into opensearch-project:main Jan 4, 2024
32 of 33 checks passed
@reta reta added the backport 2.x Backport to 2.x branch label Jan 4, 2024
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jan 4, 2024
…1209)

* Updating numeric term and terms queries to use IODVQ

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Addressing comments

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Fix formatting

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Fix changelog

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Addressing more comments + adding tests

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* renaming yaml test

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Adding skip for bwc

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Adding new SortedUnsignedLongDocValuesSetQuery to allow for BitInteger Terms query

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Fixing some tests

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Remove duplicate skip

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Remove unused points declaration

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Change unsigned exact query to be consistent

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Use slowExactQuery from Unsigned Set Query

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Merging different yaml tests into a single test

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Updating test case for main

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Fix changelog

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

---------

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>
(cherry picked from commit 178a7a0)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
reta pushed a commit that referenced this pull request Jan 4, 2024
…1209) (#11747)

* Updating numeric term and terms queries to use IODVQ



* Addressing comments



* Fix formatting



* Fix changelog



* Addressing more comments + adding tests



* renaming yaml test



* Adding skip for bwc



* Adding new SortedUnsignedLongDocValuesSetQuery to allow for BitInteger Terms query



* Fixing some tests



* Remove duplicate skip



* Remove unused points declaration



* Change unsigned exact query to be consistent



* Use slowExactQuery from Unsigned Set Query



* Merging different yaml tests into a single test



* Updating test case for main



* Fix changelog



---------


(cherry picked from commit 178a7a0)

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
rayshrey pushed a commit to rayshrey/OpenSearch that referenced this pull request Mar 18, 2024
…ensearch-project#11209)

* Updating numeric term and terms queries to use IODVQ

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Addressing comments

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Fix formatting

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Fix changelog

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Addressing more comments + adding tests

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* renaming yaml test

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Adding skip for bwc

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Adding new SortedUnsignedLongDocValuesSetQuery to allow for BitInteger Terms query

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Fixing some tests

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Remove duplicate skip

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Remove unused points declaration

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Change unsigned exact query to be consistent

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Use slowExactQuery from Unsigned Set Query

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Merging different yaml tests into a single test

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Updating test case for main

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Fix changelog

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

---------

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
…ensearch-project#11209)

* Updating numeric term and terms queries to use IODVQ

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Addressing comments

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Fix formatting

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Fix changelog

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Addressing more comments + adding tests

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* renaming yaml test

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Adding skip for bwc

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Adding new SortedUnsignedLongDocValuesSetQuery to allow for BitInteger Terms query

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Fixing some tests

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Remove duplicate skip

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Remove unused points declaration

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Change unsigned exact query to be consistent

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Use slowExactQuery from Unsigned Set Query

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Merging different yaml tests into a single test

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Updating test case for main

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

* Fix changelog

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>

---------

Signed-off-by: Harsha Vamsi Kalluri <[email protected]>
Signed-off-by: Shivansh Arora <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch bug Something isn't working Search:Performance v2.12.0 Issues and PRs related to version 2.12.0 v3.0.0 Issues and PRs related to version 3.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Performance of numeric exact-match queries can be improved
4 participants