Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce Template query #16818

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

mingshl
Copy link
Contributor

@mingshl mingshl commented Dec 9, 2024

Description

This Pull Request introduces the following key enhancements:

1. New 'Template' Query Type:

Implements a new query type that holds an object of query content.
Supports placeholders within the query content.
Passes the query content to search request processing.

2. Query Rewrite Context Refactoring:

Query Rewrite Context now carries the pipelineContext variables after all search request processors processed.
Template queries resolve placeholders during query rewrites.
Inner queries are constructed dynamically based on resolved placeholders.

Related Issues

#16823

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@mingshl
Copy link
Contributor Author

mingshl commented Dec 9, 2024

@reta this PR addressed the issue #16823, can you please help move the issue to OpenSearch Project?

Copy link
Contributor

github-actions bot commented Dec 9, 2024

❌ Gradle check result for ff1acc0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@mingshl
Copy link
Contributor Author

mingshl commented Dec 10, 2024

updated git rebase HEAD~3 --signoff

Copy link
Contributor

❌ Gradle check result for cf97416: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@reta
Copy link
Collaborator

reta commented Dec 10, 2024

@reta this PR addressed the issue opensearch-project/OpenSearch#16823, can you please help move the issue to OpenSearch Project?

@mingshl I sadly cannot do it (not a maintainers on ml-commons), @dblock @andrross could you?

@mingshl
Copy link
Contributor Author

mingshl commented Dec 10, 2024

@reta this PR addressed the issue opensearch-project/OpenSearch#16823, can you please help move the issue to OpenSearch Project?

@mingshl I sadly cannot do it (not a maintainers on ml-commons), @dblock @andrross could you?

Thanks @reta. I will reach out.

@mingshl mingshl added the backport 2.x Backport to 2.x branch label Dec 13, 2024
Copy link
Contributor

❌ Gradle check result for c3c31aa: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

msfroh and others added 10 commits December 15, 2024 16:02
Make QueryRewriteContext an interface. Rename existing impl to
BaseQueryRewriteContext. Create coordinator-level context called
QueryCoordinatorContext. It can expose pipelined search request,
including PipelineProcessingContext to rewrite methods.

Signed-off-by: Mingshi Liu <[email protected]>
Introduce template query that holds the content of query which can contain placeholders and can be filled by the variables from PipelineProcessingContext produced by search processors. This allows query rewrite by the search processors.
Signed-off-by: Mingshi Liu <[email protected]>
Signed-off-by: Mingshi Liu <[email protected]>

Add changelog

Signed-off-by: Mingshi Liu <[email protected]>
Signed-off-by: Prudhvi Godithi <[email protected]>
Signed-off-by: Mingshi Liu <[email protected]>
…Service is not in sync with cluster-state (opensearch-project#16763)

* Skip remote-repositories validations for node-joins when RepositoriesService is not in sync with cluster-state

Signed-off-by: Pranshu Shukla <[email protected]>
Signed-off-by: Mingshi Liu <[email protected]>
…/hdfs-fixture (opensearch-project#16807)

* Bump com.nimbusds:nimbus-jose-jwt in /test/fixtures/hdfs-fixture

Bumps [com.nimbusds:nimbus-jose-jwt](https://bitbucket.org/connect2id/nimbus-jose-jwt) from 9.46 to 9.47.
- [Changelog](https://bitbucket.org/connect2id/nimbus-jose-jwt/src/master/CHANGELOG.txt)
- [Commits](https://bitbucket.org/connect2id/nimbus-jose-jwt/branches/compare/9.47..9.46)

---
updated-dependencies:
- dependency-name: com.nimbusds:nimbus-jose-jwt
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Mingshi Liu <[email protected]>
* Fixing _list/shards API for closed indices

Signed-off-by: Harsh Garg <[email protected]>
Signed-off-by: Mingshi Liu <[email protected]>
…roject#16678)

* Added implementation for the stats calculation for search and regular replica in shards

Signed-off-by: Vinay Krishna Pudyodu <[email protected]>

* Updated changelog

Signed-off-by: Vinay Krishna Pudyodu <[email protected]>

* Added unit tests for TransportSegmentReplicationStatsAction

Signed-off-by: Vinay Krishna Pudyodu <[email protected]>

* fixed java style after running precommit locally

Signed-off-by: Vinay Krishna Pudyodu <[email protected]>

* refined the test cases

Signed-off-by: Vinay Krishna Pudyodu <[email protected]>

* fixed style issues

Signed-off-by: Vinay Krishna Pudyodu <[email protected]>

* Made changes in the bytes to download calculation based on comments

Signed-off-by: Vinay Krishna Pudyodu <[email protected]>

* added addReplicaStats method to SegmentReplicationPerGroupStats

Signed-off-by: Vinay Krishna Pudyodu <[email protected]>

* fixed style issues

Signed-off-by: Vinay Krishna Pudyodu <[email protected]>

* Fixed issue with immutable set

Signed-off-by: Vinay Krishna Pudyodu <[email protected]>

* Fixed PR comments and moved the integration tests to separate module

Signed-off-by: Vinay Krishna Pudyodu <[email protected]>

* Fixed failing integ tests

Signed-off-by: Vinay Krishna Pudyodu <[email protected]>

* Fixed failing integ test

Signed-off-by: Vinay Krishna Pudyodu <[email protected]>

* fixed some comments for PR

Signed-off-by: Vinay Krishna Pudyodu <[email protected]>

* fixed failing tests

Signed-off-by: Vinay Krishna Pudyodu <[email protected]>

---------

Signed-off-by: Vinay Krishna Pudyodu <[email protected]>

remove license

Signed-off-by: Mingshi Liu <[email protected]>
Copy link
Contributor

❌ Gradle check result for 4555938: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants