Named queries should return their internal score as well

[synhershko]

Description

While some internal queries may not be scored or not contrbute to the final doc score, in some cases (such as knn queries compbined with other queries) the distance metric is very interesting and should be reported back as part of the response.

This change adds that by default, so named queries always return the score (instead of just the name). It adds no additional cost.

Check List

• New functionality includes testing.
  • All tests pass
  • Backwards compatibility tests fail - let's have a discussion about what should be expected here since the response format did change when named queries are enabled
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/16718/
• CommitID: 714d802
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/16719/
• CommitID: f59816f
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/16720/
• CommitID: 30d13ef
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/16725/
• CommitID: 5b35d5c
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/16729/
• CommitID: 7869184
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/16797/
• CommitID: e715622
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[synhershko]

I'm just wondering what additional details we can provide there in the future? I'm calling YAGNI on it and let's do something simple that works first and worry about the rest later. I just don't see anything else we can possible want to provide other than score.

[macohen]

I understand why we would want the query string parameter here, but, to answer @dblock's question for me, I don't know of any precedent for this in OpenSearch. Even the query_string function is all in the body of the request and I really like that about OpenSearch (as a former old-school Solr user). I don't have a better answer for this either that doesn't create a breaking change to push this to 3.0 unfortunately.

[dblock]

Let's make a breaking change for 3.0 in this PR? I feel pretty strongly that we should avoid this problem for future changes here and do:

"matched_queries" : {
     "foo_query" : {
        "score": 0.2876821
    }
}

and not

"matched_queries" : {
     "foo_query" : 0.2876821
}

We can discuss a non-breaking version on a backport?

[opensearch-trigger-bot]

This PR is stalled because it has been open for 30 days with no activity. Remove stalled label or comment or this will be closed in 7 days.

[opensearch-trigger-bot]

This PR was closed because it has been stalled for 7 days with no activity.

[kotwanikunal]

Apologies. This PR was auto closed without reaching a resolution from the maintainers.
Re-opening to move it forward.
Thanks for your contributions to OpenSearch!

[github-actions]

Compatibility status:

Checks if related components are compatible with change 4dbf0ad

Incompatible components

Skipped components

Compatible components

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/25625/
• CommitID: 4dbf0ad
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[synhershko]

Thanks @kotwanikunal looking forward to final decision, let's get this in!

[ticheng-aws]

Hi @synhershko, the PR is stalled. Is this being worked upon? Feel free to reach out to maintainers for further reviews.

[synhershko]

@ticheng-aws I'm well aware this is stalled, see my comment #7883 (comment) and previous discussions. When maintainers pick it up and conclude their discussion I'm happy to pick it up again.

[synhershko]

cc @dblock @macohen

[dblock]

I think #7883 (comment) is still accurate? A breaking change for main, do we need more maintainers to pitch in? WDYT @msfroh / @andrross?

[andrross]

I think #7883 (comment) is still accurate? A breaking change for main, do we need more maintainers to pitch in?

@dblock I agree with your suggestion. I'll ship this PR as a breaking change to main with the structure you proposed. I think we can deliver this in 2.x with some sort of yet-to-be-decided opt-in mechanism as well, but we can work that out once this has landed in main.

[reta]

@dblock @andrross I believe the feature is already integrated #11626

[dblock]

@synhershko take a look? what shall we do with this PR?

[opensearch-trigger-bot]

This PR is stalled because it has been open for 30 days with no activity.

[andrross]

I believe this has been implemented in #11626. I'm going to close this. @synhershko Please do speak up if #11626 doesn't solve your problem.