Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]Query support on flat_object field with dotted sub field names is unreliable #11402

Closed
ngnwn opened this issue Nov 30, 2023 · 1 comment · Fixed by #11425
Closed

[BUG]Query support on flat_object field with dotted sub field names is unreliable #11402

ngnwn opened this issue Nov 30, 2023 · 1 comment · Fixed by #11425
Assignees
Labels
bug Something isn't working Search Search query, autocomplete ...etc v2.12.0 Issues and PRs related to version 2.12.0 v3.0.0 Issues and PRs related to version 3.0.0

Comments

@ngnwn
Copy link

ngnwn commented Nov 30, 2023

Describe the bug
I have an index with a field of type “flat_object,” and the JSON object for that field contains dotted sub-field names. I’m attempting to apply a term query using the entire key and value but it doesn't work as expected

To Reproduce
Steps to reproduce the behavior:

  1. Create an index with mapping:
PUT /my-test-os-index
{
  "mappings": {
    "properties": {
      "transcript": {"type": "flat_object"}
    }
  }
}
  1. Index a single document
POST my-test-os-index/_doc
{
  "transcript": {
    "one": "FAILED",
    "two.one": "FAILED",
    "three": "FAILED"
  }
}
  1. Run a term query
POST my-test-os-index/_search
{
  "query": {
    "term": {"transcript.three": "FAILED"}
  }
}
  1. The results are empty

Expected behavior
We expect the one indexed document to be returned cause it matches the criteria

Plugins
I am using OpenSearch 2.9

Host/Environment (please complete the following information):
Using OS serverless

Additional context

POST my-test-os-index/_search
{
  "query": {
    "term": {"transcript.one": "FAILED"}
  }
}

This query retrieves the document making the results order dependent and unreliable

@ngnwn ngnwn added bug Something isn't working untriaged labels Nov 30, 2023
@peternied peternied added Indexing Indexing, Bulk Indexing and anything related to indexing and removed untriaged labels Nov 30, 2023
@peternied
Copy link
Member

Thanks for filing this issue

msfroh added a commit to msfroh/OpenSearch that referenced this issue Nov 30, 2023
We have a bug where a flat object field with inner fields that contain
dots will "push" the dotted name onto the dot-path from the root, but
then would just "pop" off the last part of the dotted name.

This change adds more robust support for flat object keys and subkeys
that contain dots (i.e. it pops off the entirety of the latest key,
regardless of how many dots it contains).

Fixes opensearch-project#11402

Signed-off-by: Michael Froh <[email protected]>
msfroh added a commit to msfroh/OpenSearch that referenced this issue Nov 30, 2023
We have a bug where a flat object field with inner fields that contain
dots will "push" the dotted name onto the dot-path from the root, but
then would just "pop" off the last part of the dotted name.

This change adds more robust support for flat object keys and subkeys
that contain dots (i.e. it pops off the entirety of the latest key,
regardless of how many dots it contains).

Fixes opensearch-project#11402

Signed-off-by: Michael Froh <[email protected]>
@ankitkala ankitkala added Search Search query, autocomplete ...etc and removed Indexing Indexing, Bulk Indexing and anything related to indexing labels Dec 7, 2023
msfroh added a commit to msfroh/OpenSearch that referenced this issue Jan 4, 2024
We have a bug where a flat object field with inner fields that contain
dots will "push" the dotted name onto the dot-path from the root, but
then would just "pop" off the last part of the dotted name.

This change adds more robust support for flat object keys and subkeys
that contain dots (i.e. it pops off the entirety of the latest key,
regardless of how many dots it contains).

Fixes opensearch-project#11402

Signed-off-by: Michael Froh <[email protected]>
reta pushed a commit that referenced this issue Jan 4, 2024
We have a bug where a flat object field with inner fields that contain
dots will "push" the dotted name onto the dot-path from the root, but
then would just "pop" off the last part of the dotted name.

This change adds more robust support for flat object keys and subkeys
that contain dots (i.e. it pops off the entirety of the latest key,
regardless of how many dots it contains).

Fixes #11402

Signed-off-by: Michael Froh <[email protected]>
@github-project-automation github-project-automation bot moved this from 🆕 New to ✅ Done in Search Project Board Jan 4, 2024
@reta reta added v3.0.0 Issues and PRs related to version 3.0.0 v2.12.0 Issues and PRs related to version 2.12.0 labels Jan 4, 2024
msfroh added a commit to msfroh/OpenSearch that referenced this issue Jan 9, 2024
…ct#11425)

We have a bug where a flat object field with inner fields that contain
dots will "push" the dotted name onto the dot-path from the root, but
then would just "pop" off the last part of the dotted name.

This change adds more robust support for flat object keys and subkeys
that contain dots (i.e. it pops off the entirety of the latest key,
regardless of how many dots it contains).

Fixes opensearch-project#11402

Signed-off-by: Michael Froh <[email protected]>
(cherry picked from commit 22b628b)
msfroh added a commit to msfroh/OpenSearch that referenced this issue Jan 10, 2024
…ct#11425)

We have a bug where a flat object field with inner fields that contain
dots will "push" the dotted name onto the dot-path from the root, but
then would just "pop" off the last part of the dotted name.

This change adds more robust support for flat object keys and subkeys
that contain dots (i.e. it pops off the entirety of the latest key,
regardless of how many dots it contains).

Fixes opensearch-project#11402

Signed-off-by: Michael Froh <[email protected]>
(cherry picked from commit 22b628b)
kotwanikunal pushed a commit that referenced this issue Jan 10, 2024
We have a bug where a flat object field with inner fields that contain
dots will "push" the dotted name onto the dot-path from the root, but
then would just "pop" off the last part of the dotted name.

This change adds more robust support for flat object keys and subkeys
that contain dots (i.e. it pops off the entirety of the latest key,
regardless of how many dots it contains).

Fixes #11402

Signed-off-by: Michael Froh <[email protected]>
(cherry picked from commit 22b628b)
rayshrey pushed a commit to rayshrey/OpenSearch that referenced this issue Mar 18, 2024
…ct#11425)

We have a bug where a flat object field with inner fields that contain
dots will "push" the dotted name onto the dot-path from the root, but
then would just "pop" off the last part of the dotted name.

This change adds more robust support for flat object keys and subkeys
that contain dots (i.e. it pops off the entirety of the latest key,
regardless of how many dots it contains).

Fixes opensearch-project#11402

Signed-off-by: Michael Froh <[email protected]>
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this issue Apr 25, 2024
…ct#11425)

We have a bug where a flat object field with inner fields that contain
dots will "push" the dotted name onto the dot-path from the root, but
then would just "pop" off the last part of the dotted name.

This change adds more robust support for flat object keys and subkeys
that contain dots (i.e. it pops off the entirety of the latest key,
regardless of how many dots it contains).

Fixes opensearch-project#11402

Signed-off-by: Michael Froh <[email protected]>
Signed-off-by: Shivansh Arora <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Search Search query, autocomplete ...etc v2.12.0 Issues and PRs related to version 2.12.0 v3.0.0 Issues and PRs related to version 3.0.0
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

5 participants