Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Continuously index documents in opensearch, but there is no change in the number of documents queried through the interface #12123

Closed
liuwz02 opened this issue Feb 1, 2024 · 3 comments
Labels
bug Something isn't working Other

Comments

@liuwz02
Copy link

liuwz02 commented Feb 1, 2024

Describe the bug

when i continuously index documents in opensearch, but there is no change in the number of documents queried through the interface _stats/docs,store. However, when I stopped indexing documents in opensearch, the number of documents returned by the interface suddenly increased.

Related component

Other

To Reproduce

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior

When I continue to index documents in opensearch and use the interface to query the number of documents, the interface should return the real-time number of documents in the current time index.
https://opensearch.org/docs/latest/api-reference/index-apis/stats/

Additional Details

Plugins
Please list all plugins currently enabled.

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

  • OS: [e.g. iOS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

@liuwz02 liuwz02 added bug Something isn't working untriaged labels Feb 1, 2024
@github-actions github-actions bot added the Other label Feb 1, 2024
@shwetathareja
Copy link
Member

When you are indexing documents, the documents are not available for search immediately.

By default, OpenSearch refreshes indexes every second, i.e. segments are created and documents are available for search. OpenSearch only refreshes indexes that have received at least one search request in the last 30 seconds.
https://opensearch.org/docs/latest/tuning-your-cluster/performance/#index-refresh-interval

You can run continuous search and configure the refresh interval explicitly depending on how soon you want documents to be available for search.

@peternied
Copy link
Member

peternied commented Mar 6, 2024

[Triage - attendees 1 2 3 4 5]
@liuwz02 This sounds like it could also be related to search idle behavior. However, there is not enough detail in this issue to reproduce so we are going to close this issue.

@andrross Could you provide more details if there might be a related bug?

@andrross
Copy link
Member

andrross commented Mar 6, 2024

@liuwz02 This looks like the same issue as described in #11225.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Other
Projects
None yet
Development

No branches or pull requests

4 participants