[BUG] Unable to restore single backing index for a data stream

[lyradc]

Describe the bug

When attempting to restore a single backing index for a given data stream the resulting index is not added back to the data stream, additionally while the docs count appears correct the newly restored backing index shows a store size of 0, which is not expected.

Related component

Storage:Snapshots

To Reproduce

1. Create a new data stream
2. Add data and rollover the new data stream a few times to get a few backing indices
3. Take a snapshot that includes this newly created data stream
4. Delete one of the backing indices from the newly created data stream (not the current active index, one of the older ones)
5. Restore that one backing index that was deleted in step 4
6. After restore is complete the restored index is not listed as belonging to the original data stream and the restored index will show an expected doc count with 0 size.

Refreshing the index or data stream does not resolve the size issue.

Expected behavior

If the process to restore a single backing index from a data stream differs significantly from the process to restore a single non-datastream index then I would expect that process to be documented.

Otherwise my expectation is that the restored backing index be added back to it's data stream, and that both the doc count and store size reflect the data contained/restored.

Additional Details

Plugins
opensearch-alerting 2.13.0.0
opensearch-anomaly-detection 2.13.0.0
opensearch-asynchronous-search 2.13.0.0
opensearch-cross-cluster-replication 2.13.0.0
opensearch-custom-codecs 2.13.0.0
opensearch-flow-framework 2.13.0.0
opensearch-geospatial 2.13.0.0
opensearch-index-management 2.13.0.0
opensearch-job-scheduler 2.13.0.0
opensearch-knn 2.13.0.0
opensearch-ml 2.13.0.0
opensearch-neural-search 2.13.0.0
opensearch-notifications 2.13.0.0
opensearch-notifications-core 2.13.0.0
opensearch-observability 2.13.0.0
opensearch-performance-analyzer 2.13.0.0
opensearch-reports-scheduler 2.13.0.0
opensearch-security 2.13.0.0
opensearch-security-analytics 2.13.0.0
opensearch-skills 2.13.0.0
opensearch-sql 2.13.0.0
repository-s3 2.13.0

Host/Environment (please complete the following information):

• OS: CentOS7
• Version 2.13.0

[peternied]

[Triage - attendees 1 2 3 4 5 6 7]
@lyradc Thanks for creating this issue, we'd welcome a pull request to address this scenario.

[jed326]

Hey @lyradc whenever the backing index of a data stream is deleted it gets deleted from the datastream metadata as well. Today there is no functionality in OpenSearch to modify a data stream to add indices to it so there's no way to re-add individual backing indices back to the data stream after snapshot restore. This does seem to be a documentation miss as well. There is an open feature request for this though:

• [Feature]Add API Support for Re-adding Indices to Data Streams #10726

However, I do think it may be difficult for the snapshot restore path to take care of adding a backing index back to a data stream automatically since if we take a snapshot of individual indices it's difficult to automatically determine if they belonged to a datastream originally.

[lyradc]

@jed326 I understand, thank you for those details.

Given that, I had restored a backing index to a similar-named non-datastream index. This could still work for my use-case, however the restored index shows a doc count, but index size remains 0. Is this expected?

[jed326]

@lyradc That doesn't sound right. I believe it's normal for index size to be 0 for a little bit after the snapshot restore starts but once restore is completed it should not show 0, especially if there is a positive doc count. Are you still able to search against this index? Could you also share the full _cat/indices output for this index (or however you are getting the index size)?

[lyradc]

Well thank you again, I just checked again and found the index and it's shards do now show doc counts as well as an expected size.

The restored index is also searchable. I may not have given the cluster enough time following restore...

With the above referenced feature already requested, this issue can be closed.