[BUG] All system index interactions should happen in a stashed context

[goyamegh]

What is the bug?
As part of fixing the security tests, we observed some failing integ tests in the context of security enabled workflow. Verified this by opening new PRs after the latest docker image got published here: #1172.

After a small test in test PR: #1179, it was found that some of the system index interactions in the plugin are happening outside the stashed context, causing the tests to fail. Once this is fixed, we should be able re-run the security tests successfully.

Sample reference: opensearch-project/job-scheduler#347

How can one reproduce the bug?
Steps to reproduce the behavior:

1. Go to any PR in main or 2.x
2. Check the security tests are failing, while non-security enabled integ tests are passing.

What is the expected behavior?
Security enabled integ tests should pass

What is your host/environment?

• OS: [e.g. iOS] - linux, macOs, windows
• Version [e.g. 22] - main and 2.x
• Plugins - SAP

Do you have any screenshots?
NA

Do you have any additional context?
Add any other context about the problem.