[rgw]fix TFA issue by adding sleep of 20 seconds after rgw restart to avoid sync status failures

[hmaheswa]

TFA failures:
http://magna002.ceph.redhat.com/cephci-jenkins/results/openstack/RH/8.0/rhel-9/Weekly/19.2.0-35/rgw/10/tier-2_ssl_rgw_ms_ecpool_test/
http://magna002.ceph.redhat.com/cephci-jenkins/results/openstack/RH/8.0/rhel-9/Weekly/19.2.0-35/rgw/10/tier-2_rgw_ms-archive/
http://magna002.ceph.redhat.com/cephci-jenkins/results/openstack/RH/8.0/rhel-9/Weekly/19.2.0-35/rgw/10/tier-2_rgw_ms-archive_resharding_granular_sync/
http://magna002.ceph.redhat.com/cephci-jenkins/results/openstack/RH/8.0/rhel-9/Weekly/19.2.0-35/rgw/10/tier-2_rgw_ms_async_data_notification/

fail log before fix from my local:
http://magna002.ceph.redhat.com/cephci-jenkins/hsm/TFA_create_user_sync_issue/cephci-run-9JPIFU/

pass log after sleep of 20 seconds:
http://magna002.ceph.redhat.com/cephci-jenkins/hsm/TFA_create_user_sync_issue/cephci-run-96EMAC/

Description

Please include Automation development guidelines. Source of Test case - New Feature/Regression Test/Close loop of customer BZs

click to expand checklist

• Create a test case in Polarion reviewed and approved.
• Create a design/automation approach doc. Optional for tests with similar tests already automated.
• Review the automation design
• Implement the test script and perform test runs
• Submit PR for code review and approve
• Update Polarion Test with Automation script details and update automation fields
• If automation is part of Close loop, update BZ flag qe-test_coverage “+” and link Polarion test

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: hmaheswa

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ckulal]

waiting on end to end run log to merge

[psathyan]

Blind sleep may work today but it could fail later or in the next build.

IMHO, we could check if the rgw service is up and running using

ceph orch ls --service_type rgw --service-name <rgw_service_name> --format json |
jq '.[0].status | select(.size !=0) | .size == .running`

[psathyan]

I might be wrong with the need for checking rgw based on the heading... however the code is restarting osd_process_name

It is also possible to check for that particular daemon running status.

[github-actions]

This Pull request has been automatically marked as STALE due to inactivity for 15 days and will be CLOSED on further inactivity on the PR for another 15 days.