Skip to content

Latest commit

 

History

History
15 lines (11 loc) · 1.09 KB

README.md

File metadata and controls

15 lines (11 loc) · 1.09 KB

solr2globus

This utility script is used to query a Solr-based ESGF index, harvest the metadata information, and then submit it to a Globus-based index. This utility does not need to be installed. You will just need to install the dependencies.

pip install -r requirements.txt

Notes

  • You must have owner, admin, or writer access to the globus_index_id to which you wish to ingest.
  • The utility will use the backoff library to continue to make the required requests to both indices until they are successful, gradually increasing the time between requests.
  • As we anticipate running this utility relatively few times, you will need to authenticate for each execution. If annoying, we could add refresh token support.
  • You may ingest by searches using esg-search syntax. A sample query is provided which took ~30 seconds to submit the ingest request.
  • This utility does not monitor when the Globus ingest tasks have completed--we just ensure that they were accepted. There can be a time lag between when you make the request and when your entries will appear in the new index.