Internal-network use of ██████████ no longer works

[stscijgbot-jp]

Issue JP-3838 was created on JIRA by David Law:

With thanks to Rachel Plesha  for the original report, it looks like internal usage of CRDS from ██████████ no longer seems to be working.

Typically, most users need to set both CRDS_PATH and CRDS_SERVER_URL, so that necessary files will be pulled from the server and cached locally at CRDS_PATH.  However, I believe there was also a method where users on the STScI internal network didn't need to do either of these, as the pipeline would default to using information on a shared disk at ██████████ (thereby saving internal users the need to download their own copies of all reference files).

It's been a long time since I last tried using this approach myself, but indeed it looks like it's now not working.  Traceback in comments below, although it should be easy to reproduce for anyone by simply deleting the env variables.

Any thoughts Jonathan Eisenhamer?  Also tagging Rosa Diaz Meaghan McDonald Hunter Brown 

[stscijgbot-jp]

Comment by David Law on JIRA:

Traceback (most recent call last):
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/stpipe/cli/strun.py", line 24, in main
    cmdline.step_from_cmdline(sys.argv[1:])
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/stpipe/cmdline.py", line 395, in step_from_cmdline
    step.run(*positional)
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/jwst/stpipe/core.py", line 101, in run
    result = super().run(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/stpipe/step.py", line 505, in run
    self.prefetch(*args)
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/stpipe/step.py", line 630, in prefetch
    self._precache_references(args[0])
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/stpipe/pipeline.py", line 266, in _precache_references
    self._precache_references_opened(model)
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/stpipe/pipeline.py", line 292, in _precache_references_opened
    self._precache_references_impl(model_or_container)
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/stpipe/pipeline.py", line 319, in _precache_references_impl
    crds_refs = crds_client.get_multiple_reference_paths(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/stpipe/crds_client.py", line 53, in get_multiple_reference_paths
    return _get_refpaths(parameters, tuple(reference_file_types), observatory)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/stpipe/crds_client.py", line 65, in _get_refpaths
    bestrefs = crds.getreferences(
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/crds/core/heavy_client.py", line 122, in getreferences
    final_context, bestrefs = _initial_recommendations("getreferences",
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/crds/core/heavy_client.py", line 209, in _initial_recommendations
    mode, final_context = get_processing_mode(observatory, context)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/crds/core/utils.py", line 306, in call
    key, result = self._readonly(*args, **keys)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/crds/core/utils.py", line 293, in _readonly
    return key, self.uncached(*args, **keys)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/crds/core/heavy_client.py", line 398, in get_processing_mode
    final_context = get_final_context(info, context)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/crds/core/heavy_client.py", line 450, in get_final_context
    input_context = api.get_build_context('jwst')
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/crds/client/api.py", line 415, in get_build_context
    return get_default_context(observatory=observatory, state="latest")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/crds/core/utils.py", line 306, in call
    key, result = self._readonly(*args, **keys)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/crds/core/utils.py", line 293, in _readonly
    return key, self.uncached(*args, **keys)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/crds/client/api.py", line 384, in get_default_context
    return str(S.get_default_context(observatory))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/crds/client/proxy.py", line 157, in call
    jsonrpc = self._call(*args, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dlaw/anaconda3/envs/jwst1.17.0/lib/python3.11/site-packages/crds/client/proxy.py", line 121, in _call
    raise exceptions.ServiceError("Configured for server-less mode.  Skipping JSON RPC " + repr(self.__service_name))
crds.core.exceptions.ServiceError: Configured for server-less mode.  Skipping JSON RPC 'get_default_context'

[stscijgbot-jp]

Comment by Jonathan Eisenhamer on JIRA:

This would be a fallout from the introduction of the version-defined contexts.

At the moment, the workaround is to either define just CRDS_CONTEXT, or define the CRDS_PATH/CRDS_SERVER_URL pair.

Will create a CCD issue to explore methods of supported an offline version of the version-defined context system. In the meantime, the documentation will need to be updated. Will have a CCD issue to ensure the CRDS documentation itself is updated; I believe most of it already has been. However, I believe the other documentation, such as JDox and JWST package, will also need updating.

[stscijgbot-jp]

Comment by David Law on JIRA:

Got it, thanks Jonathan Eisenhamer 

[stscijgbot-jp]

Comment by David Law on JIRA:

Based on feedback from Rachel Plesha it sounds like usage of CRDS from central server works fine so long as the CRDS_SERVER_URL is defined.  Indeed, I've confirmed that (while on the internal network) if CRDS_SERVER_URL is defined properly but CRDS_PATH is undefined the pipeline successfully determines the correct context to use for a given pipeline version and then uses those files successfully from central store without needing to cache them.

As such, there's no work required for this ticket to have things operating as desired.  All we need to do is to update the documentation about internal usage.

As far as I'm aware the only thing that thus needs to be changed is jwst/README.md; see #9078 for a PR implementing what I think the necessary changes are.

[stscijgbot-jp]

Comment by Rachel Plesha on JIRA:

This page should also be updated:

https://jwst-pipeline.readthedocs.io/en/latest/jwst/user_documentation/reference_files_crds.html#crds-servers

[stscijgbot-jp]

Comment by Melanie Clarke on JIRA:

Fixed by #9078