Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error in Operations processing for Coron3 dataset #8899

Closed
stscijgbot-jp opened this issue Oct 17, 2024 · 3 comments · Fixed by #8907
Closed

Error in Operations processing for Coron3 dataset #8899

stscijgbot-jp opened this issue Oct 17, 2024 · 3 comments · Fixed by #8907
Labels
Software Affected: calwebb_coron3 Software Affected: klip team-chartreuse

Comments

@stscijgbot-jp
Copy link
Collaborator

Issue JP-3782 was created on JIRA by Tyler Pauly:

During processing of program 4534, Operations encountered an error:

2024-10-14 13:12:05,563 - stpipe.Coron3Pipeline.align_refs - INFO - Using PSFMASK reference file /ifs/archive/ops/jwst/ref/tmp_crds/crds/cache/references/jwst/nircam/jwst_nircam_psfmask_0246.fits
2024-10-14 13:12:05,614 - stpipe.Coron3Pipeline.align_refs - WARNING - /dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-1.15.1.20240708-py3.12/lib/python3.12/site-packages/numpy/core/fromnumeric.py:3504: RuntimeWarning: Mean of empty slice.
2024-10-14 13:12:05,614 - stpipe.Coron3Pipeline.align_refs - WARNING - return _methods._mean(a, axis=axis, dtype=dtype,
2024-10-14 13:12:05,614 - stpipe.Coron3Pipeline.align_refs - WARNING -
2024-10-14 13:12:05,614 - stpipe.Coron3Pipeline.align_refs - WARNING - /dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-1.15.1.20240708-py3.12/lib/python3.12/site-packages/numpy/core/_methods.py:129: RuntimeWarning: invalid value encountered in divide
2024-10-14 13:12:05,614 - stpipe.Coron3Pipeline.align_refs - WARNING - ret = ret.dtype.type(ret / rcount)
2024-10-14 13:12:05,614 - stpipe.Coron3Pipeline.align_refs - WARNING -
2024-10-15 21:55:35,338 - CRDS - DEBUG - Using reference file selection rules 'jwst_1293.pmap' defined by environment CRDS_CONTEXT.
2024-10-15 21:55:35,341 - stpipe.Coron3Pipeline.align_refs - INFO - Step align_refs done
2024-10-15 22:30:38,110 - stpipe.Coron3Pipeline - INFO - Saved model in jw04534001001_04106_00001_nrca2_c1002_psfalign.fits
2024-10-15 22:30:39,078 - stpipe.Coron3Pipeline.klip - INFO - Step klip running with args (<CubeModel(650, 320, 320) from jw04534001001_04106_00001_nrca2_c1002_crfints.fits>, <QuadModel(650, 700, 320, 320) from jw04534001001_04106_00001_nrca2_c1002_psfalign.fits>).
2024-10-15 22:30:39,087 - stpipe.Coron3Pipeline.klip - INFO - KL transform truncation = 50
2024-10-16 01:23:22,599 - stpipe.Coron3Pipeline.klip - INFO - Step klip done
2024-10-16 01:23:31,334 - stpipe.Coron3Pipeline - INFO - Saved model in jw04534001001_04106_00001_nrca2_c1002_psfsub.fits
Traceback (most recent call last):
File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-1.15.1.20240708-py3.12/lib/python3.12/site-packages/stpipe/cli/strun.py", line 24, in main
Step.from_cmdline(sys.argv[1:])
File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-1.15.1.20240708-py3.12/lib/python3.12/site-packages/stpipe/step.py", line 186, in from_cmdline
return cmdline.step_from_cmdline(args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-1.15.1.20240708-py3.12/lib/python3.12/site-packages/stpipe/cmdline.py", line 395, in step_from_cmdline
step.run(*positional)
File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-1.15.1.20240708-py3.12/lib/python3.12/site-packages/stpipe/step.py", line 504, in run
step_result = self.process(*args)
^^^^^^^^^^^^^^^^^^^
File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-1.15.1.20240708-py3.12/lib/python3.12/site-packages/jwst/pipeline/calwebb_coron3.py", line 191, in process
for model in to_container(psf_sub):
^^^^^^^^^^^^^^^^^^^^^
File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-1.15.1.20240708-py3.12/lib/python3.12/site-packages/jwst/pipeline/calwebb_coron3.py", line 33, in to_container
setattr(image, attribute, model.getarray_noinit(attribute)[plane])
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
IndexError: index 320 is out of bounds for axis 0 with size 320

I reproduced this on jwst/main, indicating the issue is present after the recent changes to align_refs:

2024-10-17 09:21:05,334 - stpipe.Coron3Pipeline.align_refs - INFO - Step align_refs done
2024-10-17 09:21:06,008 - stpipe.Coron3Pipeline - INFO - Saved model in jw04534001001_04106_00001_nrcalong_c1002_psfalign.fits
2024-10-17 09:21:06,074 - stpipe.Coron3Pipeline.klip - INFO - Step klip running with args (<CubeModel(650, 320, 320) from jw04534001001_04106_00001_nrcalong_c1002_crfints.fits>, <CubeModel(700, 320, 320) from jw04534001001_04106_00001_nrcalong_c1002_psfalign.fits>).
2024-10-17 09:21:06,081 - stpipe.Coron3Pipeline.klip - INFO - KL transform truncation = 50
2024-10-17 09:58:01,993 - stpipe.Coron3Pipeline.klip - INFO - Step klip done
2024-10-17 09:58:03,947 - stpipe.Coron3Pipeline - INFO - Saved model in jw04534001001_04106_00001_nrcalong_c1002_psfsub.fits
----------------------------------------------------------------------
ERROR RUNNING STEP 'Coron3Pipeline':
    index 320 is out of bounds for axis 0 with size 320
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/tpauly/miniconda3/envs/jwstdev/lib/python3.12/site-packages/stpipe/cli/strun.py", line 24, in main
    Step.from_cmdline(sys.argv[1:])
  File "/Users/tpauly/miniconda3/envs/jwstdev/lib/python3.12/site-packages/stpipe/step.py", line 191, in from_cmdline
    return cmdline.step_from_cmdline(args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tpauly/miniconda3/envs/jwstdev/lib/python3.12/site-packages/stpipe/cmdline.py", line 395, in step_from_cmdline
    step.run(*positional)
  File "/Users/tpauly/PycharmProjects/jwst/jwst/stpipe/core.py", line 101, in run
    result = super().run(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tpauly/miniconda3/envs/jwstdev/lib/python3.12/site-packages/stpipe/step.py", line 524, in run
    step_result = self.process(*args)
                  ^^^^^^^^^^^^^^^^^^^
  File "/Users/tpauly/PycharmProjects/jwst/jwst/pipeline/calwebb_coron3.py", line 197, in process
    for model in to_container(psf_sub):
                 ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tpauly/PycharmProjects/jwst/jwst/pipeline/calwebb_coron3.py", line 33, in to_container
    setattr(image, attribute, model.getarray_noinit(attribute)[plane])
                              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
IndexError: index 320 is out of bounds for axis 0 with size 320

Investigate the save issue in coron3.
@stscijgbot-jp
Copy link
Collaborator Author

Comment by Ned Molter on JIRA:

The problem is that, when converting the CubeModel to a list of ImageModels for processing through resample, the area extension has shape (x,y) whereas the rest of the extensions have shape (nints, x, y), but they are handled in an identical way. This only leads to an IndexError when nints > x, but when nints <= x it renders the area extension a bit useless, as it only has shape (y,).  Proposed fix is to check the dimensionality of the extensions being converted, and if they're 2-D, avoid indexing them.

@emolter emolter mentioned this issue Oct 22, 2024
Merged
10 tasks
@stscijgbot-jp
Copy link
Collaborator Author

Comment by Ned Molter on JIRA:

Fixed by #8907 

@stscijgbot-jp
Copy link
Collaborator Author

Comment by David Law on JIRA:

Closing based on testing by Ben Sunnquist directly on the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Software Affected: calwebb_coron3 Software Affected: klip team-chartreuse
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

JP-3782: Fix improper setting of AREA extension in coron3 pipeline emolter/jwst
1 participant
@stscijgbot-jp