Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JP-3364 allow wcs shifts for IFU individual exposures in cube_build #8720

Merged
merged 38 commits into from
Sep 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
5c5bf7a
first attempt
jemorrison Aug 19, 2024
cbc2a50
First attempt at adding offsets
jemorrison Aug 21, 2024
7d17e04
updates
jemorrison Aug 27, 2024
aa5d546
updates
jemorrison Aug 27, 2024
c121c1f
update for cos(dec)
jemorrison Aug 29, 2024
44f416a
update
jemorrison Aug 29, 2024
e1fcac6
fix docs
jemorrison Aug 29, 2024
984c0c7
update docs
jemorrison Aug 29, 2024
afec305
added schema to cube_build
jemorrison Sep 5, 2024
8e464b2
trying to include offset yaml file
jemorrison Sep 5, 2024
bb351f8
corrected using a local schema file
jemorrison Sep 8, 2024
e20bd6a
fix conflict
jemorrison Sep 9, 2024
798a5f7
added more checks for offset file
jemorrison Sep 9, 2024
a503c29
fix typo
jemorrison Sep 9, 2024
0e7cc1b
attempt on unit test
jemorrison Sep 9, 2024
9446cad
added a unit test
jemorrison Sep 12, 2024
0fd9af7
fix api for nirspec
jemorrison Sep 12, 2024
407c216
added closing asdf file
jemorrison Sep 12, 2024
a27cdaa
Update Change Log
jemorrison Sep 12, 2024
c3dbb3d
updates to docs
jemorrison Sep 13, 2024
d918322
update docs
jemorrison Sep 13, 2024
7cba5e4
fixed nirspec offset issue
jemorrison Sep 13, 2024
add9f8b
CHANGES.rst
jemorrison Sep 13, 2024
30370f6
update docs
jemorrison Sep 14, 2024
c356158
update docs
jemorrison Sep 14, 2024
cc66ae9
changed how ra offset values are determined
jemorrison Sep 16, 2024
9c0aaca
fix doc error
jemorrison Sep 17, 2024
d5c6c7d
update comments and doc
jemorrison Sep 17, 2024
78f702c
update docs and update log message
jemorrison Sep 17, 2024
32b4044
docs update
jemorrison Sep 17, 2024
4e665cb
removed openning and closing model
jemorrison Sep 18, 2024
299b68c
various updates from review
jemorrison Sep 18, 2024
3c51fca
update comments
jemorrison Sep 19, 2024
45d15f1
fix comment
jemorrison Sep 19, 2024
04cbf68
cube_build_step.py
jemorrison Sep 19, 2024
8e8540e
fix test test_offset.py ruff issue
jemorrison Sep 19, 2024
dfa83a7
fix test
jemorrison Sep 19, 2024
7683808
Merge branch 'master' into cube_build_offsets
melanieclarke Sep 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,9 @@ cube_build

- Replaced deep copies of NIRSpec WCS objects within most loops. [#8793]

- Allow the user to provide ra and dec shifts to apply for each file to fine
tune the WCS. [#8720]

datamodels
----------

Expand Down
45 changes: 45 additions & 0 deletions docs/jwst/cube_build/arguments.rst
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,12 @@ The following arguments control the size and sampling characteristics of the out
``nspax_y``
The odd integer number of spaxels to use in the y dimension of the tangent plane.

``offset_file [string]``
The string contains the name of the file holding RA and Dec offsets to apply to each input file. This file
must be an asdf file with a specific format. For details on how to construct the offset file see
:ref:`offsets`.


``coord_system [string]``
The default IFU cubes are built on the ra-dec coordinate system (``coord_system=skyalign``). In these cubes north is up
and east is left. There are two other coordinate systems an IFU cube can be built on:
Expand Down Expand Up @@ -122,3 +128,42 @@ A parameter only used for investigating which detector pixels contributed to a c

The string is the x,y,z value of the cube spaxel that is being investigated. The numbering starts counting at 0.
To print information to the screeen about the x = 10, y = 20, z = 35 spaxel the parameter string value is '10 20 35'.

.. _offsets:

Creating an offset file
-----------------------

The offset file is an ASDF formatted file :`<https://asdf-standard.readthedocs.io/>`_ stands for "Advanced Scientific Data.
For each input file in the spec3 association used to build the IFU cubes, the offset file needs to have a
corresponding right ascension and declination offset given in arc seconds.

Below is an example of how to make an ASDF offset file. It is assumed the user has determined the
offsets to apply to the data in each file. The offsets information is stored in three lists:
`filename`, `raoffset` and `decoffset`. The units of the Ra and Dec offsets
are required to be in the offset file and only the unit, `arcsec`, is allowed. The file names should
not contain the directory path. The offset file can have any name, but it must have the `asdf` extension.

An example of making an offset file for an association containing three files is:

.. code-block:: python

import asdf
import astropy.units as u

filename = ['file1.fits', 'file2.fits', 'file3.fits']
raoffset = [0.0, -1.0, 1.0]
decoffset = [0.0, 1.0, -1.0]

tree = {
"units": str(u.arcsec),
"filename": filename,
"raoffset": raoffset,
"decoffset": decoffset
}
af = asdf.AsdfFile(tree)
af.write_to('offsets.asdf')
af.close()



82 changes: 78 additions & 4 deletions jwst/cube_build/cube_build_step.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,16 @@
"""

import time

from jwst.datamodels import ModelContainer
from jwst.lib.pipe_utils import match_nans_and_flags

from . import cube_build
from . import ifu_cube
from . import data_types
import asdf
from ..assign_wcs.util import update_s_region_keyword
from ..stpipe import Step, record_step_status

from pathlib import Path
from astropy import units
__all__ = ["CubeBuildStep"]


Expand Down Expand Up @@ -64,6 +64,7 @@
search_output_file = boolean(default=false)
output_use_model = boolean(default=true) # Use filenames in the output models
suffix = string(default='s3d')
offset_file = string(default=None) # Filename containing a list of Ra and Dec offsets to apply to files.
debug_spaxel = string(default='-1 -1 -1') # Default not used
"""

Expand Down Expand Up @@ -235,7 +236,18 @@
self.output_type = 'channel'
self.pars_input['output_type'] = self.output_type
self.log.info(f'Setting output type to: {self.output_type}')

# ________________________________________________________________________________
# If an offset file is provided do some basic checks on the file and its contents.
# The offset list contains a matching list to the files in the association
# used in calspec3 (for offline cube building).
# The offset list is an asdf file.
self.offsets = None

if self.offset_file is not None:
offsets = self.check_offset_file()
if offsets is not None:
self.offsets = offsets

Check warning on line 249 in jwst/cube_build/cube_build_step.py

View check run for this annotation

Codecov / codecov/patch

jwst/cube_build/cube_build_step.py#L247-L249

Added lines #L247 - L249 were not covered by tests
# ________________________________________________________________________________
# Read in Cube Parameter Reference file
# identify what reference file has been associated with these input

Expand Down Expand Up @@ -276,6 +288,7 @@
'roiw': self.roiw,
'wavemin': self.wavemin,
'wavemax': self.wavemax,
'offsets': self.offsets,
'skip_dqflagging': self.skip_dqflagging,
'suffix': self.suffix,
'debug_spaxel': self.debug_spaxel}
Expand Down Expand Up @@ -530,3 +543,64 @@
# remove duplicates if needed
self.pars_input['grating'] = list(set(self.pars_input['grating']))
# ________________________________________________________________________________

def check_offset_file(self):
"""Read in an optional ra and dec offset for each file.

Summary
----------
Check that is file is asdf file.
Check the file has the correct format using an local schema file.
The schema file, ifuoffset.schema.yaml, is located in the jwst/cube_build directory.
For each file in the input assocation check that there is a corresponding
file in the offset file.

"""

# validate the offset file using the schema file
DATA_PATH = Path(__file__).parent

try:
af = asdf.open(self.offset_file, custom_schema=DATA_PATH/'ifuoffset.schema.yaml')
except:
schema_message = ('Validation Error for offset file. Fix the offset file. \n' + \
'The offset file needs to have the same number of elements ' + \
'in the three lists: filename, raoffset and decoffset.\n' +\
'The units need to provided and only arcsec is allowed.')

raise Exception(schema_message)

offset_filename = af['filename']
offset_ra = af['raoffset']
offset_dec = af['decoffset']
# Note:
# af['units'] is checked by the schema validation. It must be arcsec or a validation error occurs.

# check that all the file names in input_model are in the offset filename
for model in self.input_models:
file_check = model.meta.filename
if file_check in offset_filename:
continue
melanieclarke marked this conversation as resolved.
Show resolved Hide resolved
else:
af.close()
raise ValueError('Error in offset file. A file in the assocation is not found in offset list %s', file_check)

# check that all the lists have the same length
len_file = len(offset_filename)
len_ra = len(offset_ra)
len_dec = len(offset_dec)
if (len_file != len_ra or len_ra != len_dec or len_file != len_dec):
af.close()
raise ValueError('The offset file does not have the same number of values for filename, raoffset, decoffset')

Check warning on line 594 in jwst/cube_build/cube_build_step.py

View check run for this annotation

Codecov / codecov/patch

jwst/cube_build/cube_build_step.py#L593-L594

Added lines #L593 - L594 were not covered by tests

offset_ra = offset_ra* units.arcsec
offset_dec = offset_dec* units.arcsec

# The offset file has passed tests so set the offset dictionary
offsets = {}
offsets['filename'] = offset_filename
offsets['raoffset'] = offset_ra
offsets['decoffset'] = offset_dec

af.close()
return offsets
Loading
Loading