implement unit conversion in specviz2d #3253

gibsongreen · 2024-10-24T14:26:52Z

Description

This pull request is to address implementing the Unit Conversion plugin in Specviz2d.

Change log entry

Is a change log needed? If yes, is it added to CHANGES.rst? If you want to avoid merge conflicts,
list the proposed change log here for review and add to CHANGES.rst before merge. If no, maintainer
should add a no-changelog-entry-needed label.

Checklist for package maintainer(s)

This checklist is meant to remind the package maintainer(s) who will review this pull request of some common things to look for. This list is not exhaustive.

Are two approvals required? Branch protection rule does not check for the second approval. If a second approval is not necessary, please apply the trivial label.
Do the proposed changes actually accomplish desired goals? Also manually run the affected example notebooks, if necessary.
Do the proposed changes follow the STScI Style Guides?
Are tests added/updated as required? If so, do they follow the STScI Style Guides?
Are docs added/updated as required? If so, do they follow the STScI Style Guides?
Did the CI pass? If not, are the failures related?
Is a milestone set? Set this to bugfix milestone if this is a bug fix and needs to be released ASAP; otherwise, set this to the next major release milestone. Bugfix milestone also needs an accompanying backport label.
After merge, any internal documentations need updating (e.g., JIRA, Innerspace)?

…nits

codecov · 2024-12-19T04:21:59Z

Codecov Report

Attention: Patch coverage is 95.83333% with 2 lines in your changes missing coverage. Please review.

Project coverage is 88.81%. Comparing base (cf9664f) to head (98f2852).
Report is 13 commits behind head on main.

Files with missing lines	Patch %	Lines
jdaviz/core/helpers.py	87.50%	1 Missing ⚠️
jdaviz/core/marks.py	92.30%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff            @@
##             main    #3253    +/-   ##
========================================
  Coverage   88.80%   88.81%            
========================================
  Files         125      125            
  Lines       19155    19268   +113     
========================================
+ Hits        17010    17112   +102     
- Misses       2145     2156    +11

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

gibsongreen · 2024-12-19T21:44:13Z

jdaviz/configs/specviz/plugins/unit_conversion/unit_conversion.py

+                                self.has_angle = False
+                                self.has_sb = False


We don't append an angle unit in Specviz right now, see the screenshot when the native unit is Jy. This hides both from displaying in the UI and unexposes them from the API.

gibsongreen · 2024-12-19T21:50:51Z

jdaviz/conftest.py

@@ -260,7 +260,7 @@ def _create_spectrum1d_cube_with_fluxunit(fluxunit=u.Jy, shape=(2, 2, 4), with_u
    wcs_dict = {"CTYPE1": "RA---TAN", "CTYPE2": "DEC--TAN", "CTYPE3": "WAVE-LOG",
                "CRVAL1": 205, "CRVAL2": 27, "CRVAL3": 4.622e-7,
                "CDELT1": -0.0001, "CDELT2": 0.0001, "CDELT3": 8e-11,
-                "CRPIX1": 0, "CRPIX2": 0, "CRPIX3": 0,
+                "CRPIX1": 0, "CRPIX2": 0, "CRPIX3": 0, "PIXAR_SR": 10,


With this update to the test fixture, in test_spectral_extraction_with_correct_sum_units, the scale factor is no longer the default value of 1, so we need to update the expected values in the test.

kecnry · 2024-12-20T19:00:45Z

CHANGES.rst

@@ -29,6 +29,8 @@ Specviz
 Specviz2d
 ^^^^^^^^^

+- Implement the Unit Conversion plugin the Specviz2D. [#3253]


Suggested change

- Implement the Unit Conversion plugin the Specviz2D. [#3253]

- Implement the Unit Conversion plugin in Specviz2D. [#3253]

kecnry · 2024-12-20T19:01:31Z

jdaviz/configs/cubeviz/plugins/spectral_extraction/tests/test_spectral_extraction.py

+    expected_flux_values = [190., 590., 990., 1390., 1790., 2190., 2590., 2990., 3390., 3790.]
+    # After collapsing, sr is removed via the scale factor and the extracted spectrum is in Jy
+    expected_flux_values = [flux * collapsed.meta.get('_pixel_scale_factor')
+                            for flux in expected_flux_values]


can this be done with array math instead of a list comprehension?

I have now made an update to make the expect_flux_values is an array instead of a list, this will remove the list comprehension all together

kecnry · 2024-12-20T19:02:27Z

jdaviz/configs/default/plugins/markers/markers.py

+            if 'value' in row_info:
+                # format and cast flux value for Table
+                row_info['value'] = f"{row_info['value']:.5e}"
+            else:
+                row_info['value'] = ''


why is this necessary? Can/should we move it to as_dict instead?

There was an issue arising where the value column in the table couldn't be ready properly. This was happening when casting value when it was numpy.float32 to float in coords_info.py. I added a check to only cast if the value is not already float.

kecnry · 2024-12-20T19:02:55Z

jdaviz/configs/default/plugins/model_fitting/model_fitting.py

@@ -553,6 +553,7 @@ def _initialize_model_component(self, model_comp, comp_label, poly_order=None):
            # equivs for spectral density and flux<>sb
            pixar_sr = masked_spectrum.meta.get('_pixel_scale_factor', 1.0)
            equivs = all_flux_unit_conversion_equivs(pixar_sr, init_x)
+            equivs += u.spectral_density(init_x)


should this be in all_flux_unit_conversion_equivs (or at least as an additional option)?

Yes, it should already be handled by all_flux_unit_conversion_equivs, I was debugging an issue and threw it in there, just retested and it is able to be removed with no consequence.

kecnry · 2024-12-20T19:03:31Z

jdaviz/configs/default/plugins/model_fitting/model_fitting.py

+        spec = self.dataset.selected_spectrum
+        # Needs Attention:
+        # should conversion occur before or after call to _apply_subset_masks?
+        if spec.flux.unit.to_string != self.app._get_display_unit('flux'):


Isn't this a method? Is it cheaper to compare units instead of casting to string?

Suggested change

if spec.flux.unit.to_string != self.app._get_display_unit('flux'):

if spec.flux.unit.to_string() != self.app._get_display_unit('flux'):

Good catch! Is it worth converting the display units to astropy.Unit or keep to string comparison?

kecnry · 2024-12-20T19:04:45Z

jdaviz/configs/default/plugins/model_fitting/model_fitting.py

+        # Needs Attention:
+        # should conversion occur before or after call to _apply_subset_masks?
+        if spec.flux.unit.to_string != self.app._get_display_unit('flux'):
+            equivalencies = u.spectral_density(self.dataset.selected_spectrum.spectral_axis)
+            spec = self.dataset.selected_spectrum.with_flux_unit(self.app._get_display_unit('flux'),
+                                                                 equivalencies=equivalencies)


should this all be moved into the DatasetSelect.selected_spectrum logic so its reusable and cached (and then the cache would need to be invalidated for a change in flux display units)?

kecnry · 2024-12-20T19:06:02Z

jdaviz/configs/imviz/plugins/coords_info/coords_info.py

+                    self._dict['spectral_axis'] = wave.value
+                    self._dict['spectral_axis:unit'] = wave.unit.to_string()


where were these handled before (I don't see them in the else 🤔)?

X-axis conversions were not previously handled in Specviz2d, so they were previously always using the native units and value when mousing over

kecnry · 2024-12-20T19:07:40Z

jdaviz/configs/mosviz/tests/test_data_loading.py

+    # Note: spectra2d Wave loaded in meters, but we respect one spectral unit, so the meters in
+    # converted to Angstrom (the spectra1d spectral unit).


Can you explain this a little more? Is it that a previous data entry was in angstroms or just that that is the default spectral display unit?

Suggested change

# Note: spectra2d Wave loaded in meters, but we respect one spectral unit, so the meters in

# converted to Angstrom (the spectra1d spectral unit).

# Note: spectra2d Wave loaded in meters, but we respect one spectral unit, so the meters is

# converted to Angstrom (the spectra1d spectral unit).

So the loaded 1D and 2D spectra have different native spectral units, with one in meters and one in angstroms and app-wide we're respecting one choice for spectral unit.

kecnry · 2024-12-20T19:08:06Z

jdaviz/configs/specviz/plugins/unit_conversion/unit_conversion.py

            # TODO [specviz2d, mosviz] x_display_unit is not implemented in glue for image viewer
            # used by spectrum-2d-viewer


is this still relevant?

For the MosvizImageView, I will make sure to update the comment to reflect this!

kecnry · 2024-12-20T19:09:03Z

jdaviz/configs/specviz/plugins/unit_conversion/unit_conversion.py

-                            # default to sr if input spectrum is not in surface brightness units
-                            # TODO: for cubeviz, should we check the cube itself?
-                            self.angle_unit.selected = 'sr'


why remove this? It doesn't seem to be replaced with any other logic to accomplish the same thing, except the else below which is defaulting to pix2 instead.

The logic just gets moved, not removed. So if the angle_unit is None and the config is not specviz nor specviz2d, then the angle_unit will still be set to sr. By updating the has_angle and has_sb if it is one of the 2 configs, it removes the sb_unit and angle_unit from the UI/API.

I can reintroduce the comments that previously exists about setting the angle_unit to sr.

cshanahan1

When testing this with the specviz2d example notebook, the option to change flux unit disappears when you load the data. The screen shots below show an empty specviz instance, with the 'Flux' option appearing under the unit conversion plugin, and then after loading data when that dropdown disappears

cshanahan1 · 2024-12-23T19:08:04Z

What is included in the scope here for surface brightness logic, including coercing flux units to surface brightness, handling square pixel, and toggling the spectral axis? (Are images of 2d spectra even typically or commonly in surface brightness units?)

gibsongreen · 2024-12-23T19:26:14Z

What is included in the scope here for surface brightness logic, including coercing flux units to surface brightness, handling square pixel, and toggling the spectral axis? (Are images of 2d spectra even typically or commonly in surface brightness units?)

There will be no coercing for pix^2 in Specviz2d, but I think it will be applicable for the image viewers in Mosviz and Imviz.
We only apply the scale factor in Cubeviz since it is stored in the metadata when we extract the cube using the plugin (so no translations here)
I did some research into 2d spectra before starting and to my knowledge, they should only be in Flux units. Specviz2d will still be able to load surface brightness units in steradian, if applicable.

github-actions bot added cubeviz specviz imviz plugin Label for plugins common to multiple configurations specviz2d labels Oct 24, 2024

gibsongreen modified the milestones: 4.0.1, 4.1 Oct 24, 2024

rosteen force-pushed the main branch from 40cb333 to a16a3a0 Compare October 29, 2024 19:12

gibsongreen mentioned this pull request Nov 4, 2024

fix flux unit conversions in plugins #3228

Merged

gibsongreen force-pushed the uc-enable-all branch from 50e8194 to b2d178d Compare November 29, 2024 20:27

gibsongreen added 19 commits December 11, 2024 17:34

first pass specviz2d implementation

086cfa4

resolve stash

a14a39f

fix issues with stash and update get_data

8032fdb

second pass specviz2d uc implementation

35da355

revert 2D viewer for uc

b8eb091

don't allow conversion of spectral line y values

34832f0

fix bug to ensure non-scale factor data converts on _handle_display_u…

c9ef4e2

…nits

fix get_data bug using native unit for non-scale factor data

8ad5973

add line analysis uc test

ef83da1

add test for solid angle equivalency list

6010858

add MosvizProfileView test

a06159a

add change log

e8d22ac

reconcile test failures

eb45c00

add API test

37e7b6c

remove global display unit change handler, syntax error in helpers

19a2868

ensure continuum marks don't double translate, test coverage

0a82322

move line analysis test

2f1db3e

remove files that weren't removed during rebase

ea8cb25

fix styling

ff6ac14

add missing import

f2920e2

gibsongreen force-pushed the uc-enable-all branch from 22ca012 to f2920e2 Compare December 11, 2024 22:42

gibsongreen added 4 commits December 17, 2024 14:27

add spectral unit support, add eqvs to model fit, coords, marks

b876cb4

fix syntax error

f946861

2dviewer spectral conversion, first iteration of test coverage

7c54327

resolve test failures

0ffdb5d

address spec density edge case, test coverage

98f2852

gibsongreen marked this pull request as ready for review December 19, 2024 15:37

gibsongreen requested review from rosteen, javerbukh, pllim, kecnry, bmorris3, haticekaratay and cshanahan1 as code owners December 19, 2024 15:37

gibsongreen commented Dec 19, 2024

View reviewed changes

kecnry reviewed Dec 20, 2024

View reviewed changes

rosteen modified the milestones: 4.1, 4.2 Dec 23, 2024

cshanahan1 reviewed Dec 23, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

implement unit conversion in specviz2d #3253

implement unit conversion in specviz2d #3253

gibsongreen commented Oct 24, 2024 •

edited

Loading

codecov bot commented Dec 19, 2024 •

edited

Loading

gibsongreen Dec 19, 2024

gibsongreen Dec 19, 2024

kecnry Dec 20, 2024

kecnry Dec 20, 2024

gibsongreen Dec 23, 2024

kecnry Dec 20, 2024

gibsongreen Dec 23, 2024

kecnry Dec 20, 2024

gibsongreen Dec 23, 2024

kecnry Dec 20, 2024

gibsongreen Dec 23, 2024

kecnry Dec 20, 2024

kecnry Dec 20, 2024

gibsongreen Dec 23, 2024

kecnry Dec 20, 2024

gibsongreen Dec 23, 2024

kecnry Dec 20, 2024

gibsongreen Dec 23, 2024

kecnry Dec 20, 2024

gibsongreen Dec 23, 2024

cshanahan1 left a comment •

edited

Loading

cshanahan1 commented Dec 23, 2024

gibsongreen commented Dec 23, 2024 •

edited

Loading

	- Implement the Unit Conversion plugin the Specviz2D. [#3253]
	- Implement the Unit Conversion plugin in Specviz2D. [#3253]

	if spec.flux.unit.to_string != self.app._get_display_unit('flux'):
	if spec.flux.unit.to_string() != self.app._get_display_unit('flux'):

		self._dict['spectral_axis'] = wave.value
		self._dict['spectral_axis:unit'] = wave.unit.to_string()

		# Note: spectra2d Wave loaded in meters, but we respect one spectral unit, so the meters in
		# converted to Angstrom (the spectra1d spectral unit).

		# TODO [specviz2d, mosviz] x_display_unit is not implemented in glue for image viewer
		# used by spectrum-2d-viewer

implement unit conversion in specviz2d #3253

Are you sure you want to change the base?

implement unit conversion in specviz2d #3253

Conversation

gibsongreen commented Oct 24, 2024 • edited Loading

Description

Change log entry

Checklist for package maintainer(s)

codecov bot commented Dec 19, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cshanahan1 left a comment • edited Loading

Choose a reason for hiding this comment

cshanahan1 commented Dec 23, 2024

gibsongreen commented Dec 23, 2024 • edited Loading

gibsongreen commented Oct 24, 2024 •

edited

Loading

codecov bot commented Dec 19, 2024 •

edited

Loading

cshanahan1 left a comment •

edited

Loading

gibsongreen commented Dec 23, 2024 •

edited

Loading