Removed redundant calculations, unneeded array allocations from jump step

[t-brandt]

Resolves JP-3697

Closes #

This PR affects the two-point-difference jump step only. It removes duplicate and redundant calculations and unneeded array allocations, which make a significant difference for large uncal files. Masked array median is also changed to nanmedian, and a few variables are renamed to avoid overwriting a quantity that is needed at the end of the routine.

Tasks

• update or add relevant tests
• update relevant docstrings and / or docs/ page
• Does this PR change any API used downstream? (if not, label with no-changelog-entry-needed)
  • write news fragment(s) in changes/: echo "changed something" > changes/<PR#>.<changetype>.rst (see below for change types)
  • run regression tests with this branch installed ("git+https://github.com/<fork>/stcal@<branch>")
    • jwst regression test
    • romancal regression test

news fragment change types...

• changes/<PR#>.apichange.rst: change to public API
• changes/<PR#>.bugfix.rst: fixes an issue
• changes/<PR#>.general.rst: infrastructure or miscellaneous change

[codecov]

Codecov Report

Attention: Patch coverage is 85.71429% with 10 lines in your changes missing coverage. Please review.

Project coverage is 86.89%. Comparing base (35726e9) to head (412f078).
Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
src/stcal/jump/twopoint_difference.py	85.71%	10 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##             main     #302       +/-   ##
===========================================
+ Coverage   29.57%   86.89%   +57.31%     
===========================================
  Files          36       49       +13     
  Lines        7949     8949     +1000     
===========================================
+ Hits         2351     7776     +5425     
+ Misses       5598     1173     -4425     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[kmacdonald-stsci]

There are some down stream test failures. Are those related to this PR? If not, this looks fine.

[ddavis-stsci]

The romancal tests for these changes pass,
https://github.com/spacetelescope/RegressionTests/actions/runs/11328631054

[melanieclarke]

JWST regtests are running here:
https://github.com/spacetelescope/RegressionTests/actions/runs/11672280970

Failures are unrelated (from spacetelescope/jwst#8893).

[melanieclarke]

These changes look pretty straightforward, and seem worth merging as is, before we do any more profiling or refactoring for jump.

One quick question about the change to the flag handling, below. Also, the change note needs to be moved out of the CHANGES.rst file to a newer style towncrier note -- see the PR template description for the new method. That should fix the readthedocs error and the changelog check.

[melanieclarke]

@t-brandt - just checking in. I think this PR just needs some minor updates and it should be ready to go. Do you want to make the changes, or would you like me to do it for you?

[t-brandt]

@melanieclarke Sorry for dropping the ball on this for a while. If you have the time and bandwidth to make the minor changes, please go ahead. Otherwise I'll try to do so over the weekend or next week.

[melanieclarke]

Rerunning jwst regtests here:
https://github.com/spacetelescope/RegressionTests/actions/runs/12323165118

and romancal here:
https://github.com/spacetelescope/RegressionTests/actions/runs/12323205161

[melanieclarke]

Romancal tests look fine. There are some failures in jwst unit and regression tests that are related to some ongoing gwcs work. I will let that settle and rerun before we merge this.

[t-brandt]

I actually want to make a few more changes--I can squeeze a significant amount of additional speedup while shortening the code. I will push those changes tonight or tomorrow. We can work on merging this after 11.2 :-)

[melanieclarke]

We can work on merging this after 11.2 :-)

Sounds good, thanks. Can you please mark this draft for now, and let us know when you're ready?

[t-brandt]

@melanieclarke: I updated the PR with new changes. The new changes are more significant so I think a bit more testing/more eyeballs are warranted, which presumably will wait until after 11.2 is done. I checked that the generated gdq arrays were identical for a few test files. Memory improvements are modest, speed improvements are large, and crashes/virtual memory/very slow performance when overflagging jumps should all be gone.