IM-Suite-2102: Threshold bug fix for vicinity processing of masked fields

[bayliffe]

Bug fix to threshold plugin.
When applying fuzzy thresholds we use a rescaling that uses numpy.where. This does not exclude masked points. The fill values beneath the mask are typically 1E36. These points always exceed any threshold and are rescaled to probabilities of 1.0 for all thresholds.

If a masked field is processed with fuzzy thresholds, and uses a vicinity radius, then these probabilities of 1 given by the masked area are pulled into the unmasked area around the edge of the masking up to the vicinity distance. This results in a permanent ring of probabilities of 1 around the edge of a masked region at all thresholds.

This bug originates in the redesign of the threshold method last year in which the re-application of the mask to the truth values prior to the vicinity application but after rescaling was lost.

I've added a new acceptance test to cover this: test data: metoppv/improver_test_data#68

Testing:

• Ran tests and they passed OK
• Added new tests for the new feature(s)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.40%. Comparing base (84a8944) to head (2d0dac0).
Report is 54 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff            @@
##           master    #2071    +/-   ##
========================================
  Coverage   98.39%   98.40%            
========================================
  Files         124      134    +10     
  Lines       12212    13146   +934     
========================================
+ Hits        12016    12936   +920     
- Misses        196      210    +14     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[gavinevans]

Thanks @bayliffe 👍

I've added one very minor comment, but otherwise this looks fine.

[gavinevans]

You could specify this above the args call, particularly given that kgo_dir already exists as a variable e.g.
config_path = kgo_dir / "precip_accumulation_thresholds.json".

[Kat-90]

Separate to Gavin's comment. Reviewed the code and ran the tests alongside the test_data changes. All looks good to me.