Overhaul to plot_proftrace: options for multiple figure configuration…

…s in YAML, plotting per-ob DA penalty statistics (#131)

* Bug fix for _get_conv() to pull appropriate ptop, pbot based on variable

* plot_proftrace.py for plotting ob type/subtype, add .png to .gitignore

* Switched from multi-index slicing to .isin(), inputs can now be provided as a list and aggregated together

* Fixed bug with dataframe filters for ==None

* Added some pycodestyle compliance to scripts/plot_proftrace, finalized bug fix for ==None filtering options

* generalized input filters with some in-line documentation

* modified plot_proftrace to use yaml input; pycodestyle and code readability changes

* Rearranged into more discrete functions, greater pythonic compliance

* Added PyGSI/gsi_stat and scripts/plot_proftrace for plotting penalty data

* Fixed collect_statistics() to properly handle case of missing gsistat file in list for a given set

* Overhaul to plot_proftrace.py and proftrace_yaml.yaml: YAML provides series of figure-cards to produce multiple figures with different configurations, capability to set a figure name to None to suppress producing that figure for that figure-card

* Overhaul to plot_proftrace.py: Rename figure-name keys in YAML, allow for both error (rms/bias) and penalty (cpen) data to be plotted, allow missing figure-name keys in YAML to be interpreted as None (no plotting)

* Fixed pyGSI/gsi_stat.py pycodestyle error

* Removed legacy pyGSI/git_stat_pen.py test-code

* Update proftrace_yaml.yaml

proftrace_yaml.yaml

@@ -1,24 +1,55 @@
-set1:
-     name:    'MDCRS-OMB-t'
-     var:     't'
-     it:      1
-     use:     'asm'
-     typ:     133
-     styp:    None
-     statdir: /full/path/to/gsistat/files/
-set2:
-     name:    'MDCRS-OMA-t'
-     var:     't'
-     it:      3
-     use:     'asm'
-     typ:     133
-     styp:    None
-     statdir: /full/path/to/gsistat/files/
-figures:
-     pname:  'gsistat_profs.png'
-     tname:  'gsistat_trace.png'
-     tskip:  8
-dates:
-     dt_beg: '2020090800'
-     dt_end: '2020093018'
-     hrs: 6
+LWIR_figs:
+     set1:
+          name:    'LWIR-G17'
+          var:     'uv'
+          it:      1
+          use:     'asm'
+          typ:     245
+          styp:    '0271'
+          statdir: /path/to/set1/gsistat/files
+     set2:
+          name:    'LWIR-G18'
+          var:     'uv'
+          it:      1
+          use:     'asm'
+          typ:     245
+          styp:    '0272'
+          statdir: /path/to/set2/gsistat/files
+     figopts:
+          errProName:  gsistat_errprofs.png
+          errTraName:  gsistat_errtrace.png
+          penProName:  gsistat_penprofs.png
+          penTraName:  gsistat_pentrace.png
+          tskip:  16
+     dates:
+          dt_beg: '2022100100'
+          dt_end: '2022112018'
+          hrs: 6
+WVct_figs:
+     set1:
+          name:    'WVct-G17'
+          var:     'uv'
+          it:      1
+          use:     'asm'
+          typ:     246
+          styp:    '0271'
+          statdir: /path/to/set1/gsistat/files
+     set2:
+          name:    'WVct-G18'
+          var:     'uv'
+          it:      1
+          use:     'asm'
+          typ:     246
+          styp:    '0272'
+          statdir: /path/to/set2/gsistat/files
+     figopts:
+          errProName:  gsistat_errprofs_wvct.png
+          errTraName:  gsistat_errtrace_wvct.png
+          penProName:  None
+          penTraName:  None
+          tskip:  16
+     dates:
+          dt_beg: '2022100100'
+          dt_end: '2022112018'
+          hrs: 6
+

pyGSI/gsi_stat.py

@@ -285,8 +285,8 @@ def _get_conv(self, name):
                 # don't add cpen or qcpen either
                 # careful here, cpen here also removes qcpen
                 # hence the extra space before qcpen and cpen
-                if any(x in line for x in [' qcpen', ' cpen']):
-                    continue
+                # if any(x in line for x in [' qcpen', ' cpen']):
+                #    continue
                 tmp.append(line.strip().split())
 
         columns = header.split()