PR#295 update shared lib fixes

docs/README.clang

@@ -23,29 +23,5 @@ you are using an old version of RedHat or Ubuntu, it is probably time to
 update to a newer version.  AOCC will work on intel cpus.
 
 
-    https://developer.amd.com/amd-aocc/
+Only the AOCC version of the clang compilers are tested. (12/2024)
 
-To compile with AOCC
-  export CC=clang
-  export FC=flang
-  make
-
-To compile for Zen3 with AOCC
-  export CC=clang
-  export FC=flang
-  export CFFLAGS=-march=znversion3
-  export FFLAGS=-march=znversion3
-  make
-
-AOCC: Jasper does not compile correctly.  Use OpenJPEG to support jpeg2000.
-      OpenJPEG requires Cmake
-
-   USE_JASPER=0
-   USE_OPENJPEG=1
-
-Netcdf4 does not compile correctly. Must use
-
-   USE_NETCDF3=1
-   USE_NETCDF4=0
-
-Shared libraries work on linux (Ubuntu 20.04LTS)

docs/README.g2clib

@@ -1,29 +1,30 @@
-                      Changes for wgrib2 v0.1.9.8   revised 9.2017
+                              Wgrib2 & g2clib (g2c)
 
-Wgrib2 v0.1.9.8 changes the default decoders from g2clib to an emulation
-of g2clib using the internal decoders.  The g2clib decoders are now an 
-optional package.
 
-The change was precipitated by reports of segmentation faults.  The
-problem was finally traced back to a repository of precompiled programs.
-They were compiling wgrib2 using a precompiled official version of g2clib.
-This was bad because the official version of g2clib did not have
-the bug fixes included with the wgrib2 source code (since fixed).
-In addition the g2clib is often compiled with a weird makefile.  On 64-bit 
-systems, the library API uses 32-bit integers.  However, on 32-bit systems, 
-the library API uses 64-bit integers.  (Wgrib2 was using the opposite
-convention.)
+wgrib2 v0.1.9.8 change the default decoders from g2clib to an emulation
+                of g2clib using the internal decoders.  To enable the
+                g2clib decoder, use the option "-g2clib 1".
 
+wgrib2 v0.1.9.9 The g2clib decoders are now an optional package.
+
+wgrib2 v3.5.0   The routines that handled the png, jpeg2000 compression
+                were originally from g2clib.  The wgrib2 improvements to
+                to the routines have been ported back to g2clib.  With
+                move of the wgrib2 source code to github, it is easier to
+                use the github version of g2c and only have one copy of
+                the routines to maintain.
+
+                So to build wgrib2, you need to have g2c installed.  However,
+                the USE_G2CLIB enable the optional use of the g2c decoders.
+                As before, the g2c encoders are not used.
 
-                      Changes for wgrib2 v0.1.9.9
 
-The default configuration is not to include the g2clib decoders in the build.
 
 
                                  WMO Standard
 
 The g2clib and g2lib libraries do not follow the WMO standard when encoding
-and decoding non-zero constant fields when the decimal scaling is not zero .  
+and decoding non-zero constant fields when the decimal scaling is not zero.  
 This is a known problem for several years.  The post-processor developers 
 at NCEP work around the problem when encoding a constant field so the problem 
 is not commonly encountered.   
@@ -37,7 +38,11 @@ this problem is avoided.
 
 Wgrib2 internal routines either follow the WMO standard or the g2clib/g2lib
 behavior on non-zero constant fields with a non-zero decimal scaling.  The
-default for wgrib2 to emulate the g2clib/g2lib bug for NCEP files.  However, 
-if a non-zero constant field with a non-zero decimal scaling is encountered, 
-a warning message is given.  
-
+default for wgrib2 to emulate the g2clib/g2lib bug for reading NCEP files.  
+However, if wgrib2 has to emulate the bug, a warning message is given.
+(NCEP file has the center id set to 7.) For writing (not copying), wgrib2 
+writes grib data in a manner that is WMO compliant and can be read correctly 
+by g2clib/g2lib.
+
+     copying:   wgrib2 IN.grb -match "HGT" -grib hgt.grb
+     writing:   wgrib2 IN.grb -match "HGT" -grib_out hgt.grb

docs/README.gcc

@@ -1,4 +1,4 @@
 gnu:  default development compilers
 
-If you compile without OPENMP (the default), then you will see warnings 
+If you compile without OPENMP, then you will see warnings 
 about Ignoring #pragma omp ....  This is normal.  

docs/README.ipolates

@@ -1,3 +1,26 @@
+                          IPOLATES: 12/2024
+
+Wgrib2 v3.5.0 now uses NCEPLIB:ip v5.1+.  This means the the ip
+library is now current, maintained and supported by the NCEPlibs
+team.  Believe or not "supported" is actually an important word!
+With the older IP grib1 and IP grib2 libraries, high priority support 
+was for operational codes and a lower priority for internal users.
+
+IP v5.1 adds (1) standard C interface to fortran codes in IP, and
+support for NCEP and WMO defined rotated lat-lon grids.
+
+Wgrib2 v3.5.0 drops the suport for older grib1 and grib2 IP libraries.
+Otherwise, the use of the ipolates library remains the same as 
+default the double-precision grib2 IP library.  
+
+The spectral library is included with NCEPLIB:ip and their no
+longer an option to use the IPlib without including the SPlib.
+(SP wass originally not included because some SPlib was of unknown
+origin. 10 years later, the major code was identified as open 
+source, and the minor code was simple recoded.)
+
+-----------------------------------------------------------------
+
                  IPOLATES: grid interpolation library   8/2018
 
 IPOLATES is a fortran library from NCEP that interpolates between 

docs/README.jpeg2000

@@ -2,53 +2,21 @@
 
 
 Jpeg2000 produces one of the smallest grib files. The problem with
-jpeg2000 is that the open source libraries are single threaded and
-the algorithm is involved.  As a result, encoding and decoding
-jpeg2000 is slow. NCEP has adopted complex-packing for most of its
-models output because of the speed of decoding.  Parallelized decoders
-can by up to 20 times faster.
+jpeg2000 was that the opensource jpeg2000 were slow, and decoding
+jpeg2000 was up to 20x slower than complex packing.  Consequently
+NCEP has adopted complex-packing for most of its models output because 
+of the speed of decoding.  
 
 Support for jpeg2000 compressed grib files can be enabled by compiling 
-wgrib2 with either the Japser or OpenJPEG library.  Currently the
-default is the Jasper library.
+wgrib2 to use either the Japser or OpenJPEG library.
 
 Jasper:     Pro:  long-term use by wgrib2
-                  no need for CMake
-                  similar perforance to OpenJPEG
                   most people use Jasper
 
-                  compiles on my linux box at work (no cmake)
-                  Jasper is used by NCEP operational codes
-
             Con:  support is not as good as OpenJPEG (subjective)
-                  doesn't work on clang, icc, arm (?)
-
-		  default up to wgrib2 v3.1.2
+                  Historically supported by fewer compilers.  (current?)
 
 OpenJPEG:   Pro:  better suport than Jasper (subjective)
                   similar performance to Jasper
-                  haven't found a C compiler that can't compile
-
-            Con:  fewer builds using OpenJPEG
-		  minor: requires CMake
-
-                  default for wgrib2 v3.1.3+
-		  the AEC library used by ECMWF requires CMake,
-		  so using CMake for OpenJPEG is not a big deal.
-
-
-Wgrib2 v3.1.3+ transitions from Jasper to OpenJPEG for jpeg2000 support.
-My automated testing says that Jasper and OpenJPEG produce files that have
-the same decoded values.
-
-
-                         To Compile With OpenJPEG                  
-
-
-requires cmake v3.9+ and g++ to be installed
-wgrib2/decenc_jpeg.c is based on ECMWF code and licensed under 
-   the terms of the Apache Licence Version 2.0
 
-To enable, edit grib2/makefile
-   USE_JASPER=0
-   USE_OPENJPEG=1
+status 12/2024: github 3.5.0 can use either library.

docs/README.ncep

@@ -1,57 +1,6 @@
-How do I configure the makefile to support the ncep local defined rotated lat-lon grid
-edit grib2/makefile
-   USE_IPOLATES=1
-   USE_SPECTRAL=0
+   How to build on an NCEP machine
 
-How do I configure the makefile to support the WMO defined rotated lat-lon grid
-edit grib2/makefile  (default)
-   USE_IPOLATES=3
-   USE_SPECTRAL=1
+The current wgrib2 now lives on github
+     https://github.com/NOAA-EMC/wgrib2
 
-How do I configure the makefile to support netcdf4?
-edit grib2/makefile
-   USE_NETCDF3=0
-   USE_NETCDF4=1
-Note: you will be prompted to download the netcdf4 and hdf5 libraries
-in order to complete the make.
-
-
-How do I compile on NCEP's WCOSS2 system
-
-export CC=icc
-export FC=ifort
-export COMP_SYS=intel_linux
-make 
-(fails)
-cp lib64/*.a lib/
-make
-
-comment: WCOSS2 has altered how libraries are compiled using configure.
-Normally libraries are built in lib/; however, a local routine puts the
-libraries in lib64/.  Since this behavor has only been observed in
-WCOSS2, no fixes to the makefile ar planned.
-
-
-How do I compile on NCEP's workstations
-
-edit grib2/makefile
-cd (whatever)/grib2
-vi makefile
-change 
-USE_AEC=1
-to
-USE_AEC=0
-change
-USE_JASPER=0
-to
-USE_JASPER=1
-change
-USE_OPENJPEG=1
-to
-USE_OPENJPEG=0
-
-(from bash)
-export CC=gcc
-export FC=gfortran
-make 
 

docs/README.netcdf4

@@ -1,55 +1,16 @@
 		     Compiling wgrib2 with netcdf-4
 
-9/2022: wgrib2 v3.1.2beta3
-        compiles in Ubuntu 20.04LTS gcc 9.4.0
-        compiles in RedHat release 7.9 gcc 4.8.5
-        does not compile (hdf5) with AMD clang version 13.0.0
 
-4/2022: netcdf4 seems to be working gcc v9.4.0. clang is a no go.
-I am trying to compile on cygwin but it has been compiling for 3 hours.
+With the github version of wgrib2, you have to use the system version of netcdf4.
+To install netcdf under Ubuntu, I did
 
-8/2020:
+    used synaptic to install libnetcdf-dev  .. installs many packages
+    sudo apt install netcdf-bin             .. need ncdump to pass tests
 
-NetCDF4 support has been in shambles for many years.  The good news,
-Redhat with its vintage compilers has been working fine. If you
-are using the Intel compilers, it compiles and probably works.
+To build wgrib2 with netcdf, you modify the top level CMakeLists.txt
+from
+  option(USE_NETCDF "Use NetCDF" off)
+to
+  option(USE_NETCDF "Use NetCDF" on)
 
-
-Support:  There is no way to support using XYZ compiler in ABC operating
-system.  The user has to test whether their copy of wgrib2 with the NetCDF4 
-options works.  Don't assume that "it compiled" means "it worked". I have
-seen "it compiled" but "it doesn't work".  The NetCDF4 code seems to be
-robust but the hdf5 library is sensitive to the version of the compiler.
-
-Configuration:
-
-In the makefile, change the configuration to 
-
-USE_NETCDF3=0
-USE_NETCDF4=1
-
-
-Then you have to set the environment variables to the C and F90 compilers
-and run gnu make.
-
-(bash example)
-
-export CC=gcc               (set the C compiler)
-export FC=gfortran          (set the f90 compiler)
-make                        (on some machines this gnu make could be called gmake)
-
-The make will respond with an error message that you need to get the netcdf4 source code.
-You can get the source code using wget.
-
-make                        (on some machines this gnu make could be called gmake)
-
-The make will respond with another error message that you need to get the hdf5 source code.
-You can get the source code using wget.
-
-make                        (third time is a charm)
-
-
-This has been tested with gcc/gfortran and clang/gfortran on Ubuntu 12.04 (64 bits) and
-with gcc/fortran on Redhat 5 and 6 (64-bits).
-
-Notes: UCAR has a web page with help in compiling netcdf.
+and build normally.

docs/README.openmp

@@ -1,4 +1,4 @@
-                                OpenMP                    revised 7.2016, 1.2020, 10.2023
+                           OpenMP                    revised 7.2016, 1.2020, 10.2023, 12.2024
 
 OpenMP is a shared-memory parallel-programming API.  As the grid dimensions get larger,
 the need for parallel computing becomes more important.  With OpenMP, wgrib2 will use
@@ -13,24 +13,25 @@ You can control the number of cores used by the environment variable OMP_NUM_THR
 
 Status: requires OpenMP v3.1 or greater
 wgrib2 v3.1.3: allows code to be written for SIMD and GPUs
-               only SIMD is present in v3.1.3
-
+               SIMD used in in wgrib2 v3.1.3; however, only in the few
+                places. Limited to SIMD loops inside of threaded loops.
+               Short loops should parallelized by SIMD to avoid the 
+                OpenMP overhead.
 Hints:
      complex-packing reading is parallelized
-     complex-packing writing is partly parallelized.
+     complex-packing writing is slightly parallelized.
      simple-packing is parallelized
      jpeg2000, png, AEC are not parallelized because they depends on external libraries
-     geolocation is parallelized except when using Proj4.
+      (jpeg2000 may have some parallelizations depending on libraries used)
+     geolocation is parallelized except when using Proj.
      Running multiple copies of wgrib2 can be done along with OpenMP.
      Nodes with 24+ cores should have environment variable OMP_NUM_THREADS set to
-      a number less than the number of cores. Little perforance gain for
+      a number less than the number of free cores. Little perforance gain for
       a large number of cores.
      NUMA should be considered.
      wgrib2ms/wgrib2mv is faster than using wgrib2 with OpenMP.  
         really fast: wgrib2ms/wgrib2mv with AEC compression
-
-Future:
-     wgrib2 allows writting SIMD and GPU code through OpenMP pragmas.
-     Wgrib2 is frequently I/O bound, and the run time is usually quite fast.
-       So speeding up wgrib2 will have little impact.  Consider writing
-       SIMD and gpu code for wgrib2 an exercise on non-trivial code.
+     Most sections of wgrib2 do not scale well over a handfull of cores;
+      however, a couple of codes (-ens_processing) is both computationally
+      intensive, and hightly parallelized and it can use as many cores
+      as you give it.

docs/README.questions

@@ -0,0 +1,15 @@
+                  How to submit questions?
+
+Questions are now handled by github
+
+https://github.com/NOAA-EMC/wgrib2
+
+Select the issue tab.  You might want to see if the
+problem has been previously addressed.
+
+Why do you want to submit questions by github vs the old way?
+  - multiple people can answer
+  - you can help train these muiltiple people
+        (only one is an experienced wgrib2 user)
+  - solved issues are documented and can help others
+