diff --git a/.gitignore b/.gitignore index 27823e7f54..e24a481063 100644 --- a/.gitignore +++ b/.gitignore @@ -85,9 +85,6 @@ ctsm.input_data_list.previous # mksurfdata unit tests unit_test_build -# Tools executables -/tools/mkprocdata_map/mkprocdata_map - # run_neon output directories /tools/site_and_regional/listing.csv /tools/site_and_regional/????/ diff --git a/README b/README index 8b4e15e557..bde1485450 100644 --- a/README +++ b/README @@ -130,8 +130,6 @@ cime_config/usermods_dirs -------- Directories of sets of user-modification subd tools/mksurfdata_esmf --------- Directory to build program to create surface dataset at any resolution. -tools/mkprocdata_map ---------- Process history data from unstructed grids to a gridded - format. tools/mkmapgrids -------------- NCL script to create a SCRIP grid file for a regular lat/lon grid (deprecated) tools/crop_calendars ---------- Tools to process and process and create crop calendar datasets for CTSM tools/modify_input_files ------ Script to modify existing CTSM input datasets in standard ways diff --git a/doc/source/users_guide/using-clm-tools/building-the-clm-tools.rst b/doc/source/users_guide/using-clm-tools/building-the-clm-tools.rst index 95e0333d6d..a397834c2b 100644 --- a/doc/source/users_guide/using-clm-tools/building-the-clm-tools.rst +++ b/doc/source/users_guide/using-clm-tools/building-the-clm-tools.rst @@ -4,96 +4,9 @@ .. include:: ../substitutions.rst -The CLM FORTRAN tools all have similar makefiles, and similar options for building. The tools **cprnc** and **gen_domain** use the CIME configure/build system which is described in the next section. +The tools **cprnc** and **gen_domain** use the CIME configure/build system which is described in the next section. -The Makefiles (for **mksurfdata_esmf** and **mkprocdata_map**) use GNU Make extensions and thus require that you use GNU make to use them. They also auto detect the type of platform you are on, using "uname -s" and set the compiler, compiler flags and such accordingly. There are also environment variables that can be set to set things that must be customized. All the tools use NetCDF and hence require the path to the NetCDF libraries and include files. On some platforms (such as Linux) multiple compilers can be used, and hence there are env variables that can be set to change the FORTRAN and/or "C" compilers used. The tools also allow finer control, by also allowing the user to add compiler flags they choose, for both FORTRAN and "C", as well as picking the compiler, linker and and add linker options. Finally the tools allow you to turn optimization on (which is off by default but on for **mksurfdata_esmf**) with the OPT flag so that the tool will run faster. - -Options used by all: **mksurfdata_esmf** - -- ``LIB_NETCDF`` -- sets the location of the NetCDF library. -- ``INC_NETCDF`` -- sets the location of the NetCDF include files. -- ``USER_FC`` -- sets the name of the FORTRAN compiler. - -Options used by: **mkprocdata_map**, and **mksurfdata_esmf** - -- ``MOD_NETCDF`` -- sets the location of the NetCDF FORTRAN module. -- ``USER_LINKER`` -- sets the name of the linker to use. -- ``USER_CPPDEFS`` -- adds any CPP defines to use. -- ``USER_CFLAGS`` -- add any "C" compiler flags to use. -- ``USER_FFLAGS`` -- add any FORTRAN compiler flags to use. -- ``USER_LDFLAGS`` -- add any linker flags to use. -- ``USER_CC`` -- sets the name of the "C" compiler to use. -- ``OPT`` -- set to TRUE to compile the code optimized (TRUE or FALSE) -- ``SMP`` -- set to TRUE to turn on shared memory parallelism (i.e. OpenMP) (TRUE or FALSE) -- ``Filepath`` -- list of directories to build source code from. -- ``Srcfiles`` -- list of source code filenames to build executable from. -- ``Makefile`` -- customized makefile options for this particular tool. -- ``mkDepends`` -- figure out dependencies between source files, so make can compile in order.. -- ``Makefile.common`` -- General tool Makefile that should be the same between all tools. - -More details on each environment variable. - -``LIB_NETCDF`` - This variable sets the path to the NetCDF library file (``libnetcdf.a``). If not set it defaults to ``/usr/local/lib``. In order to use the tools you need to build the NetCDF library and be able to link to it. In order to build the model with a particular compiler you may have to compile the NetCDF library with the same compiler (or at least a compatible one). - -``INC_NETCDF`` - This variable sets the path to the NetCDF include directory (in order to find the include file ``netcdf.inc``). if not set it defaults to ``/usr/local/include``. - -``MOD_NETCDF`` - This variable sets the path to the NetCDF module directory (in order to find the NetCDF FORTRAN-90 module file when NetCDF is used with a FORTRAN-90 **use statement**. When not set it defaults to the ``LIB_NETCDF`` value. - -``USER_FC`` - This variable sets the command name to the FORTRAN-90 compiler to use when compiling the tool. The default compiler to use depends on the platform. And for example, on the AIX platform this variable is NOT used - -``USER_LINKER`` - This variable sets the command name to the linker to use when linking the object files from the compiler together to build the executable. By default this is set to the value of the FORTRAN-90 compiler used to compile the source code. - -``USER_CPPDEFS`` - This variable adds additional optional values to define for the C preprocessor. Normally, there is no reason to do this as there are very few CPP tokens in the CLM tools. However, if you modify the tools there may be a reason to define new CPP tokens. - -``USER_CC`` - This variable sets the command name to the "C" compiler to use when compiling the tool. The default compiler to use depends on the platform. And for example, on the AIX platform this variable is NOT used - -``USER_CFLAGS`` - This variable adds additional compiler options for the "C" compiler to use when compiling the tool. By default the compiler options are picked according to the platform and compiler that will be used. - -``USER_FFLAGS`` - This variable adds additional compiler options for the FORTRAN-90 compiler to use when compiling the tool. By default the compiler options are picked according to the platform and compiler that will be used. - -``USER_LDFLAGS`` - This variable adds additional options to the linker that will be used when linking the object files into the executable. By default the linker options are picked according to the platform and compiler that is used. - -``SMP`` - This variable flags if shared memory parallelism (using OpenMP) should be used when compiling the tool. It can be set to either TRUE or FALSE, by default it is set to FALSE, so shared memory parallelism is NOT used. When set to TRUE you can set the number of threads by using the OMP_NUM_THREADS environment variable. Normally, the most you would set this to would be to the number of on-node CPU processors. Turning this on should make the tool run much faster. - -.. warning:: Note, that depending on the compiler answers may be different when SMP is activated. - -``OPT`` - This variable flags if compiler optimization should be used when compiling the tool. It can be set to either ``TRUE`` or ``FALSE``, by default it is set to for both **mksurfdata_esmf** and **mkprocdata_map**. Turning this on should make the tool run much faster. - -.. warning:: Note, you should expect that answers will be different when ``OPT`` is activated. - -``Filepath`` - All of the tools are stand-alone and don't need any outside code to operate. The Filepath is the list of directories needed to compile and hence is always simply "." the current directory. Several tools use copies of code outside their directory that is in the CESM distribution (either ``csm_share`` code or CLM source code). - -``Srcfiles`` - The ``Srcfiles`` lists the filenames of the source code to use when building the tool. - -``Makefile`` - The ``Makefile`` is the custom GNU Makefile for this particular tool. It will customize the ``EXENAME`` and the optimization settings for this particular tool. - -``Makefile.common`` - The ``Makefile.common`` is the copy of the general GNU Makefile for all the CLM tools. This file should be identical between the different tools. This file has different sections of compiler options for different Operating Systems and compilers. - -``mkDepends`` - The ``mkDepends`` is the copy of the perl script used by the ``Makefile.common`` to figure out the dependencies between the source files so that it can compile in the necessary order. This file should be identical between the different tools. - -.. note:: There are several files that are copies of the original files. By having copies the tools can all be made stand-alone, but any changes to the originals will have to be put into the tool directories as well. - -The *README.filecopies* (which can be found in ``$CTSMROOT/tools``) is repeated here. - -.. include:: ../../../../tools/README.filecopies - :literal: +The only CLM FORTRAN tool is mksurfdata_esmf which has it's own build system that takes advantage of the cime build. ================================================================ Building the CLM tools that use the CIME configure/build system diff --git a/doc/source/users_guide/using-clm-tools/what-are-the-clm-tools.rst b/doc/source/users_guide/using-clm-tools/what-are-the-clm-tools.rst index 664e23a220..4778294df9 100644 --- a/doc/source/users_guide/using-clm-tools/what-are-the-clm-tools.rst +++ b/doc/source/users_guide/using-clm-tools/what-are-the-clm-tools.rst @@ -18,15 +18,13 @@ The list of generally important scripts and programs are as follows. #. *mksurfdata_esmf* to create surface datasets from grid datasets (clm4_0 and |version| versions). -#. *./mkprocdata_map* to interpolate output unstructured grids (such as the CAM HOMME dy-core "ne" grids like ne30np4) into a 2D regular lat/long grid format that can be plotted easily. Can be used by either clm4_0 or |version|. - #. *$CIMEROOT/tools/mapping/gen_domain_files/gen_domain* to create a domain file for datm from a mapping file. The domain file is then used by BOTH datm AND CLM to define the grid and land-mask. #. *$CIMEROOT/tools/cprnc* to compare two NetCDF files. In the sections to come we will go into detailed description of how to use each of these tools in turn. First, however we will discuss the common environment variables and options that are used by all of the FORTRAN tools. Second, we go over the outline of the entire file creation process for all input files needed by CLM for a new resolution, then we turn to each tool. In the last section we will discuss how to customize files for particular observational sites. -The FORTRAN tools (mksurfdata_esmf and mkprocdata_map) run, with a namelist (mksurfdata_esmf) to provide options, or with command line arguments (mkprocdata_map). +The FORTRAN tool (mksurfdata_esmf) runs, with a namelist and has a namelist builder for it. In the following sections, we will outline how to make these files available for build-namelist so that you can easily create simulations that include them. In the chapter on single-point and regional datasets we also give an alternative way to enter new datasets without having to edit files. @@ -39,29 +37,13 @@ Running FORTRAN tools with namelists ./program < namelist -There is a sample namelist called ``$CTSMROOT/tools/mksurfdata_esmf/mksurfdata_esmf.namleist`` that shows you what the namelist should look like. **mksurfdata_esmf** also has a script that creates the namelist and runs the program for you. Namelists that you create should be similar to the example namelist. The namelist values are also documented along with the other namelists in the: +**mksurfdata_esmf** also has a script that creates the namelist and runs the program for you. The namelist values are also documented along with the other namelists in the: :: - $CTSMROOT/bld/namelist_files/namelist_definition.xml`` file - and default values in the: - $CTSMROOT/bld/namelist_files/namelist_defaults_clm_tools.xml`` file. + $CTSMROOT/tools/mksurfdata_esmf/gen_mksurfdata_namelist.xml`` file ----------------------------------------------- Running FORTRAN tools with command line options ----------------------------------------------- -**gen_domain**, mkprocdata_map, and **cprnc** run with command line arguments. The detailed sections below will give you more information on the command line arguments specific to each tool. Also running the tool without any arguments will give you a general synopsis on how to run the tool. - ------------------------------------------ -Running FORTRAN tools built with SMP=TRUE ------------------------------------------ - -When you enable ``SMP=TRUE`` on your build of one of the tools that make use of it, you are using OpenMP for shared memory parallelism (SMP). In SMP loops are run in parallel with different threads run on different processors all of which access the same memory (called on-node). Thus you can only usefully run up to the number of processors that are available on a single-node of the machine you are running on. For example, on the NCAR machine cheyenne there are 36 processors per node, so you can use up to 36 processors. - -.. _using-ncl: - ---------- -Using NCL ---------- - -In the tools directory ``$CTSMROOT/tools/ncl_scripts`` and in a few other locations there are scripts that use NCAR Command Language (NCL). Unlike the FORTRAN tools, you will need to get a copy of NCL in order to use them. You also won't have to build an executable in order to use them, hence no Makefile is provided. NCL is provided for free download as either binaries or source code from: `http://www.ncl.ucar.edu/ `_. The NCL web-site also contains documentation on NCL and it's use. These scripts are stand-alone and at most use environment variables to control how to use them. In some cases there are perl scripts with command line arguments that call the NCL scripts to control what they do. +**gen_domain** and **cprnc** run with command line arguments. The detailed sections below will give you more information on the command line arguments specific to each tool. Also running the tool without any arguments will give you a general synopsis on how to run the tool. diff --git a/tools/README b/tools/README index 568dc1239b..693e793370 100644 --- a/tools/README +++ b/tools/README @@ -11,8 +11,6 @@ I. General directory structure: crop_calendars --- Regrid and process GGCMI sowing and harvest date files for use in CTSM. mkmapgrids ------- Create regular lat/lon SCRIP grid files - mkprocdata_map --- Convert output unstructured grids into a 2D format that - can be plotted easily site_and_regional Scripts for handling input datasets for site and regional cases. These scripts both help with creation of datasets using the @@ -33,49 +31,6 @@ I. General directory structure: II. Notes on building/running for each of the above tools: - mkprocdata_map has the following files to facilitate building the FORTRAN code: - - README ------- Specific help for using the specific tool and help on specific - files in that directory. - src/Filepath ----- List of directories needed to build the tool - (some files in ../src directories are required). - src/Makefile ----- GNU Makefile to build the tool - (these are identical between tools. - src/Macros.custom Customization of make macros for the particular tool in question - src/Srcfiles ----- List of source files that are needed. - src/Mkdepends ---- Dependency generator program - - To build: - - cd - setenv INC_NETCDF - setenv LIB_NETCDF - gmake - - The process will create a file called "Depends" which has the dependencies - for the build of each file on other files. - - By default some codes may be compiled non-optimized - so that you can use the debugger, and with bounds-checking, and float trapping on. - To speed up do the following... - - gmake OPT=TRUE - - Also some of the tools allow for OpenMP shared memory parallelism - (such as mksurfdata) with - - gmake SMP=TRUE - - To run a program with a namelist: - - ./program < namelist - - To run a program built with SMP=TRUE: - - setenv OMP_NUM_THREADS= - - run normally as above - mksurfdata_esmf has a cime configure and CMake based build using the following files: gen_mksurfdata_build ---- Build mksurfdata_esmf diff --git a/tools/README.testing b/tools/README.testing deleted file mode 100644 index 9c386a3b26..0000000000 --- a/tools/README.testing +++ /dev/null @@ -1,58 +0,0 @@ -tools/README.testing May/23/2011 - -There is automated testing for all of the tools and scripts under this tools directory. -The tests are in the test/tools directory and are any of the scripts -that have "tools" in the name. There are several assumptions made in order for the -testing to work. - - -1.) Executable name is the same as the directory name - -The name of the executable program is the same as the directory name of the tool. - -2.) Build works the same for any Fortran tools - -The build for any Fortran tools should work the same way, with the same options -and required files for it. The files: Makefile, Mkdepends, Filepath and Srcfile -are expected to exist in the tool "src" sub-directory. To make maintaining these files easier -in general the Makefile and Mkdepends files should be kept identical other than -default settings for OPT and SMP and the output executable name. - -Options to the Makefile: - - LIB_NETCDF --- Library directory location of NetCDF. (defaults to /usr/local/lib) - INC_NETCDF --- Include directory location of NetCDF. (defaults to /usr/local/include) - MOD_NETCDF --- Module directory location of NetCDF. (defaults to $LIB_NETCDF) - USER_FC ------ Allow user to override the default Fortran compiler specified in Makefile. - USER_FCTYP --- Allow user to override the default type of Fortran compiler - (Linux and USER_FC=ftn only). - USER_CC ------ Allow user to override the default C compiler specified in Makefile - (Linux only). - USER_LINKER -- Allow user to override the default linker specified in Makefile. - SMP ---------- Shared memory Multi-processing (TRUE or FALSE) [default is FALSE] - OPT ---------- Use optimized options. (TRUE or FALSE) - -3.) Successful completion of the tool ends with "Successfully ..." - -After the tool completes it should have an unique string telling of the -successful completion of the file that is searchable in the log file starting -with "Successfully ". If this string isn't found in the log file, it -will be assumed by the test setup that the test failed. - -4.) NetCDF files are created or modified - -It is assumed that NetCDF files are created and/or modified by the process. And if -NetCDF files are NOT created -- something went wrong. For some scripts that copy files -to other locations this means an "-nomv" option needs to be invoked (and one provided) -so that it leaves the files created in the current directory. - -5.) Namelist examples exist .* namelist files - -To specify options for the running of the tool, sample namelist files -are provided or a sample run options file. These files can EITHER be in the -tool directory OR the ../../test/tools/nl_files directory. - -6.) Specific tests for run scripts - -For tools that have scripts to create namelists and run the tool for you, there -are customized tests to run these tools. diff --git a/tools/mkprocdata_map/README b/tools/mkprocdata_map/README deleted file mode 100644 index 92ffb4856c..0000000000 --- a/tools/mkprocdata_map/README +++ /dev/null @@ -1,151 +0,0 @@ -$CTSMROOT/tools/mkprocdata_map/clm/README Oct 30, 2012 - -This directory contains scripts for regridding CLM output from an -unstructured grid (1-d output using the lndgrid dimension) to a 2-d -(lat/lon) grid. The regridding method is area-conservative. - -The following steps provide a method to create the necessary inputs to -this script, produce an executable, and regrid output: - -In the following instructions, the "original" resolution is the -resolution of the run on an unstructured grid, and the "target" -resolution is the regular lat/lon resolution to which you will regrid -the output. - -(0) Install prerequisites: - - (a) If you do not already have a mapping file from the original - resolution to the target resolution, you will need the - ESMF_RegridWeightGen tool installed on your system. - - (b) The wrapper scripts describe below require the netCDF operators - (NCO). These nco tools (ncks, ncap2, etc.) must be in your path. - -(1) Determine the target resolution. This resolution must be a regular - lat/lon resolution. Generally, this should be a resolution close - to the resolution of the CLM run. For example, when running CLM at - ne30_np4 resolution, a good target resolution is 0.9x1.25 (i.e., - finite volume 1 degree: f09); when running CLM at ne120_np4 - resolution, a good target resolution is 0.23x0.31 (i.e., finitev - volume 1/4 degree: f02). - -(2) Perform a short CLM run at the target resolution, producing at - least one history file. After this run completes, set the - environment variable $TEMPLATE_FILE to point to one of the history - files created by this run. - -(3) Create a conservative mapping file from the original resolution to - the target resolution using the ESMF regrid weight generator. The - basic method for doing this is: - - $ESMF_PATH/bin/ESMF_RegridWeightGen -s $INGRID -d $OUTGRID -m conserve -w $MAP_FILE -i - - where $INGRID gives the path to a SCRIP grid file at the original - resolution, $OUTGRID gives the path to a SCRIP grid file at the - template resolution, and $MAP_FILE gives the name of the mapping - file that will be generated. - - However, you may want to wrap this in a job script to run it on - multiple processors (using mpirun), and you may have to set other - machine-specific environment variables. - -(4) Build the mkprocdata_map tool. From the current directory, do the - following: - - > cd src - > gmake - > cd .. - - By default code compiles optimized so it's reasonably fast. If you want - to use the debugger, with bounds-checking, and float trapping on do the - following: - gmake OPT=FALSE - See Also: See the components/clm/tools/README file for notes about setting - the path for NetCDF. - - This builds the mkprocdata_map executable. However, you generally - will not want to run this executable directly: instead, you should - use one of the wrapper scripts described below. - -(5) Do the regridding using one of the wrapper scripts in this - directory. To determine which script is most appropriate: Do you - need to regrid just one or a few output files, or most/all of the - output files in a directory? - - (a) If you are regridding just one or a few output files, you can - use mkprocdata_map_wrap. Its usage is: - - > mkprocdata_map_wrap -i input_file -o output_file -m $MAP_FILE -t $TEMPLATE_FILE - - where: - - input_file is the CLM history file you want to regrid - - output_file is the name of the regridded file that will be - created - - $MAP_FILE is the ESMF conservative mapping file created in - step (3) - - $TEMPLATE_FILE is a CLM history file at the target resolution, - created in step (2) - - You may also specify the '-l' option to this script. This option - determines whether to determine landfrac and related variables - by regridding the input file (when you don't give the '-l' - option), or by copying these variables from the template file - (when you give the '-l' option). These variables are important - for computing regional and global averages, e.g., as is done in - the land diagnostics package. Each method may be reasonable, - depending on the purposes of the regridding. For example, if you - want regional/global integrals to be as true as possible to the - original run, you should run withOUT the '-l' option; but if you - want to compare regional/global integrals between the original - run and a run at the target resolution, then you may want to run - WITH the '-l' option. - - Run 'mkprocdata_map_wrap -h' for full usage - - (b) If you need to regrid most or all of the output files in a - directory, you can use the convenience script - mkprocdata_map_all. This script runs mkprocdata_map_wrap on all - files matching a given pattern within a directory. Its basic - usage is the following, done from a directory containing many - CLM history files: - - > /path/to/mkprocdata_map_all -p $CASE -m $MAP_FILE -t $TEMPLATE_FILE - - where: - - $CASE is the case name of the original run (this -p argument - is actually more general: it provides the prefix of files on - which mkprocdata_map_wrap should be run; run - 'mkprocdata_map_all -h' for details) - - $MAP_FILE is the ESMF conservative mapping file created in - step (3) - - $TEMPLATE_FILE is a CLM history file at the target resolution, - created in step (2) - - There are a number of additional optional arguments to this - script, including the '-l' option described in (a), above. Run - 'mkprocdata_map_all -h' for full usage. - - ------------------------------------------------------------------------- -Some miscellaneous notes on the scripts contained here ------------------------------------------------------------------------- - -- area vs. area_regridded in the output of mkprocdata_map_wrap and - mkprocdata_map_all: The 'area' variable gives the actual grid cell - area on the destination grid. The 'area_regridded' variable is the - result of performing the regridding procedure on the 'area' variable - in the original source data. This seems to be the wrong way to - regrid areas (e.g., it leads to global totals that do not make - sense). However, area_regridded is left in the regridded files as a - diagnostic. BUT PLEASE USE CAUTION IF USING THIS AREA_REGRIDDED - VALUE, UNLESS YOU KNOW WHAT IT REALLY REPRESENTS! - -- At least as of this writing (Oct 29, 2012), there is insufficient - metadata on the CLM history files to regrid all variables - perfectly. In particular, note that many CLM history variables apply - only over a subset of the grid cell (e.g., over the non-lake portion - of the grid cell). Thus, to regrid these variables appropriately, we - would need to weight each grid cell's value by the portion of the - grid cell over which the field applies. However, doing this would - require metadata about each field that is not currently - available. diff --git a/tools/mkprocdata_map/README.filedescriptions b/tools/mkprocdata_map/README.filedescriptions deleted file mode 100644 index e657e7c7d9..0000000000 --- a/tools/mkprocdata_map/README.filedescriptions +++ /dev/null @@ -1,25 +0,0 @@ -$CTSMROOT/tools/mkprocdata_map/README.filedescriptions Erik Kluzek - 06/08/2018 - -mkprocdata_map_all ------------ Script to run over a list of files -mkprocdata_map_wrap ----------- Main script to actually use -mkprocdata_map_functions.bash - Bash shell functions to use in other scripts -README ------------------------ Description and how to run -src --------------------------- Directory with FORTRAN source code - -Also there are some sample files that can be used for testing under inputdata in - -$DIN_LOC_ROOT/lnd/clm2/test_mkprocdata_map - -See how this is done by looking at the file for testing mkprocdata_map: - -../../test/tools/nl_files/mkprocdata_ne30_to_f19_I2000 - -Which does something like the following: - -./mkprocdata_map_wrap \ --i $DIN_LOC_ROOT/lnd/clm2/test_mkprocdata_map/clm4054_ne30g16_I2000.clm2.h0.2000-01_c170430.nc \ --o ne30output_onf19grid.nc \ --m $DIN_LOC_ROOT/lnd/clm2/test_mkprocdata_map/map_ne30np4_nomask_to_fv1.9x2.5_nomask_aave_da_c121107.nc \ --t $DIN_LOC_ROOT/lnd/clm2/test_mkprocdata_map/clm4054_f19g16_I2000.clm2.h0.2000-01_c170430.nc - diff --git a/tools/mkprocdata_map/mkprocdata_map_all b/tools/mkprocdata_map/mkprocdata_map_all deleted file mode 100755 index 73e8abedf1..0000000000 --- a/tools/mkprocdata_map/mkprocdata_map_all +++ /dev/null @@ -1,202 +0,0 @@ -#!/bin/bash - -# This script runs mkprocdata_map_wrap on all files matching a given -# pattern within a directory. - -# Created by Bill Sacks, 5-26-11 - -# ---------------------------------------------------------------------- -# LOCAL FUNCTIONS DEFINED HERE -# ---------------------------------------------------------------------- - -function Usage { - script_name=`basename $0` - echo "Usage: $script_name -p prefix -m map_file -t template_file [-d] [-e executable-path] [-h] [-i] [-l] [-o output_suffix] [-r diRectory] [-s suffix]" - echo "" - echo "This script runs mkprocdata_map_wrap on all files matching a" - echo "given pattern within a directory." - echo "" - echo "'prefix' gives the prefix of the files on which" - echo "mkprocdata_map_wrap should be run; 'prefix' should NOT contain" - echo "wildcard characters. The prefix is also used to translate" - echo "from input to output file names (see examples below)" - echo "" - echo "'map_file' gives the name (and full path if not in the current" - echo "directory) of the mapping file" - echo "" - echo "'template_file' gives the name (and full path if not in the" - echo "current directory) of the template file, from which we read" - echo "lats, lons and some other variables" - echo "" - echo "The following are optional arguments:" - echo "" - echo "[-d]: Do a test (Dry run): do all error-checking on" - echo " arguments and print commands that would be run, but" - echo " don't actually run commands" - echo "" - echo "[-e executable-path]: Gives the path of the mkprocdata_map executable." - echo " If not specified, the path is determined by the" - echo " default value in mkprocdata_map_wrap." - echo "" - echo "[-h]: Print this help message and exit" - echo "" - echo "[-i]: Ignore (skip) existing output files; if this option is" - echo " not specified, then the script dies with an error if" - echo " any of the desired output files already exist" - echo "" - echo "[-l]: Option passed to mkprocdata_map_wrap: rather than computing" - echo " landfrac and related variables by regridding the input file," - echo " instead copy these variables directly from the template file." - echo "" - echo "[-o output_suffix]: suffix to append to the end of the prefix" - echo " on the output files" - echo " If not specified, '_2d' is used" - echo "" - echo "[-r diRectory]: Do the processing in the given directory." - echo " If not specified, processing is done in the" - echo " current working directory." - echo "" - echo "[-s suffix]: Run mkprocdata_map_wrap on all files matching the" - echo " pattern '\${prefix}\${suffix}'. The suffix can -" - echo " and often will - contain wildcards; but" - echo " remember to enclose 'suffix' in quotes to" - echo " prevent shell expansion." - echo " If not specified, run mkprocdata_map_wrap on all" - echo " files matching '\${prefix}*'" - echo "" - echo "" - echo "Example: $script_name -p Ib14_ne30np4_gx1v6 -m map_ne30np4_to_fv1.9x2.5_aave_da_091230.nc -t Ib19_1.9x2.5_gx1v6.clm2.h0.0001-01.nc" - echo "This will run mkprocdata_map_wrap on all files whose names begin" - echo "with 'Ib14_ne30np4_gx1v6' in the current directory, using the" - echo "mapping file named 'map_ne30np4_to_fv1.9x2.5_aave_da_091230.nc'" - echo "and the template file named 'Ib19_1.9x2.5_gx1v6.clm2.h0.0001-01.nc'" - echo "For an input file named:" - echo " Ib14_ne30np4_gx1v6.clm2.h0.0001-01-06-00000.nc" - echo "The output file will be named:" - echo " Ib14_ne30np4_gx1v6_2d.clm2.h0.0001-01-06-00000.nc" - echo "" - echo "Example: $script_name -o '_remap' -s '*.h0.0001*.nc' -p Ib14_ne30np4_gx1v6 -m map_ne30np4_to_fv1.9x2.5_aave_da_091230.nc -t Ib19_1.9x2.5_gx1v6.clm2.h0.0001-01.nc" - echo "This will run mkprocdata_map_wrap on all files whose names match" - echo "the pattern 'Ib14_ne30np4_gx1v6*.h0.0001*.nc', in the" - echo "current directory, using the mapping file named" - echo "'map_ne30np4_to_fv1.9x2.5_aave_da_091230.nc' and the" - echo "template file named Ib19_1.9x2.5_gx1v6.clm2.h0.0001-01.nc" - echo "For an input file named:" - echo " Ib14_ne30np4_gx1v6.clm2.h0.0001-01-06-00000.nc" - echo "The output file will be named:" - echo " Ib14_ne30np4_gx1v6_remap.clm2.h0.0001-01-06-00000.nc" - echo "" -} - -# ---------------------------------------------------------------------- -# BEGIN MAIN SCRIPT -# ---------------------------------------------------------------------- - -script_dir=`dirname $0` -source $script_dir/mkprocdata_map_functions.bash - -# ---------------------------------------------------------------------- -# Handle command-line arguments -# ---------------------------------------------------------------------- - -# define default values: -# required arguments: -prefix="" -map_file="" -template_file="" -# optional arguments: -directory="." -ignore_existing=0 -output_suffix="_2d" -suffix="*" -dryrun=0 -extra_args="" - -while getopts de:hilm:o:p:r:s:t: opt; do - case $opt in - d) dryrun=1;; - e) extra_args="$extra_args -e $OPTARG";; - h) Usage; exit;; - i) ignore_existing=1;; - l) extra_args="$extra_args -l";; - m) map_file=$OPTARG;; - o) output_suffix=$OPTARG;; - p) prefix=$OPTARG;; - r) directory=$OPTARG;; - s) suffix=$OPTARG;; - t) template_file=$OPTARG;; - \?) Usage; exit 1 - esac -done - -# ---------------------------------------------------------------------- -# Error checking on arguments -# ---------------------------------------------------------------------- - -if [ -z "$prefix" ]; then - echo "Must specify a prefix" - Usage - exit 1 -fi - -check_file_arg "$map_file" "map" -check_file_arg "$template_file" "template" - -# Make sure directory is really a directory -if [ ! -d $directory ]; then - echo "ERROR: $directory is not a directory" - echo "" - Usage - exit 1 -fi - - -# ---------------------------------------------------------------------- -# Change to desired directory -# ---------------------------------------------------------------------- - -olddir=`pwd` -cd $directory - -# ---------------------------------------------------------------------- -# Get list of files matching the given pattern; make sure there really -# are some matching files -# ---------------------------------------------------------------------- - -files=`ls ${prefix}${suffix}` -if [ $? -ne 0 ]; then - echo "ERROR trying to find files matching: ${prefix}${suffix}" - echo "" - Usage - exit 1 -fi - -# ---------------------------------------------------------------------- -# Loop through files matching the given pattern; run mkprocdata_map_wrap for each -# ---------------------------------------------------------------------- - -for infile in $files; do - outfile=${infile/$prefix/${prefix}${output_suffix}} - if [ -e $outfile ]; then - if [ $ignore_existing -eq 0 ]; then - echo "" - echo "ERROR: output file $outfile already exists" - exit 1 - else - echo "" - echo "WARNING: output file $outfile already exists: skipping" - echo "" - fi - - else # outfile does not exist - echo "" - do_cmd "${script_dir}/mkprocdata_map_wrap -i $infile -o $outfile -m $map_file -t $template_file $extra_args" $dryrun - fi -done - -# ---------------------------------------------------------------------- -# Clean up -# ---------------------------------------------------------------------- - -cd $olddir - diff --git a/tools/mkprocdata_map/mkprocdata_map_functions.bash b/tools/mkprocdata_map/mkprocdata_map_functions.bash deleted file mode 100644 index bbc359fb89..0000000000 --- a/tools/mkprocdata_map/mkprocdata_map_functions.bash +++ /dev/null @@ -1,57 +0,0 @@ -#!/bin/bash - -# This file contains functions used by other bash scripts in this directory. - -# This function echoes the command given by $1 (cmd), then executes it. -# However, if $2 (dryrun) is non-zero, then it only does the echo, not the execution. -# Usage: do_cmd cmd dryrun -# Returns 0 on success, non-zero on failure; if there is an error, the error string is echoed. -function do_cmd { - if [[ $# -ne 2 ]]; then - echo "ERROR in do_cmd: wrong number of arguments: expected 2, received $#" - exit 1 - fi - - local cmd=$1 - local dryrun=$2 - - echo $cmd - if [ $dryrun -eq 0 ]; then - # We use 'eval $cmd' rather than just '$cmd', because the - # latter doesn't work right if the command contains any quoted - # strings (e.g., svn ci -m "this is my message") - eval $cmd - if [ $? -ne 0 ]; then - echo "ERROR in do_cmd: error executing command" - exit 2 - fi - fi - - return 0 -} - -# make sure that the given file name argument was provided, and that -# the file exists; exit the script with a usage message if either of -# these is not true -# -# Usage: check_file_arg filename_arg description -# (description is echoed if there is an error) -# Example: check_file_arg "$input_file" "input" -# (note that $input_file must be in quotes) -function check_file_arg { - local filename=$1 - local description=$2 - - if [ -z "$filename" ]; then - echo "ERROR: Must specify $description file" - Usage - exit 1 - fi - - if [ ! -f $filename ]; then - echo "ERROR: Can't find $description file: $filename" - Usage - exit 1 - fi -} - diff --git a/tools/mkprocdata_map/mkprocdata_map_wrap b/tools/mkprocdata_map/mkprocdata_map_wrap deleted file mode 100755 index 4744b0eacc..0000000000 --- a/tools/mkprocdata_map/mkprocdata_map_wrap +++ /dev/null @@ -1,250 +0,0 @@ -#!/bin/bash - -# This script is a wrapper around mkprocdata_map that runs that -# program and then copies some additional variables from the template -# file to the output file. It also does some additional pre and -# post-processing in order to create some additional variables. - -# Created by Bill Sacks, 5-25-11 - -# ---------------------------------------------------------------------- -# SET PARAMETERS HERE -# ---------------------------------------------------------------------- - -# comma-delimited list of extra variables to copy directly from -# template file; note that these variables should not be written out -# by mkprocdata_map (i.e., everything in this list should be listed in -# the 'ignore_var' function in mkprocdata_map.F90); however, there may -# be some variables in the 'ignore_var' function that are not listed -# here - e.g., variables that we treat specially. -copy_vars="lon,lat" - -# comma-delimited list of extra variables to copy from the template -# file if the -l option is specified -- this option says to copy -# landfrac and related variables. Note that some of these variables -# may be written out by mkprocdata_map, in which case they will be -# overwritten afterwards (slighly less efficient, but that keeps -# things simpler). -landfrac_copy_vars="landfrac,landmask,pftmask" - -# name of the executable; -# expected to be in the same directory as this script unless -e option is given -executable="mkprocdata_map" - -# minimum value for regridded pftmask variable for the output variable to be 1 -pftmask_min="1.e-6" - -# fill value for landmask -landmask_fill=-9999 - -# ---------------------------------------------------------------------- -# LOCAL FUNCTIONS DEFINED HERE -# ---------------------------------------------------------------------- - -function Usage { - script_name=`basename $0` - echo "Usage: $script_name -i input_file -o output_file -m map_file -t template_file [-e executable-path] [-h] [-l]" - echo "" - echo "This script runs mkprocdata_map with the given arguments (-i, -o, -m and -t)," - echo "then copies some additional variables from the template file" - echo "to the output file. It also does some additional pre and" - echo "post-processing in order to create some additional variables." - echo "" - echo "Additional optional arguments:" - echo "" - echo "[-e executable-path]: Gives the path of the mkprocdata_map executable." - echo " If not specified, the executable is assumed to be" - echo " in the same directory as this script." - echo "" - echo "[-h]: Print this help message and exit" - echo "" - echo "[-l]: Rather than computing landfrac and related variables" - echo "by regridding the input file, instead copy these variables" - echo "directly from the template file. The variables this pertains" - echo "to are:" - echo $landfrac_copy_vars -} - -# This function operates on a single variable in a file, changing all -# places where that variable is missing to some new (non-missing) -# value. The _FillValue attribute remains unchanged. -# Usage: change_missing_to_value varname newval infile outfile -# - varname: the name of the variable to change -# - newval: all instances of the missing value will be replaced with -# this new value -# - infile: input file name -# - outfile: output file name (can be the same as infile) -function change_missing_to_value { - if [[ $# -ne 4 ]]; then - echo "ERROR in change_missing_to_value: wrong number of arguments: expected 2, received $#" - exit 1 - fi - - varname=$1 - newval=$2 - infile=$3 - outfile=$4 - - varname_tmp=${varname}_tmp_$$ - - cat > cmds.nco.tmp.$$ <= $pftmask_min)' $output_file $output_file" 0 - do_cmd "ncks -O -x -v pftmask_float $output_file $output_file" 0 - - # --- Calculate landmask from landfrac --- - echo "" - - cat > cmds.nco.tmp.$$ < 0); -landmask_float.change_miss($landmask_fill); -landmask=int(landmask_float); -EOF - - do_cmd "ncap2 -O -S cmds.nco.tmp.$$ $output_file $output_file" 0 - rm cmds.nco.tmp.$$ - - change_missing_to_value landmask 0 $output_file $output_file - - # in the following, note that we need to manually set missing_value, because it doesn't get changed through the .set_miss call in nco: - do_cmd "ncatted -a long_name,landmask,o,c,'land/ocean mask (0.=ocean and 1.=land)' -a missing_value,landmask,o,i,$landmask_fill $output_file" 0 -fi - -echo "Successfully regridded data" diff --git a/tools/mkprocdata_map/src/Filepath b/tools/mkprocdata_map/src/Filepath deleted file mode 100644 index 9c558e357c..0000000000 --- a/tools/mkprocdata_map/src/Filepath +++ /dev/null @@ -1 +0,0 @@ -. diff --git a/tools/mkprocdata_map/src/Makefile b/tools/mkprocdata_map/src/Makefile deleted file mode 100644 index 42f797b3c2..0000000000 --- a/tools/mkprocdata_map/src/Makefile +++ /dev/null @@ -1,10 +0,0 @@ -# Makefile for mkprocdata_map - -EXENAME = ../mkprocdata_map - -# Set optimization on by default -ifeq ($(OPT),$(null)) - OPT := TRUE -endif - -include Makefile.common diff --git a/tools/mkprocdata_map/src/Makefile.common b/tools/mkprocdata_map/src/Makefile.common deleted file mode 100644 index ab79f94144..0000000000 --- a/tools/mkprocdata_map/src/Makefile.common +++ /dev/null @@ -1,360 +0,0 @@ -#----------------------------------------------------------------------- -# This Makefile is for building clm tools on AIX, Linux (with pgf90 or -# lf95 compiler), Darwin or IRIX platforms. -# -# These macros can be changed by setting environment variables: -# -# LIB_NETCDF --- Library directory location of netcdf. (defaults to /usr/local/lib) -# INC_NETCDF --- Include directory location of netcdf. (defaults to /usr/local/include) -# MOD_NETCDF --- Module directory location of netcdf. (defaults to $LIB_NETCDF) -# USER_FC ------ Allow user to override the default Fortran compiler specified in Makefile. -# USER_FCTYP --- Allow user to override the default type of Fortran compiler (linux and USER_FC=ftn only). -# USER_CC ------ Allow user to override the default C compiler specified in Makefile (linux only). -# USER_LINKER -- Allow user to override the default linker specified in Makefile. -# USER_CPPDEFS - Additional CPP defines. -# USER_CFLAGS -- Additional C compiler flags that the user wishes to set. -# USER_FFLAGS -- Additional Fortran compiler flags that the user wishes to set. -# USER_LDLAGS -- Additional load flags that the user wishes to set. -# SMP ---------- Shared memory Multi-processing (TRUE or FALSE) [default is FALSE] -# OPT ---------- Use optimized options. -# -#------------------------------------------------------------------------ - -# Set up special characters -null := - -# Newer makes set the CURDIR variable. -CURDIR := $(shell pwd) - -RM = rm - -# Check for the netcdf library and include directories -ifeq ($(LIB_NETCDF),$(null)) - LIB_NETCDF := /usr/local/lib -endif - -ifeq ($(INC_NETCDF),$(null)) - INC_NETCDF := /usr/local/include -endif - -ifeq ($(MOD_NETCDF),$(null)) - MOD_NETCDF := $(LIB_NETCDF) -endif - -# Set user specified Fortran compiler -ifneq ($(USER_FC),$(null)) - FC := $(USER_FC) -endif - -# Set user specified C compiler -ifneq ($(USER_CC),$(null)) - CC := $(USER_CC) -endif - -# Set if Shared memory multi-processing will be used -ifeq ($(SMP),$(null)) - SMP := FALSE -endif - -CPPDEF := $(USER_CPPDEFS) - -# Set optimization on by default -ifeq ($(OPT),$(null)) - OPT := TRUE -endif - -ifeq ($(OPT),TRUE) - CPPDEF := -DOPT -endif - -# Determine platform -UNAMES := $(shell uname -s) - -# Load dependency search path. -dirs := . $(shell cat Filepath) - -# Set cpp search path, include netcdf -cpp_dirs := $(dirs) $(INC_NETCDF) $(MOD_NETCDF) -cpp_path := $(foreach dir,$(cpp_dirs),-I$(dir)) # format for command line - -# Expand any tildes in directory names. Change spaces to colons. -# (the vpath itself is set elsewhere, based on this variable) -vpath_dirs := $(foreach dir,$(cpp_dirs),$(wildcard $(dir))) -vpath_dirs := $(subst $(space),:,$(vpath_dirs)) - -#Primary Target: build the tool -all: $(EXENAME) - -# Get list of files and build dependency file for all .o files -# using perl scripts mkSrcfiles and mkDepends - -SOURCES := $(shell cat Srcfiles) - -OBJS := $(addsuffix .o, $(basename $(SOURCES))) - -# Set path to Mkdepends script; assumes that any Makefile including -# this file is in a sibling of the src directory, in which Mkdepends -# resides -Mkdepends := ../src/Mkdepends - -$(CURDIR)/Depends: $(CURDIR)/Srcfiles $(CURDIR)/Filepath - $(Mkdepends) Filepath Srcfiles > $@ - - -# Architecture-specific flags and rules -#------------------------------------------------------------------------ -# AIX -#------------------------------------------------------------------------ - -ifeq ($(UNAMES),AIX) -CPPDEF += -DAIX -cpre = $(null)-WF,-D$(null) -FPPFLAGS := $(patsubst -D%,$(cpre)%,$(CPPDEF)) -FFLAGS = -c -I$(INC_NETCDF) -q64 -qsuffix=f=f90 -qsuffix=f=f90:cpp=F90 \ - $(FPPFLAGS) -g -qfullpath -qarch=auto -qtune=auto -qsigtrap=xl__trcedump -qsclk=micro - -LDFLAGS = -L$(LIB_NETCDF) -q64 -lnetcdff -lnetcdf -ifneq ($(OPT),TRUE) - FFLAGS += -qinitauto=7FF7FFFF -qflttrap=ov:zero:inv:en -qspillsize=4000 -C -else - FFLAGS += -O2 -qmaxmem=-1 -Q - LDFLAGS += -Q -endif -CFLAGS := -q64 -g $(CPPDEF) -O2 -FFLAGS += $(cpp_path) -CFLAGS += $(cpp_path) - -ifeq ($(SMP),TRUE) - FC = xlf90_r - FFLAGS += -qsmp=omp - LDFLAGS += -qsmp=omp -else - FC = xlf90 -endif - -endif - -#------------------------------------------------------------------------ -# Darwin -#------------------------------------------------------------------------ - -ifeq ($(UNAMES),Darwin) - -# Set the default Fortran compiler -ifeq ($(USER_FC),$(null)) - FC := g95 -endif -ifeq ($(USER_CC),$(null)) - CC := gcc -endif - -CFLAGS := -g -O2 -CPPDEF += -DSYSDARWIN -DDarwin -DLINUX -LDFLAGS := - -ifeq ($(FC),g95) - - CPPDEF += -DG95 - FFLAGS := -c -fno-second-underscore $(CPPDEF) $(cpp_path) -I$(MOD_NETCDF) - ifeq ($(OPT),TRUE) - FFLAGS += -O2 - else - FFLAGS += -g -fbounds-check - endif - -endif - -ifeq ($(FC),gfortran) - - CPPDEF += -DG95 - FFLAGS := -c -fno-second-underscore $(CPPDEF) $(cpp_path) -I$(MOD_NETCDF) \ - -fno-range-check - ifeq ($(OPT),TRUE) - FFLAGS += -O2 - else - FFLAGS += -g -fbounds-check - endif - -endif - -ifeq ($(FC),ifort) - - CPPDEF += -DFORTRANUNDERSCORE - FFLAGS += -c -ftz -g -fp-model precise $(CPPDEF) $(cpp_path) \ - -convert big_endian -assume byterecl -traceback -FR - LDFLAGS += -m64 - - ifneq ($(OPT),TRUE) - FFLAGS += -CB -O0 - else - FFLAGS += -O2 - endif - ifeq ($(SMP),TRUE) - FFLAGS += -qopenmp - LDFLAGS += -qopenmp - endif -endif - -ifeq ($(FC),pgf90) - - CPPDEF += -DFORTRANUNDERSCORE - FFLAGS += -c $(CPPDEF) $(cpp_path) - ifneq ($(OPT),TRUE) - FFLAGS += -g -Ktrap=fp -Mbounds -Kieee - else - FFLAGS += -fast -Kieee - endif - - ifeq ($(SMP),TRUE) - FFLAGS += -mp - LDFLAGS += -mp - endif - -endif - -ifeq ($(CC),icc) - CFLAGS += -m64 -g - ifeq ($(SMP),TRUE) - CFLAGS += -qopenmp - endif -endif -ifeq ($(CC),pgcc) - CFLAGS += -g -fast -endif - -CFLAGS += $(CPPDEF) $(cpp_path) -LDFLAGS += -L$(LIB_NETCDF) -lnetcdf -lnetcdff - -endif - -#------------------------------------------------------------------------ -# Linux -#------------------------------------------------------------------------ - -ifeq ($(UNAMES),Linux) - ifeq ($(USER_FC),$(null)) - FC := ifort - FCTYP := ifort - else - ifeq ($(USER_FC),ftn) - ifneq ($(USER_FCTYP),$(null)) - FCTYP := $(USER_FCTYP) - else - FCTYP := pgf90 - endif - else - FCTYP := $(USER_FC) - endif - endif - CPPDEF += -DLINUX -DFORTRANUNDERSCORE - CFLAGS := $(CPPDEF) - LDFLAGS := $(shell $(LIB_NETCDF)/../bin/nf-config --flibs) - FFLAGS = - - ifeq ($(FCTYP),pgf90) - CC := pgcc - ifneq ($(OPT),TRUE) - FFLAGS += -g -Ktrap=fp -Mbounds -Kieee - else - FFLAGS += -fast -Kieee - CFLAGS += -fast - endif - - ifeq ($(SMP),TRUE) - FFLAGS += -mp - LDFLAGS += -mp - endif - - endif - - ifeq ($(FCTYP),lf95) - ifneq ($(OPT),TRUE) - FFLAGS += -g --chk a,e,s,u -O0 - else - FFLAGS += -O - endif - # Threading only works by putting thread memory on the heap rather than the stack - # (--threadheap). - # As of lf95 version 6.2 the thread stacksize limits are (still) too small to run - # even small - # resolution problems (FV at 10x15 res fails). - ifeq ($(SMP),TRUE) - FFLAGS += --openmp --threadheap 4096 - LDFLAGS += --openmp --threadheap 4096 - endif - endif - ifeq ($(FCTYP),pathf90) - FFLAGS += -extend_source -ftpp -fno-second-underscore - ifneq ($(OPT),TRUE) - FFLAGS += -g -O0 - else - FFLAGS += -O - endif - ifeq ($(SMP),TRUE) - FFLAGS += -mp - LDFLAGS += -mp - endif - endif - ifeq ($(FCTYP),ifort) - - FFLAGS += -ftz -g -fp-model precise -convert big_endian -assume byterecl -traceback -FR - CFLAGS += -m64 -g - LDFLAGS += -m64 - - ifneq ($(OPT),TRUE) - FFLAGS += -CB -O0 - else - FFLAGS += -O2 - endif - ifeq ($(SMP),TRUE) - FFLAGS += -qopenmp - CFLAGS += -qopenmp - LDFLAGS += -qopenmp - endif - endif - FFLAGS += -c -I$(INC_NETCDF) $(CPPDEF) $(cpp_path) - CFLAGS += $(cpp_path) -endif - -#------------------------------------------------------------------------ -# Default rules and macros -#------------------------------------------------------------------------ - -.SUFFIXES: -.SUFFIXES: .F90 .c .o - -# Set the vpath for all file types EXCEPT .o -# We do this for individual file types rather than generally using -# VPATH, because for .o files, we don't want to use files from a -# different build (e.g., in building the unit tester, we don't want to -# use .o files from the main build) -vpath %.F90 $(vpath_dirs) -vpath %.c $(vpath_dirs) -vpath %.h $(vpath_dirs) - -# Append user defined compiler and load flags to Makefile defaults -CFLAGS += $(USER_CFLAGS) -FFLAGS += $(USER_FFLAGS) -LDFLAGS += $(USER_LDFLAGS) - -# Set user specified linker -ifneq ($(USER_LINKER),$(null)) - LINKER := $(USER_LINKER) -else - LINKER := $(FC) -endif - -.F90.o: - $(FC) $(FFLAGS) $< - -.c.o: - $(CC) -c $(CFLAGS) $< - - -$(EXENAME): $(OBJS) - $(LINKER) -o $@ $(OBJS) $(LDFLAGS) - -clean: - $(RM) -f $(OBJS) *.mod Depends - -include $(CURDIR)/Depends diff --git a/tools/mkprocdata_map/src/Mkdepends b/tools/mkprocdata_map/src/Mkdepends deleted file mode 100755 index a75e8fdde0..0000000000 --- a/tools/mkprocdata_map/src/Mkdepends +++ /dev/null @@ -1,327 +0,0 @@ -#!/usr/bin/env perl - -# Generate dependencies in a form suitable for inclusion into a Makefile. -# The source filenames are provided in a file, one per line. Directories -# to be searched for the source files and for their dependencies are provided -# in another file, one per line. Output is written to STDOUT. -# -# For CPP type dependencies (lines beginning with #include) the dependency -# search is recursive. Only dependencies that are found in the specified -# directories are included. So, for example, the standard include file -# stdio.h would not be included as a dependency unless /usr/include were -# one of the specified directories to be searched. -# -# For Fortran module USE dependencies (lines beginning with a case -# insensitive "USE", possibly preceded by whitespace) the Fortran compiler -# must be able to access the .mod file associated with the .o file that -# contains the module. In order to correctly generate these dependencies -# two restrictions must be observed. -# 1) All modules must be contained in files that have the same base name as -# the module, in a case insensitive sense. This restriction implies that -# there can only be one module per file. -# 2) All modules that are to be contained in the dependency list must be -# contained in one of the source files in the list provided on the command -# line. -# The reason for the second restriction is that since the makefile doesn't -# contain rules to build .mod files the dependency takes the form of the .o -# file that contains the module. If a module is being used for which the -# source code is not available (e.g., a module from a library), then adding -# a .o dependency for that module is a mistake because make will attempt to -# build that .o file, and will fail if the source code is not available. -# -# Author: B. Eaton -# Climate Modelling Section, NCAR -# Feb 2001 - -use Getopt::Std; -use File::Basename; - -# Check for usage request. -@ARGV >= 2 or usage(); - -# Process command line. -my %opt = (); -getopts( "t:w", \%opt ) or usage(); -my $filepath_arg = shift() or usage(); -my $srcfile_arg = shift() or usage(); -@ARGV == 0 or usage(); # Check that all args were processed. - -my $obj_dir; -if ( defined $opt{'t'} ) { $obj_dir = $opt{'t'}; } - -open(FILEPATH, $filepath_arg) or die "Can't open $filepath_arg: $!\n"; -open(SRCFILES, $srcfile_arg) or die "Can't open $srcfile_arg: $!\n"; - -# Make list of paths to use when looking for files. -# Prepend "." so search starts in current directory. This default is for -# consistency with the way GNU Make searches for dependencies. -my @file_paths = ; -close(FILEPATH); -chomp @file_paths; -unshift(@file_paths,'.'); -foreach $dir (@file_paths) { # (could check that directories exist here) - $dir =~ s!/?\s*$!!; # remove / and any whitespace at end of directory name - ($dir) = glob $dir; # Expand tildes in path names. -} - -# Make list of files containing source code. -my @src = ; -close(SRCFILES); -chomp @src; - -# For each file that may contain a Fortran module (*.[fF]90 *.[fF]) convert the -# file's basename to uppercase and use it as a hash key whose value is the file's -# basename. This allows fast identification of the files that contain modules. -# The only restriction is that the file's basename and the module name must match -# in a case insensitive way. -my %module_files = (); -my ($f, $name, $path, $suffix, $mod); -my @suffixes = ('\.[fF]90', '\.[fF]' ); -foreach $f (@src) { - ($name, $path, $suffix) = fileparse($f, @suffixes); - ($mod = $name) =~ tr/a-z/A-Z/; - $module_files{$mod} = $name; -} - -# Now make a list of .mod files in the file_paths. If a .o source dependency -# can't be found based on the module_files list above, then maybe a .mod -# module dependency can if the mod file is visible. -my %trumod_files = (); -my ($dir); -my ($f, $name, $path, $suffix, $mod); -my @suffixes = ('\.mod' ); -foreach $dir (@file_paths) { - @filenames = (glob("$dir/*.mod")); - foreach $f (@filenames) { - ($name, $path, $suffix) = fileparse($f, @suffixes); - ($mod = $name) =~ tr/a-z/A-Z/; - $trumod_files{$mod} = $name; - } -} - -#print STDERR "\%module_files\n"; -#while ( ($k,$v) = each %module_files ) { -# print STDERR "$k => $v\n"; -#} - -# Find module and include dependencies of the source files. -my ($file_path, $rmods, $rincs); -my %file_modules = (); -my %file_includes = (); -my @check_includes = (); -foreach $f ( @src ) { - - # Find the file in the seach path (@file_paths). - unless ($file_path = find_file($f)) { - if (defined $opt{'w'}) {print STDERR "$f not found\n";} - next; - } - - # Find the module and include dependencies. - ($rmods, $rincs) = find_dependencies( $file_path ); - - # Remove redundancies (a file can contain multiple procedures that have - # the same dependencies). - $file_modules{$f} = rm_duplicates($rmods); - $file_includes{$f} = rm_duplicates($rincs); - - # Make a list of all include files. - push @check_includes, @{$file_includes{$f}}; -} - -#print STDERR "\%file_modules\n"; -#while ( ($k,$v) = each %file_modules ) { -# print STDERR "$k => @$v\n"; -#} -#print STDERR "\%file_includes\n"; -#while ( ($k,$v) = each %file_includes ) { -# print STDERR "$k => @$v\n"; -#} -#print STDERR "\@check_includes\n"; -#print STDERR "@check_includes\n"; - -# Find include file dependencies. -my %include_depends = (); -while (@check_includes) { - $f = shift @check_includes; - if (defined($include_depends{$f})) { next; } - - # Mark files not in path so they can be removed from the dependency list. - unless ($file_path = find_file($f)) { - $include_depends{$f} = -1; - next; - } - - # Find include file dependencies. - ($rmods, $include_depends{$f}) = find_dependencies($file_path); - - # Add included include files to the back of the check_includes list so - # that their dependencies can be found. - push @check_includes, @{$include_depends{$f}}; - - # Add included modules to the include_depends list. - if ( @$rmods ) { push @{$include_depends{$f}}, @$rmods; } -} - -#print STDERR "\%include_depends\n"; -#while ( ($k,$v) = each %include_depends ) { -# print STDERR (ref $v ? "$k => @$v\n" : "$k => $v\n"); -#} - -# Remove include file dependencies that are not in the Filepath. -my $i, $ii; -foreach $f (keys %include_depends) { - - unless (ref $include_depends{$f}) { next; } - $rincs = $include_depends{$f}; - unless (@$rincs) { next; } - $ii = 0; - $num_incs = @$rincs; - for ($i = 0; $i < $num_incs; ++$i) { - if ($include_depends{$$rincs[$ii]} == -1) { - splice @$rincs, $ii, 1; - next; - } - ++$ii; - } -} - -# Substitute the include file dependencies into the %file_includes lists. -foreach $f (keys %file_includes) { - my @expand_incs = (); - - # Initialize the expanded %file_includes list. - my $i; - unless (@{$file_includes{$f}}) { next; } - foreach $i (@{$file_includes{$f}}) { - push @expand_incs, $i unless ($include_depends{$i} == -1); - } - unless (@expand_incs) { - $file_includes{$f} = []; - next; - } - - # Expand - for ($i = 0; $i <= $#expand_incs; ++$i) { - push @expand_incs, @{ $include_depends{$expand_incs[$i]} }; - } - - $file_includes{$f} = rm_duplicates(\@expand_incs); -} - -#print STDERR "expanded \%file_includes\n"; -#while ( ($k,$v) = each %file_includes ) { -# print STDERR "$k => @$v\n"; -#} - -# Print dependencies to STDOUT. -foreach $f (sort keys %file_modules) { - $f =~ /(.+)\./; - $target = "$1.o"; - if ( defined $opt{'t'} ) { $target = "$opt{'t'}/$1.o"; } - print "$target : $f @{$file_modules{$f}} @{$file_includes{$f}}\n"; -} - -#-------------------------------------------------------------------------------------- - -sub find_dependencies { - - # Find dependencies of input file. - # Use'd Fortran 90 modules are returned in \@mods. - # Files that are "#include"d by the cpp preprocessor are returned in \@incs. - - my( $file ) = @_; - my( @mods, @incs ); - - open(FH, $file) or die "Can't open $file: $!\n"; - - while ( ) { - # Search for "#include" and strip filename when found. - if ( /^#include\s+[<"](.*)[>"]/ ) { - push @incs, $1; - } - # Search for Fortran include dependencies. - elsif ( /^\s*include\s+['"](.*)['"]/ ) { #" for emacs fontlock - push @incs, $1; - } - # Search for module dependencies. - elsif ( /^\s*USE\s+(\w+)/i ) { - ($module = $1) =~ tr/a-z/A-Z/; - # Return dependency in the form of a .o version of the file that contains - # the module. this is from the source list. - if ( defined $module_files{$module} ) { - if ( defined $obj_dir ) { - push @mods, "$obj_dir/$module_files{$module}.o"; - } else { - push @mods, "$module_files{$module}.o"; - } - } - # Return dependency in the form of a .mod version of the file that contains - # the module. this is from the .mod list. only if .o version not found - elsif ( defined $trumod_files{$module} ) { - if ( defined $obj_dir ) { - push @mods, "$obj_dir/$trumod_files{$module}.mod"; - } else { - push @mods, "$trumod_files{$module}.mod"; - } - } - } - } - close( FH ); - return (\@mods, \@incs); -} - -#-------------------------------------------------------------------------------------- - -sub find_file { - -# Search for the specified file in the list of directories in the global -# array @file_paths. Return the first occurance found, or the null string if -# the file is not found. - - my($file) = @_; - my($dir, $fname); - - foreach $dir (@file_paths) { - $fname = "$dir/$file"; - if ( -f $fname ) { return $fname; } - } - return ''; # file not found -} - -#-------------------------------------------------------------------------------------- - -sub rm_duplicates { - -# Return a list with duplicates removed. - - my ($in) = @_; # input arrary reference - my @out = (); - my $i; - my %h = (); - foreach $i (@$in) { - $h{$i} = ''; - } - @out = keys %h; - return \@out; -} - -#-------------------------------------------------------------------------------------- - -sub usage { - ($ProgName = $0) =~ s!.*/!!; # name of program - die < shr_kind_r8 - implicit none - save - - real(R8),parameter :: SHR_CONST_REARTH = 6.37122e6_R8 ! radius of earth ~ m - real(r8),parameter :: re_km = SHR_CONST_REARTH*0.001 ! radius of earth (km) - -end module constMod diff --git a/tools/mkprocdata_map/src/fileutils.F90 b/tools/mkprocdata_map/src/fileutils.F90 deleted file mode 100644 index e1f8e633da..0000000000 --- a/tools/mkprocdata_map/src/fileutils.F90 +++ /dev/null @@ -1,282 +0,0 @@ -module fileutils - -!----------------------------------------------------------------------- -!BOP -! -! !MODULE: fileutils -! -! !DESCRIPTION: -! Module containing file I/O utilities -! -! !USES: -! -! !PUBLIC TYPES: - implicit none - save -! -! !PUBLIC MEMBER FUNCTIONS: - public :: get_filename !Returns filename given full pathname - public :: opnfil !Open local unformatted or formatted file - public :: getfil !Obtain local copy of file - public :: relavu !Close and release Fortran unit no longer in use - public :: getavu !Get next available Fortran unit number -! -! !REVISION HISTORY: -! Created by Mariana Vertenstein -! -! -! !PRIVATE MEMBER FUNCTIONS: None -!EOP -!----------------------------------------------------------------------- - -contains - -!----------------------------------------------------------------------- -!BOP -! -! !IROUTINE: get_filename -! -! !INTERFACE: - character(len=256) function get_filename (fulpath) -! -! !DESCRIPTION: -! Returns filename given full pathname -! -! !ARGUMENTS: - implicit none - character(len=*), intent(in) :: fulpath !full pathname -! -! !REVISION HISTORY: -! Created by Mariana Vertenstein -! -! -! !LOCAL VARIABLES: -!EOP - integer i !loop index - integer klen !length of fulpath character string -!------------------------------------------------------------------------ - - klen = len_trim(fulpath) - do i = klen, 1, -1 - if (fulpath(i:i) == '/') go to 10 - end do - i = 0 -10 get_filename = fulpath(i+1:klen) - - end function get_filename - -!------------------------------------------------------------------------ -!BOP -! -! !IROUTINE: set_filename -! -! !INTERFACE: - character(len=256) function set_filename (rem_dir, loc_fn) -! -! !DESCRIPTION: -! -! !ARGUMENTS: -! - implicit none - character(len=*), intent(in) :: rem_dir !remote directory - character(len=*), intent(in) :: loc_fn !local full path filename -! -! !REVISION HISTORY: -! Created by Mariana Vertenstein -! -! -! !LOCAL VARIABLES: -!EOP - integer :: i !integer -!------------------------------------------------------------------------ - - set_filename = ' ' - do i = len_trim(loc_fn), 1, -1 - if (loc_fn(i:i)=='/') go to 10 - end do - i = 0 -10 set_filename = trim(rem_dir) // loc_fn(i+1:len_trim(loc_fn)) - - end function set_filename - -!------------------------------------------------------------------------ -!BOP -! -! !IROUTINE: getfil -! -! !INTERFACE: - subroutine getfil (fulpath, locfn, iflag) -! -! !DESCRIPTION: -! Obtain local copy of file -! First check current working directory -! Next check full pathname[fulpath] on disk -! Finally check full pathname[fulpath] on archival system -! -! !USES: -! -! !ARGUMENTS: - implicit none - character(len=*), intent(in) :: fulpath !Archival or permanent disk full pathname - character(len=*), intent(out) :: locfn !output local file name - integer, optional, intent(in) :: iflag !0=>abort if file not found 1=>do not abort -! -! !REVISION HISTORY: -! Created by Mariana Vertenstein -! -! -! !LOCAL VARIABLES: -!EOP - integer i !loop index - integer klen !length of fulpath character string - integer ierr !error status - logical lexist !true if local file exists - character(len=len(fulpath)+5) :: fulpath2 !Archival full pathname -!------------------------------------------------------------------------ - - ! get local file name from full name: start at end. look for first "/" - - klen = len_trim(fulpath) - do i = klen, 1, -1 - if (fulpath(i:i).eq.'/') go to 100 - end do - i = 0 -100 locfn = fulpath(i+1:klen) - if (len_trim(locfn) == 0) then - write(6,*)'(GETFIL): local filename has zero length' - stop 1 - else - write(6,*)'(GETFIL): attempting to find local file ',trim(locfn) - endif - - ! first check if file is in current working directory. - - inquire (file=locfn,exist=lexist) - if (lexist) then - write(6,*) '(GETFIL): using ',trim(locfn),' in current working directory' - RETURN - endif - - ! second check for full pathname on disk - - inquire(file=fulpath, exist=lexist) - if (lexist) then - locfn = trim(fulpath) - write(6,*) '(GETFIL): using ',trim(fulpath) - RETURN - else - write(6,*) 'GETFIL: FAILED to get '//trim(fulpath) - stop 1 - end if - - end subroutine getfil - -!------------------------------------------------------------------------ -!BOP -! -! !IROUTINE: opnfil -! -! !INTERFACE: - subroutine opnfil (locfn, iun, form) -! -! !DESCRIPTION: -! Open file locfn in unformatted or formatted form on unit iun -! -! !ARGUMENTS: -! - implicit none - character(len=*), intent(in):: locfn !file name - integer, intent(in):: iun !fortran unit number - character(len=1), intent(in):: form !file format: u = unformatted, - !f = formatted -! -! !REVISION HISTORY: -! Created by Mariana Vertenstein -! -! -! !LOCAL VARIABLES: -!EOP - integer ioe !error return from fortran open - character(len=11) ft !format type: formatted. unformatted -!------------------------------------------------------------------------ - - if (len_trim(locfn) == 0) then - write(6,*)'OPNFIL: local filename has zero length' - stop 1 - endif - if (form=='u' .or. form=='U') then - ft = 'unformatted' - else - ft = 'formatted ' - end if - open (unit=iun,file=locfn,status='unknown',form=ft,iostat=ioe) - if (ioe /= 0) then - write(6,*)'(OPNFIL): failed to open file ',trim(locfn), & - & ' on unit ',iun,' ierr=',ioe - stop 1 - else - write(6,*)'(OPNFIL): Successfully opened file ',trim(locfn),' on unit= ',iun - end if - - end subroutine opnfil - -!------------------------------------------------------------------------ -!BOP -! -! !IROUTINE: getavu -! -! !INTERFACE: - integer function getavu() -! -! !DESCRIPTION: -! Get next available Fortran unit number. -! -! !USES: - use shr_file_mod, only : shr_file_getUnit -! -! !ARGUMENTS: - implicit none -! -! !REVISION HISTORY: -! Created by Gordon Bonan -! Modified for clm2 by Mariana Vertenstein -! -! -! !LOCAL VARIABLES: -!EOP -!------------------------------------------------------------------------ - - getavu = shr_file_getunit() - - end function getavu - -!------------------------------------------------------------------------ -!BOP -! -! !IROUTINE: relavu -! -! !INTERFACE: - subroutine relavu (iunit) -! -! !DESCRIPTION: -! Close and release Fortran unit no longer in use! -! -! !USES: - use shr_file_mod, only : shr_file_freeUnit -! -! !ARGUMENTS: - implicit none - integer, intent(in) :: iunit !Fortran unit number -! -! !REVISION HISTORY: -! Created by Gordon Bonan -! -!EOP -!------------------------------------------------------------------------ - - close(iunit) - call shr_file_freeUnit(iunit) - - end subroutine relavu - -end module fileutils diff --git a/tools/mkprocdata_map/src/fmain.F90 b/tools/mkprocdata_map/src/fmain.F90 deleted file mode 100644 index ba9e593c1d..0000000000 --- a/tools/mkprocdata_map/src/fmain.F90 +++ /dev/null @@ -1,78 +0,0 @@ -program fmain - - use mkprocdata_map, only : mkmap - implicit none - - character(len= 256) :: arg - integer :: n !index - integer :: nargs !number of arguments - integer, external :: iargc !number of arguments function - character(len=256) :: filei !input file - character(len=256) :: fileo !output mapped file - character(len=256) :: fmap !maping file - character(len=256) :: ftemplate !template file, containing lat & lon arrays desired in output file - character(len=256) :: cmdline !input command line - integer, parameter :: inival = -999 !initial value for command-line integers - !---------------------------------------------------- - - filei = ' ' - fileo = ' ' - fmap = ' ' - ftemplate = ' ' - - cmdline = 'mkprocdata_map' - nargs = iargc() - n = 1 - do while (n <= nargs) - arg = ' ' - call getarg (n, arg) - n = n + 1 - - select case (arg) - case ('-i') - call getarg (n, arg) - n = n + 1 - filei = trim(arg) - cmdline = trim(cmdline) // ' -i ' // trim(arg) - case ('-o') - call getarg (n, arg) - n = n + 1 - fileo = trim(arg) - cmdline = trim(cmdline) // ' -o ' // trim(arg) - case ('-m') - call getarg (n, arg) - n = n + 1 - fmap = trim(arg) - cmdline = trim(cmdline) // ' -m ' // trim(arg) - case ('-t') - call getarg (n, arg) - n = n + 1 - ftemplate = trim(arg) - cmdline = trim(cmdline) // ' -t ' // trim(arg) - case default - write (6,*) 'Argument ', arg,' is not known' - call usage_exit (' ') - cmdline = trim(cmdline) // ' ' // trim(arg) - end select - end do - - if (filei == ' ' .or. fileo == ' ' .or. fmap == ' ' & - .or. ftemplate == ' ') then - call usage_exit ('Must specify all the following arguments') - end if - - call mkmap (filei, fileo, fmap, ftemplate) - -end program fmain - - -subroutine usage_exit (arg) - implicit none - character(len=*) :: arg - if (arg /= ' ') write (6,*) arg - write (6,*) 'Usage: mkprocdata_map -i -o -m -t