Skip to content
This repository has been archived by the owner on Oct 23, 2020. It is now read-only.

How to setup an MPAS core

Jump to bottom
Douglas Jacobsen edited this page Feb 28, 2014 · 1 revision

This document should help developer learn the requirements of an MPAS core. After reading this document, a developer should be able to create a barebones MPAS core.

Requirements for MPAS cores can be split into several sections. These are:

  • Registry Requirements
  • Module Requirements
  • Makefile Requirements

These requirements will be described in the sections below.

Registry Requirements

Within the Registry.xml file for each core, some dimensions/variables/streams/namelist options are expected and/or required to be defined.

Dimension Requirements

The MPAS framework requires several variables to be defined. These are only dimensions that required variables make use of, but they are explicitly referenced in shared parts of code, so they must always be defined. Below is an example from a Registry.xml file defining the required dimensions.

<dims>
	<dim name="nCells" units="unitless"
		 description="The number of polygons in the primary grid."
	/>
	<dim name="nEdges" units="unitless"
		 description="The number of edge midpoints in either the primary or dual grid."
	/>
	<dim name="maxEdges" units="unitless"
		 description="The largest number of edges any polygon within the grid has."
	/>
	<dim name="maxEdges2" units="unitless"
		 description="Two times the largest number of edges any polygon within the grid has."
	/>
	<dim name="nVertices" units="unitless"
		 description="The total number of cells in the dual grid. Also the number of corners in the primary grid."
	/>
	<dim name="vertexDegree" units="unitless"
		 description="The number of cells or edges touching each vertex."
	/>
</dims>

Variable Requirements

The MPAS framework requires certain variables to be defined. These only include general variables that related to our mesh specification. In addition to these variables being defined, they must be contained within a var_struct named mesh. Below is a bare-bones example of a mesh var_struct from a Registry.xml file.

<var_struct name="mesh" time_levs="0">
	<var name="latCell" type="real" dimensions="nCells" units="radians"
		 description="Latitude location of cell centers in radians."
	/>
	<var name="lonCell" type="real" dimensions="nCells" units="radians"
		 description="Longitude location of cell centers in radians."
	/>
	<var name="xCell" type="real" dimensions="nCells" units="unitless"
		 description="X Coordinate in cartesian space of cell centers."
	/>
	<var name="yCell" type="real" dimensions="nCells" units="unitless"
		 description="Y Coordinate in cartesian space of cell centers."
	/>
	<var name="zCell" type="real" dimensions="nCells" units="unitless"
		 description="Z Coordinate in cartesian space of cell centers."
	/>
	<var name="indexToCellID" type="integer" dimensions="nCells" units="unitless"
		 description="List of global cell IDs."
	/>
	<var name="latEdge" type="real" dimensions="nEdges" units="radians"
		 description="Latitude location of edge midpoints in radians."
	/>
	<var name="lonEdge" type="real" dimensions="nEdges" units="radians"
		 description="Longitude location of edge midpoints in radians."
	/>
	<var name="xEdge" type="real" dimensions="nEdges" units="unitless"
		 description="X Coordinate in cartesian space of edge midpoints."
	/>
	<var name="yEdge" type="real" dimensions="nEdges" units="unitless"
		 description="Y Coordinate in cartesian space of edge midpoints."
	/>
	<var name="zEdge" type="real" dimensions="nEdges" units="unitless"
		 description="Z Coordinate in cartesian space of edge midpoints."
	/>
	<var name="indexToEdgeID" type="integer" dimensions="nEdges" units="unitless"
		 description="List of global edge IDs."
	/>
	<var name="latVertex" type="real" dimensions="nVertices" units="radians"
		 description="Latitude location of vertices in radians."
	/>
	<var name="lonVertex" type="real" dimensions="nVertices" units="radians"
		 description="Longitude location of vertices in radians."
	/>
	<var name="xVertex" type="real" dimensions="nVertices" units="unitless"
		 description="X Coordinate in cartesian space of vertices."
	/>
	<var name="yVertex" type="real" dimensions="nVertices" units="unitless"
		 description="Y Coordinate in cartesian space of vertices."
	/>
	<var name="zVertex" type="real" dimensions="nVertices" units="unitless"
		 description="Z Coordinate in cartesian space of vertices."
	/>
	<var name="indexToVertexID" type="integer" dimensions="nVertices" units="unitless"
		 description="List of global vertex IDs."
	/>
	<var name="meshDensity" type="real" dimensions="nCells" units="unitless"
		 description="Value of density function used to generate a particular mesh at cell centers."
	/>
	<var name="cellsOnEdge" type="integer" dimensions="TWO nEdges" units="unitless"
		 description="List of cells that straddle each edge."
	/>
	<var name="nEdgesOnCell" type="integer" dimensions="nCells" units="unitless"
		 description="Number of edges that border each cell."
	/>
	<var name="nEdgesOnEdge" type="integer" dimensions="nEdges" units="unitless"
		 description="Number of edges that surround each of the cells that straddle each edge. These edges are used to reconstruct the tangential velocities."
	/>
	<var name="edgesOnCell" type="integer" dimensions="maxEdges nCells" units="unitless"
		 description="List of edges that border each cell."
	/>
	<var name="edgesOnEdge" type="integer" dimensions="maxEdges2 nEdges" units="unitless"
		 description="List of edges that border each of the cells that straddle each edge."
	/>
	<var name="weightsOnEdge" type="real" dimensions="maxEdges2 nEdges" units="unitless"
		 description="Reconstruction weights associated with each of the edgesOnEdge."
	/>
	<var name="dvEdge" type="real" dimensions="nEdges" units="m"
		 description="Length of each edge, computed as the distance between verticesOnEdge."
	/>
	<var name="dcEdge" type="real" dimensions="nEdges" units="m"
		 description="Length of each edge, computed as the distance between cellsOnEdge."
	/>
	<var name="angleEdge" type="real" dimensions="nEdges" units="radians"
		 description="Angle the edge normal makes with local eastward direction."
	/>
	<var name="areaCell" type="real" dimensions="nCells" units="m^2"
		 description="Area of each cell in the primary grid."
	/>
	<var name="areaTriangle" type="real" dimensions="nVertices" units="m^2"
		 description="Area of each cell (triangle) in the dual grid."
	/>
	<var name="edgeNormalVectors" type="real" dimensions="R3 nEdges" units="unitless"
		 description="Normal unit vector defined at an edge."
	/>
	<var name="cellsOnCell" type="integer" dimensions="maxEdges nCells" units="unitless"
		 description="List of cells that neighbor each cell."
	/>
	<var name="verticesOnCell" type="integer" dimensions="maxEdges nCells" units="unitless"
		 description="List of vertices that border each cell."
	/>
	<var name="verticesOnEdge" type="integer" dimensions="TWO nEdges" units="unitless"
		 description="List of vertices that straddle each edge."
	/>
	<var name="edgesOnVertex" type="integer" dimensions="vertexDegree nVertices" units="unitless"
		 description="List of edges that share a vertex as an endpoint."
	/>
	<var name="cellsOnVertex" type="integer" dimensions="vertexDegree nVertices" units="unitless"
		 description="List of cells that share a vertex."
	/>
	<var name="kiteAreasOnVertex" type="real" dimensions="vertexDegree nVertices" units="m^2"
		 description="Area of the portions of each dual cell that are part of each cellsOnVertex."
	/>
</var_struct>

Namelist Requirements

The MPAS framework expects a core to have several namelist options defined. The MPAS framework does not defined the namelist records these options should exist in, so cores are allowed to place these in namelist records how ever they would like. Expected namelist options are:

config_calendar_type
config_input_name
config_output_name
config_restart_name
config_restart_timestamp_name
config_frames_per_outfile
config_pio_num_iotasks
config_pio_stride
config_num_halos
config_block_decomp_file_prefix
config_number_of_blocks
config_explicit_proc_decomp
config_proc_decomp_file_prefix

Below is an example snippet from a Registry.xml that shows how these variables should be defined. A core is free to modify any attributes other than name and type for these namelist options.

<nml_option name="config_calendar_type" type="character" default_value="360day" units="unitless"
			description="Selection of the type of calendar that should be used in the simulation."
			possible_values="'gregorian', 'gregorian_noleap', or '360day'"
/>
<nml_option name="config_input_name" type="character" default_value="grid.nc" units="unitless"
			description="The path to the input file for the simulation."
			possible_values="path/to/grid.nc"
/>
<nml_option name="config_output_name" type="character" default_value="output.nc" units="unitless"
			description="The template path and name to the output file from the simulation. A time stamp is prepended to the extension of the file (.nc)."
			possible_values="path/to/output.nc"
/>
<nml_option name="config_restart_name" type="character" default_value="restart.nc" units="unitless"
			description="The template path and name to the restart file for the simulation. A time stamp is prepended to the extension of the file (.nc) both for input and output."
			possible_values="path/to/restart.nc"
/>
<nml_option name="config_restart_timestamp_name" type="character" default_value="restart_timestamp" units="unitless"
			description="The name of the file timestamps for latest restart files are written to. This file is also read when config_start_time is set to 'file' and config_do_restart is set to .true."
			possible_values="path/to/restart_timestamp"
/>
<nml_option name="config_frames_per_outfile" type="integer" default_value="1000" units="unitless"
			description="Integer specifying how many time frames should be included in an output file. Once the maximum is reached, a new output file is created."
			possible_values="Any positive integer value greater than 0"
/>
<nml_option name="config_pio_num_iotasks" type="integer" default_value="0" units="unitless"
			description="Integer specifying how many IO tasks should be used within the PIO library. A value of 0 causes all MPI tasks to also be IO tasks. IO tasks are requried to write contiguous blocks of data to a file."
			possible_values="Any positive integer value greater than or equal to 0."
/>
<nml_option name="config_pio_stride" type="integer" default_value="1" units="unitless"
			description="Integer specifying the stride of each IO task."
			possible_values="Any positive integer value greater than 0."
/>
<nml_option name="config_num_halos" type="integer" default_value="3" units="unitless"
			description="Determines the number of halo cells extending from a blocks owned cells (Called the 0-Halo). The default of 3 is the minimum that can be used with monotonic advection."
			possible_values="Any positive interger value."
/>
<nml_option name="config_block_decomp_file_prefix" type="character" default_value="graph.info.part." units="unitless"
			description="Defines the prefix for the block decomposition file. Can include a path. The number of blocks is appended to the end of the prefix at run-time."
			possible_values="Any path/prefix to a block decomposition file."
/>
<nml_option name="config_number_of_blocks" type="integer" default_value="0" units="unitless"
			description="Determines the number of blocks a simulation should be run with. If it is set to 0, the number of blocks is the same as the number of MPI tasks at run-time."
			possible_values="Any integer $>=$ 0."
/>
<nml_option name="config_explicit_proc_decomp" type="logical" default_value=".false." units="unitless"
			description="Determines if an explicit processor decomposition should be used. This is only useful if multiple blocks per processor are used."
			possible_values=".true. or .false."
/>
<nml_option name="config_proc_decomp_file_prefix" type="character" default_value="graph.info.part." units="unitless"
			description="Defines the prefix for the processor decomposition file. This file is only read if config_explicit_proc_decomp is .true. The number of processors is appended to the end of the prefix at run-time."
			possible_values="Any path/prefix to a processor decomposition file."
/>

Stream Requirements

In order to fill out the required mesh var_struct variables, a stream should be defined that contains at a minimum all variable listed in the required variables section. Below is an example stream definition with these variables.

<streams>
	<stream name="input" type="input">
		<var name="latCell"/>
		<var name="lonCell"/>
		<var name="xCell"/>
		<var name="yCell"/>
		<var name="zCell"/>
		<var name="indexToCellID"/>
		<var name="latEdge"/>
		<var name="lonEdge"/>
		<var name="xEdge"/>
		<var name="yEdge"/>
		<var name="zEdge"/>
		<var name="indexToEdgeID"/>
		<var name="latVertex"/>
		<var name="lonVertex"/>
		<var name="xVertex"/>
		<var name="yVertex"/>
		<var name="zVertex"/>
		<var name="indexToVertexID"/>
		<var name="meshDensity"/>
		<var name="cellsOnEdge"/>
		<var name="nEdgesOnCell"/>
		<var name="nEdgesOnEdge"/>
		<var name="edgesOnCell"/>
		<var name="edgesOnEdge"/>
		<var name="weightsOnEdge"/>
		<var name="dvEdge"/>
		<var name="dcEdge"/>
		<var name="angleEdge"/>
		<var name="areaCell"/>
		<var name="areaTriangle"/>
		<var name="cellsOnCell"/>
		<var name="verticesOnCell"/>
		<var name="verticesOnEdge"/>
		<var name="edgesOnVertex"/>
		<var name="cellsOnVertex"/>
		<var name="kiteAreasOnVertex"/>
	</stream>
</streams>

Module Requirements

Each core is required to define a module named mpas_core. This core is required to have four subroutines, each with different purposes. The four required subroutines are:

  • mpas_core_setup_packages
  • mpas_core_init
  • mpas_core_run
  • mpas_core_finalize

Each of these will be described below.

mpas_core_setup_packages subroutine

This subroutine is called during the initialization process after the namelist file is read in, but before any of the data strucutres are allocated. It allows developer to write arbitrary logic depending on valuse of namelist options to control if packages are active or not.

Then, when variables and data structures are allocated, they are checked against the packages.

mpas_core_init subroutine

This subroutine is called during the initialization process after the input file is read and data structures are allocated. It allows a core to perform "one time" setup of variables they might need for the remainder of the simulation.

mpas_core_run subroutine

This subroutine is called after initialization is complete. It is intended to be the main time stepping subroutine for a core. Each core is expected to handle their own I/O alarms and perform their own time stepping.

mpas_core_finalize subroutine

This subroutine is called after a run is complte. It is intended to allow a core to clean up any variables that might have been setup or allocated during the course of a simulation.

Makefile Requirements

MPAS has a few requirements on Makefiles for each core. They are not very restrictive, and a core is largely allowed to perform their build however they want, as long as they define certain things.

To begin, each core is required to have a 'build_options.mk' file defined in the root of their core directory (e.g. src/core_sw/build_options.mk). This file is expected to contain three things.

First, it is expected to define the EXE_NAME variable. The EXE_NAME variable will contain the name of the executable the core wants generated from the build process.

Second, it is expected to define the NAMELIST_SUFFIX variable. The NAMELIST_SUFFIX variable will contain the suffix for a namelist file. For example, when building the shallow water core, a NAMELIST_SUFFIX might be 'sw' in which case, the generated namelist file and the namelist file read by the executable will be named namelist.sw.

Finally, it is expected to define the report_builds target. The report_builds target should like all possible ways to build a core. This target is requried because cores are allowed to modify their build process however they want. So each core could require a completely different build method. When this target is run, it should simply echo all possible ways of building a core. For example, the shallow water core might have a report_builds target that echos CORE=sw, as that is the only way to build the shallow water core.

In addition to the requirements for build_options.mk, each core has two additional required targets. The first is all, which will be run when the core is being built, and should build all of the relevant files for a core. The second is core_reg, which should process all Registry.xml files and generate a Registry_processed.xml file in the root of the core directory (e.g. src/core_sw/Registry_processed.xml). This Registry_processed.xml file will be parsed when creating the auto-generated Fortran files.

Clone this wiki locally