/[MITgcm]/manual/s_phys_pkgs/mnc.tex
ViewVC logotype

Annotation of /manual/s_phys_pkgs/mnc.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph


Revision 1.14 - (hide annotations) (download) (as text)
Sat Dec 11 22:03:32 2004 UTC (20 years, 7 months ago) by edhill
Branch: MAIN
CVS Tags: checkpoint57l_post
Changes since 1.13: +42 -4 lines
File MIME type: application/x-tex
 o add text describing MNC compile-time size settings

1 edhill 1.14 % $Header: /u/gcmpack/manual/part6/mnc.tex,v 1.13 2004/10/12 21:44:59 edhill Exp $
2 edhill 1.1 % $Name: $
3    
4 edhill 1.6 \section{NetCDF I/O Integration: MNC}
5 edhill 1.3 \label{sec:pkg:mnc}
6 edhill 1.12 \begin{rawhtml}
7     <!-- CMIREDIR:package_mnc: -->
8     \end{rawhtml}
9 edhill 1.1
10 edhill 1.3 The \texttt{mnc} package is a set of convenience routines written to
11     expedite the process of creating, appending, and reading NetCDF files.
12     NetCDF is an increasingly popular self-describing file format
13     \cite{rew:97} intended primarily for scientific data sets. An
14     extensive collection of NetCDF reference papers, user guides,
15     software, FAQs, and other information can be obtained from UCAR's web
16     site at:
17     \begin{rawhtml} <A href="http://www.unidata.ucar.edu/packages/netcdf/"> \end{rawhtml}
18     \begin{verbatim}
19     http://www.unidata.ucar.edu/packages/netcdf/
20     \end{verbatim}
21     \begin{rawhtml} </A> \end{rawhtml}
22 edhill 1.1
23    
24 edhill 1.12 \subsection{Using MNC}
25    
26 edhill 1.14 \subsubsection{MNC Configuration}
27 edhill 1.12
28 edhill 1.14 As with all MITgcm packages, MNC can be turned on or off at compile time
29     using the \texttt{packages.conf} file or the \texttt{genmake2}
30 edhill 1.12 \texttt{-enable=mnc} or \texttt{-disable=mnc} switches.
31    
32 edhill 1.14 While MNC is likely to work ``as is'', there are a few compile--time
33     constants that may need to be increased for simulations that employ
34     large numbers of tiles within each process. Note that the important
35     quantity is the maximum number of tiles \textbf{per process}. Since
36     MPI configurations tend to distribute large numbers of tiles over
37     relatively large numbers of MPI processes, these constants will rarely
38     need to be increased.
39    
40     If MNC runs out of space within its ``lookup'' tables during a
41     simulation, then it will provide an error message along with a
42     recommendation of which parameter to increase. The parameters are all
43     located within \filelink{pkg/mnc/mnc\_common.h}{pkg-mnc-mnc_common.h}
44     and the ones that may need to be increased are:
45    
46     \begin{center}
47     {\footnotesize
48     \begin{tabular}[htb]{|l|r|l|}\hline
49     \textbf{Name} &
50     \textbf{Default} & \textbf{Description} \\\hline
51     & & \\
52     \texttt{MNC\_MAX\_ID} & 1000 &
53     \textbf{IDs for various low-level entities} \\
54     \texttt{MNC\_MAX\_INFO} & 400 &
55     \textbf{IDs (mostly for object sizes)} \\
56     \texttt{MNC\_CW\_MAX\_I} & 150 &
57     \textbf{IDs for the ``wrapper'' layer} \\\hline
58     \end{tabular}
59     }
60     \end{center}
61    
62     In those rare cases where MNC ``out-of-memory'' error messages are
63     encountered, it is a good idea to increase the too-small parameter by
64     a factor of \textbf{2--10} in order to avoid wasting time on an
65     iterative compile--test sequence.
66    
67    
68     \subsubsection{MNC Inputs}
69    
70 edhill 1.12 For run-time configuration, most of the MNC--related model parameters
71     are contained within a Fortran namelist file called \texttt{data.mnc}.
72     If this file does not exist, then the MNC package will interpret that
73     as an indication that it is not to be used. If the \texttt{data.mnc}
74     file does exist, then it may contain the following parameters:
75    
76     \begin{center}
77     {\footnotesize
78     \begin{tabular}[htb]{|l|c|l|l|}\hline
79     \textbf{Name} & \textbf{T} &
80     \textbf{Default} & \textbf{Description} \\\hline
81     & & & \\
82     \texttt{useMNC} & L & \texttt{.FALSE.} &
83     \textbf{overall MNC ON/OFF switch} \\
84     \texttt{mnc\_echo\_gvtypes} & L & \texttt{.FALSE.} &
85     echo pre-defined ``types'' (debugging) \\
86     \texttt{mnc\_use\_outdir} & L & \texttt{.FALSE.} &
87     create a directory for output \\
88     \texttt{mnc\_outdir\_str} & S & \texttt{'mnc\_'} &
89     output directory name \\
90     \texttt{mnc\_outdir\_date} & L & \texttt{.FALSE.} &
91     embed date in the output dir name \\
92     \texttt{pickup\_write\_mnc} & L & \texttt{.FALSE.} &
93     use MNC to write (create) pickup files \\
94     \texttt{pickup\_read\_mnc} & L & \texttt{.FALSE.} &
95     use MNC to read pickup files \\
96     \texttt{mnc\_use\_indir} & L & \texttt{.FALSE.} &
97     use a directory (path) for input \\
98     \texttt{mnc\_indir\_str} & S & \texttt{''} &
99     input directory (or path) name \\
100     \texttt{snapshot\_mnc} & L & \texttt{.FALSE.} &
101     write \texttt{snapshot} (instantaneous) w/MNC \\
102     \texttt{monitor\_mnc} & L & \texttt{.FALSE.} &
103     write \texttt{monitor} w/MNC \\
104     \texttt{timeave\_mnc} & L & \texttt{.FALSE.} &
105     write \texttt{timeave} w/MNC \\
106     \texttt{autodiff\_mnc} & L & \texttt{.FALSE.} &
107     write \texttt{autodiff} w/MNC \\\hline
108     \end{tabular}
109     }
110     \end{center}
111    
112     Additional MNC--related parameters are contained within the main
113     \texttt{data} namelist file and in some of the namelist files for
114     individual packages. These options are:
115     \begin{center}
116     {\footnotesize
117     \begin{tabular}[htb]{|l|c|l|l|}\hline
118     \textbf{Name} & \textbf{T} &
119     \textbf{Default} & \textbf{Description} \\\hline
120     \multicolumn{4}{|c|}{\ } \\
121     \multicolumn{4}{|c|}{Main namelist file:
122     ``\textbf{data}''} \\\hline
123     \texttt{snapshot\_ioinc} & L & \texttt{.FALSE.} &
124     write \texttt{snapshot} ``inclusively'' \\
125     \texttt{timeave\_ioinc} & L & \texttt{.FALSE.} &
126     write \texttt{timeave} ``inclusively'' \\
127     \texttt{monitor\_ioinc} & L & \texttt{.FALSE.} &
128     write \texttt{monitor} ``inclusively'' \\
129     \texttt{the\_run\_name} & C & ``name...'' &
130     name is included in all MNC output \\\hline
131     \multicolumn{4}{|c|}{\ } \\
132     \multicolumn{4}{|c|}{Diagnostics namelist file:
133     ``\textbf{data.diagnostics}''} \\\hline
134     \texttt{diag\_mnc} & L & \texttt{.FALSE.} &
135     write \texttt{diagnostics} w/MNC \\
136     \texttt{diag\_ioinc} & L & \texttt{.FALSE.} &
137     write \texttt{diagnostics} ``inclusively'' \\\hline
138     \end{tabular}
139     }
140     \end{center}
141    
142     By default, turning on MNC for a particular output type will result in
143     turning off all the corresponding (usually, default) MDSIO or STDOUT
144     output mechanisms. In other words, output defaults to being an
145     exclusive selection. To enable multiple kinds of simultaneous output,
146     flags of the form \texttt{NAME\_ioinc} have been created where
147     \texttt{NAME} corresponds to the various MNC output flags. When a
148     \texttt{NAME\_ioinc} flag is set to \texttt{.TRUE.}, then multiple
149     simultaneous forms of output are allowed for the \texttt{NAME} output
150     mechanism. The intent of this design is that typical users will only
151     want one kind of output while people debugging the code (particularly
152     the I/O routines) may want simultaneous types of output.
153    
154     This ``inclusive'' versus ``exclusive'' design is easily applied in
155     cases where three or more kinds of output may be generated. Thus, it
156     can be readily extended to additional new output types (eg. HDF5).
157    
158     Input types are always exclusive.
159    
160     \subsubsection{MNC Output}
161    
162     While NetCDF files are supposed to be ``self-describing'', it is
163     helpful to note the following:
164    
165     \begin{itemize}
166     \item The constraints placed upon the ``unlimited'' (or ``record'')
167     dimension inherent with NetCDF v3.x make it very inefficient to put
168     variables written at potentially different intervals within the same
169     file. For this reason, MNC output is split into a few file ``base
170     names'' which try to reflect the nature of their content.
171    
172     \item All MNC output is currently done in a ``tile-per-file'' fashion
173     since most NetCDF v3.x implementions cannot write safely within MPI
174     or multi-threaded environments. This tiling is done in a global
175     fashion and the tile numbers are appended to the base names
176     described above. Some scripts to ``assemble'' output are available
177     (\texttt{MITgcm/utils/matlab}). More general manipulations can be
178     accomplished with the
179     \begin{rawhtml}
180     <A href="http://nco.sourceforge.net">
181     \end{rawhtml}
182     \begin{verbatim}
183     NetCDF Operators (or ``NCO'') at http://nco.sourceforge.net
184     \end{verbatim}
185     \begin{rawhtml} </A> \end{rawhtml}
186     which is a very powerful and convenient set of tools for working
187     with all NetCDF files.
188    
189 edhill 1.13 \item On many systems, NetCDF has practical file size limits on the
190     order of 2--4GB (the maximium memory addressable with 32bit
191     pointers) due to a lack of operating system, compiler, and/or
192     library support. In cases where this limit is reached, it is
193     generally a good idea to reduce write frequencies or restart from
194     pickups.
195    
196 edhill 1.12 \item MNC does not (yet) provide a mechanism for reading information
197     from a single ``global'' file as can be done with the MDSIO
198 edhill 1.13 package. This is in progress.
199 edhill 1.12
200     \end{itemize}
201    
202    
203     \subsection{MNC Internals}
204 edhill 1.1
205 edhill 1.3 The \texttt{mnc} package is a two-level convenience library (or
206     ``wrapper'') for most of the NetCDF Fortran API. Its purpose is to
207     streamline the user interface to NetCDF by maintaining internal
208 edhill 1.6 relations (look-up tables) keyed with strings (or names) and entities
209     such as NetCDF files, variables, and attributes.
210 edhill 1.3
211     The two levels of the \texttt{mnc} package are:
212     \begin{description}
213    
214     \item[Upper level] \
215    
216     The upper level contains information about two kinds of
217     associations:
218     \begin{description}
219 edhill 1.6 \item[grid type] is lookup table indexed with a grid type name.
220 edhill 1.3 Each grid type name is associated with a number of dimensions, the
221     dimension sizes (one of which may be unlimited), and starting and
222     ending index arrays. The intent is to store all the necessary
223     size and shape information for the Fortran arrays containing
224     MITgcm--style ``tile'' variables (that is, a central region
225     surrounded by a variably-sized ``halo'' or exchange region as
226     shown in Figures \ref{fig:communication_primitives} and
227     \ref{fig:tiling-strategy}).
228    
229 edhill 1.6 \item[variable type] is a lookup table indexed by a variable type
230 edhill 1.3 name. For each name, the table contains a reference to a grid
231     type for the variable and the names and values of various
232     attributes.
233     \end{description}
234    
235     Within the upper level, these associations are not permanently tied
236     to any particular NetCDF file. This allows the information to be
237     re-used over multiple file reads and writes.
238    
239     \item[Lower level] \
240    
241     In the lower (or internal) level, associations are stored for NetCDF
242     files and many of the entities that they contain including
243     dimensions, variables, and global attributes. All associations are
244     on a per-file basis. Thus, each entity is tied to a unique NetCDF
245     file and will be created or destroyed when files are, respectively,
246     opened or closed.
247    
248     \end{description}
249 edhill 1.1
250    
251 edhill 1.12 \subsubsection{MNC Grid--Types and Variable--Types}
252 edhill 1.5
253     As a convenience for users, the MNC package includes numerous routines
254     to aid in the writing of data to NetCDF format. Probably the biggest
255     convenience is the use of pre-defined ``grid types'' and ``variable
256     types''. These ``types'' are simply look-up tables that store
257     dimensions, indicies, attributes, and other information that can all
258     be retrieved using a single character string.
259    
260     The ``grid types'' are a way of mapping variables within MITgcm to
261     NetCDF arrays. Within MITgcm, most spatial variables are defined
262     using two-- or three--dimensional arrays with ``overlap'' regions (see
263     Figures \ref{fig:communication_primitives}, a possible vertical index,
264     and \ref{fig:tiling-strategy}) and tile indicies such as the following
265     ``U'' velocity:
266     \begin{verbatim}
267     _RL uVel (1-OLx:sNx+OLx,1-OLy:sNy+OLy,Nr,nSx,nSy)
268     \end{verbatim}
269     as defined in \filelink{model/inc/DYNVARS.h}{model-inc-DYNVARS.h}
270    
271     The grid type is a character string that encodes the presence and
272     types associated with the four possible dimensions. The character
273     string follows the format
274     \begin{center}
275     \texttt{H0\_H1\_H2\_\_V\_\_T}
276     \end{center}
277 edhill 1.6 where the terms \textit{H0}, \textit{H1}, \textit{H2}, \textit{V},
278     \textit{T} can be almost any combination of the following:
279 edhill 1.5 \begin{center}
280     \begin{tabular}[h]{|ccc|c|c|}\hline
281 edhill 1.6 \multicolumn{3}{|c|}{Horizontal} & Vertical & Time \\
282 edhill 1.7 \textbf{H0}: location & \textbf{H1}: dimensions & \textbf{H2}: halo
283     & \textbf{V}: location & \textbf{T}: level \\\hline
284 edhill 1.5 \texttt{-} & xy & Hn & \texttt{-} & \texttt{-} \\
285     U & x & Hy & i & t \\
286     V & y & & c & \\
287     Cen & & & & \\
288     Cor & & & & \\\hline
289     \end{tabular}
290     \end{center}
291     A example list of all pre-defined combinations is contained in the
292     file
293     \begin{center}
294     \texttt{pkg/mnc/pre-defined\_grids.txt}.
295     \end{center}
296 edhill 1.7
297     The variable type is an association between a variable type name and the
298     following items:
299     \begin{center}
300 edhill 1.9 \begin{tabular}[h]{|l|l|}\hline
301 edhill 1.7 \textbf{Item} & \textbf{Purpose} \\\hline
302     grid type & defines the in-memory arrangement \\
303     \texttt{bi,bj} dimensions & tiling indices, if present \\\hline
304     \end{tabular}
305     \end{center}
306     and is used by the \texttt{mnc\_cw\_*\_[R|W]} subroutines for reading
307     and writing variables.
308 edhill 1.5
309    
310 edhill 1.12 \subsubsection{Using MNC: Examples}
311 edhill 1.5
312     Writing variables to NetCDF files can be accomplished in as few as two
313     function calls. The first function call defines a variable type,
314     associates it with a name (character string), and provides additional
315 edhill 1.6 information about the indicies for the tile (\texttt{bi},\texttt{bj})
316     dimensions. The second function call will write the data at, if
317     necessary, the current time level within the model.
318 edhill 1.5
319     Examples of the initialization calls can be found in the file
320 edhill 1.8 \filelink{model/src/ini\_mnc\_io.F}{model-src-ini_mnc_io.F}
321 edhill 1.9 where these function calls:
322 edhill 1.6 {\footnotesize
323 edhill 1.5 \begin{verbatim}
324 edhill 1.8 C Create MNC definitions for DYNVARS.h variables
325     CALL MNC_CW_ADD_VNAME('iter', '-_-_--__-__t', 0,0, myThid)
326     CALL MNC_CW_ADD_VATTR_TEXT('iter',1,
327     & 'long_name','iteration_count', myThid)
328    
329     CALL MNC_CW_ADD_VNAME('model_time', '-_-_--__-__t', 0,0, myThid)
330     CALL MNC_CW_ADD_VATTR_TEXT('model_time',1,
331     & 'long_name','Model Time', myThid)
332     CALL MNC_CW_ADD_VATTR_TEXT('model_time',1,'units','s', myThid)
333    
334     CALL MNC_CW_ADD_VNAME('U', 'U_xy_Hn__C__t', 4,5, myThid)
335     CALL MNC_CW_ADD_VATTR_TEXT('U',1,'units','m/s', myThid)
336     CALL MNC_CW_ADD_VATTR_TEXT('U',1,
337     & 'coordinates','XU YU RC iter', myThid)
338    
339     CALL MNC_CW_ADD_VNAME('T', 'Cen_xy_Hn__C__t', 4,5, myThid)
340     CALL MNC_CW_ADD_VATTR_TEXT('T',1,'units','degC', myThid)
341     CALL MNC_CW_ADD_VATTR_TEXT('T',1,'long_name',
342     & 'potential_temperature', myThid)
343     CALL MNC_CW_ADD_VATTR_TEXT('T',1,
344     & 'coordinates','XC YC RC iter', myThid)
345 edhill 1.6 \end{verbatim}
346     }
347 edhill 1.9 {\noindent initialize four \texttt{VNAME}s and add one or more NetCDF
348     attributes to each.}
349 edhill 1.5
350 edhill 1.9 The four variables defined above are subsequently written at specific
351 edhill 1.6 time steps within
352     \filelink{model/src/write\_state.F}{model-src-write_state.F}
353     using the function calls:
354     {\footnotesize
355 edhill 1.5 \begin{verbatim}
356 edhill 1.8 C Write dynvars using the MNC package
357     CALL MNC_CW_SET_UDIM('state', -1, myThid)
358 edhill 1.10 CALL MNC_CW_I_W('I','state',0,0,'iter', myIter, myThid)
359 edhill 1.8 CALL MNC_CW_SET_UDIM('state', 0, myThid)
360     CALL MNC_CW_RL_W('D','state',0,0,'model_time',myTime, myThid)
361     CALL MNC_CW_RL_W('D','state',0,0,'U', uVel, myThid)
362     CALL MNC_CW_RL_W('D','state',0,0,'T', theta, myThid)
363 edhill 1.6 \end{verbatim}
364     }
365 edhill 1.5
366 edhill 1.12 While it is easiest to write variables within typical 2D and 3D fields
367     where all data is known at a given time, it is also possible to write
368     fields where only a portion (\textit{eg.} a ``slab'' or ``slice'') is
369     known at a given instant. An example is provided within
370     \filelink{pkg/mom\_vecinv/mom\_vecinv.F}{pkg-mom_vecinv-mom_vecinv.F}
371     where an offset vector is used: {\footnotesize
372     \begin{verbatim}
373     IF (useMNC .AND. snapshot_mnc) THEN
374     CALL MNC_CW_RL_W_OFFSET('D','mom_vi',bi,bj, 'fV', uCf,
375     & offsets, myThid)
376     CALL MNC_CW_RL_W_OFFSET('D','mom_vi',bi,bj, 'fU', vCf,
377     & offsets, myThid)
378     ENDIF
379     \end{verbatim}
380     }
381     to write a 3D field one depth slice at a time.
382 edhill 1.1
383 edhill 1.12 Each element in the offset vector corresponds (in order) to the
384     dimensions of the ``full'' (or virtual) array and specifies which are
385     known at the time of the call. A zero within the offset array means
386     that all values along that dimension are available while a positive
387     integer means that only values along that index of the dimension are
388     available. In all cases, the matrix passed is assumed to start (that
389     is, have an in-memory structure) coinciding with the start of the
390     specified slice. Thus, using this offset array mechanism, a slice
391     can be written along any single dimension or combinations of
392     dimensions.
393 edhill 1.9

  ViewVC Help
Powered by ViewVC 1.1.22