Conventions for staggering of variables and loops over 2d/3d arrays

M\+O\+M6 is written in Fortran90 and uses the {\ttfamily i,j,k} order of indexing. {\ttfamily i} corresponds to the fastest index (stride-\/1 in memory) and thus should be the inner-\/most loop variable. We often refer to the i-\/direction as the x-\/ or zonal direction, and similarly to the j-\/direction as y-\/ or meridional direction. The model can use curvilinear grids/coordinates in the horizontal and so these labels have loose meanings but convenient.\hypertarget{Horizontal_Indexing_section_Staggering}{}\section{Loops and staggered variables}\label{Horizontal_Indexing_section_Staggering}
Many variables are staggered horizontally with respect to each other. The dynamics and tracer equations are discretized on an Arakawa C grid. Staggered variables must still have integer indices and we use a north-\/east convention centered on the h-\/points. These means a variable with indices {\ttfamily i,j} will be either collocated, to the east, to the north, or to the north-\/east of the h-\/point with the same indices.

 
\begin{DoxyImage}
\includegraphics[width=\textwidth,height=\textheight/2,keepaspectratio=true]{Arakawa_C_grid.png}
\doxyfigcaption{M\+O\+M6 uses an Arakawa C grid staggering of variables with a North-\/\+East indexing convention. \char`\"{}\+Cells\char`\"{} refer to the control volumes around tracer-\/ or h-\/point located variables unless labelled otherwise.}
\end{DoxyImage}
\hypertarget{Horizontal_Indexing_Soft_convention}{}\subsection{Soft convention for loop variables}\label{Horizontal_Indexing_Soft_convention}
To ease reading the code we use a \char`\"{}soft\char`\"{} convection (soft because there is no syntax checking) where an upper-\/case index variable can be interpreted as the lower-\/case index variable plus $\frac{1}{2}$.

For example, when a loop is over h-\/points collocated variables
\begin{DoxyItemize}
\item the do-\/loop statements will be for lower-\/case {\ttfamily i,j} variables
\item references to h-\/point variables will be {\ttfamily h(i,j)}, {\ttfamily D(i+1,j)}, etc.
\item references to u-\/point variables will be {\ttfamily u(\+I,j)} (meaning $u_{i+\frac{1}{2},j}$), {\ttfamily u(I-\/1,j)} (meaning $u_{i-\frac{1}{2},j}$), etc.
\item references to v-\/point variables will be {\ttfamily v(i,\+J)} (meaning $v_{i,j+\frac{1}{2}}$), {\ttfamily u(I-\/1,j)} (meaning $u_{i,j-\frac{1}{2}}$), etc.
\item references to q-\/point variables will be {\ttfamily q(\+I,\+J)} (meaning $q_{i+\frac{1}{2},j+\frac{1}{2}}$), etc.
\end{DoxyItemize}

In contrast, when a loop is over u-\/points collocated variables
\begin{DoxyItemize}
\item the do-\/loop statements will be for upper-\/case {\ttfamily I} and lower-\/case {\ttfamily j} variables
\item the expression $ u_{i+\frac{1}{2},j} ( h_{i,j} + h_{i+1,j} ) $ is {\ttfamily u(\+I,j) $\ast$ ( h(i,j) + h(i+1,j)}.
\end{DoxyItemize}\hypertarget{Horizontal_Indexing_section_Memory}{}\section{Declaration of variables}\label{Horizontal_Indexing_section_Memory}
 
\begin{DoxyImage}
\includegraphics[width=\textwidth,height=\textheight/2,keepaspectratio=true]{Horizontal_NE_indexing_nonsym.png}
\doxyfigcaption{Non-\/symmetric mode\+: All arrays are declared with the same shape (isd\+:ied,jsd\+:jed).}
\end{DoxyImage}


 
\begin{DoxyImage}
\includegraphics[width=\textwidth,height=\textheight/2,keepaspectratio=true]{Horizontal_NE_indexing_sym.png}
\doxyfigcaption{Symmetric mode\+: Arrays have different shapes depending on their staggering location on the Arakawa C grid.}
\end{DoxyImage}


A field is described by 2D or 3D arrays which are distributed across parallel processors. Each processor only sees a small window of the global field. The processor \char`\"{}owns\char`\"{} the computational domain (red in above figure) but arrays are extended horizontally with halos which are intermittently updated with the values from neighboring processors. The halo regions (blue in above figure) may not always be up-\/to-\/date. Data in halo regions (blue in above figure) will be overwritten my mpp\+\_\+updates.

M\+O\+M6 has two memory models, \char`\"{}symmetric\char`\"{} and \char`\"{}non-\/symmetric\char`\"{}. In non-\/symmetric mode all arrays are given the same shape. The consequence of this is that there are fewer staggered variables to the south-\/west of the computational domain. An operator applied at h-\/point locations involving u-\/ or v-\/ point data can not have as wide a stencil on the south-\/west side of the processor domain as it can on the north-\/east side.

In symmetric mode, declarations are dependent on the variables staggered location on the Arakawa C grid. This allows loops to be symmetric and stencils to be applied more uniformly.

In the code, declarations are consistent with the symmetric memory model. The non-\/symmetric mode is implemented by setting the start values of the staggered data domain to be the same as the non-\/staggered start value.

The horizontal index type (\mbox{\hyperlink{structmom__hor__index_1_1hor__index__type}{mom\+\_\+hor\+\_\+index\+::hor\+\_\+index\+\_\+type}}) provides the data domain start values. The values are also copied into the \mbox{\hyperlink{structmom__grid_1_1ocean__grid__type}{mom\+\_\+grid\+::ocean\+\_\+grid\+\_\+type}} for convenience although we might deprecate this convenience in the future.

Declarations of h-\/point data take the form\+:
\begin{DoxyItemize}
\item {\ttfamily real, dimension(HI\%isd\+:HI\%ied, HI\%jsd\+:HI\%jed) \+:: D !$<$ Depth at h-\/points (m)}
\item {\ttfamily real, dimension(HI\%isd\+:HI\%ied, HI\%jsd\+:HI\%jed, GV\%ke) \+:: h !$<$ Layer thickness (H units)}
\end{DoxyItemize}

Declarations of u-\/point data take the form\+:
\begin{DoxyItemize}
\item {\ttfamily real, dimension(HI\%IsdB\+:HI\%IedB, HI\%jsd\+:HI\%jed) \+:: Du !$<$ Depth at u-\/points (m)}
\item {\ttfamily real, dimension(HI\%IsdB\+:HI\%IedB, HI\%jsd\+:HI\%jed, GV\%ke) \+:: h !$<$ Zonal flow (m/s)}
\end{DoxyItemize}

Declarations of v-\/point data take the form\+:
\begin{DoxyItemize}
\item {\ttfamily real, dimension(HI\%isd\+:HI\%ied, HI\%JsdB\+:HI\%JedB) \+:: Dv !$<$ Depth at v-\/points (m)}
\item {\ttfamily real, dimension(HI\%isd\+:HI\%ied, HI\%JsdB\+:HI\%JedB, GV\%ke) \+:: h !$<$ Zonal flow (m/s)}
\end{DoxyItemize}

Declarations of q-\/point data take the form\+:
\begin{DoxyItemize}
\item {\ttfamily real, dimension(HI\%IsdB\+:HI\%IedB, HI\%JsdB\+:HI\%JedB) \+:: Dq !$<$ Depth at q-\/points (m)}
\item {\ttfamily real, dimension(HI\%IsdB\+:HI\%IedB, HI\%JsdB\+:HI\%JedB, GV\%ke) \+:: vort !$<$ Vertical componentof vorticity (s-\/1)}
\end{DoxyItemize}

The file \mbox{\hyperlink{MOM__memory__macros_8h}{M\+O\+M\+\_\+memory\+\_\+macros.\+h}} provides the macros {\ttfamily S\+Z\+I\+\_\+}, {\ttfamily S\+Z\+J\+\_\+}, {\ttfamily S\+Z\+I\+B\+\_\+} and {\ttfamily S\+Z\+J\+B\+\_\+} that help make the above more concise\+:
\begin{DoxyItemize}
\item {\ttfamily real, dimension(\mbox{\hyperlink{MOM__memory__macros_8h_a778d74c20afedc458a527b6d5ca06fdc}{S\+Z\+I\+\_\+(\+H\+I)}}, \mbox{\hyperlink{MOM__memory__macros_8h_a982673508fcf471cd799850a8ec04cca}{S\+Z\+J\+\_\+(\+H\+I)}}) \+:: D !$<$ Depth at h-\/points (m)}
\item {\ttfamily real, dimension(\mbox{\hyperlink{MOM__memory__macros_8h_a368ddc75628f875f9628830c26651e18}{S\+Z\+I\+B\+\_\+(\+H\+I)}}, \mbox{\hyperlink{MOM__memory__macros_8h_a982673508fcf471cd799850a8ec04cca}{S\+Z\+J\+\_\+(\+H\+I)}}) \+:: Du !$<$ Depth at u-\/points (m)}
\item {\ttfamily real, dimension(\mbox{\hyperlink{MOM__memory__macros_8h_a778d74c20afedc458a527b6d5ca06fdc}{S\+Z\+I\+\_\+(\+H\+I)}}, \mbox{\hyperlink{MOM__memory__macros_8h_a2c9d3957c5b3b5e1b650ae0bc4250531}{S\+Z\+J\+B\+\_\+(\+H\+I)}}) \+:: Dv !$<$ Depth at v-\/points (m)}
\item {\ttfamily real, dimension(\mbox{\hyperlink{MOM__memory__macros_8h_a368ddc75628f875f9628830c26651e18}{S\+Z\+I\+B\+\_\+(\+H\+I)}}, \mbox{\hyperlink{MOM__memory__macros_8h_a2c9d3957c5b3b5e1b650ae0bc4250531}{S\+Z\+J\+B\+\_\+(\+H\+I)}}) \+:: Dq !$<$ Depth at q-\/points (m)}
\end{DoxyItemize}

See \mbox{\hyperlink{MOM__memory__macros_8h}{M\+O\+M\+\_\+memory\+\_\+macros.\+h}} for the complete list of macros used in various memory modes.\hypertarget{Horizontal_Indexing_Global_index}{}\section{Calculating a global index}\label{Horizontal_Indexing_Global_index}
For the most part M\+O\+M6 code should be independent of an equivalent absolute global index. There are exceptions and when the global index of a cell {\ttfamily i,j} is needed is can be calculated as follows\+:

{\ttfamily i\+\_\+global = i + HI\%idg\+\_\+offset}

Before the \mbox{\hyperlink{structmom__hor__index_1_1hor__index__type}{mom\+\_\+hor\+\_\+index\+::hor\+\_\+index\+\_\+type}} was introduced, this conversion was done use variables in the \mbox{\hyperlink{structmom__grid_1_1ocean__grid__type}{mom\+\_\+grid\+::ocean\+\_\+grid\+\_\+type}}\+:

{\ttfamily i\+\_\+global = (i-\/G\%isd) + G\%isd\+\_\+global}

which is no longer preferred.

Note that a global index only makes sense for a rectangular global domain. If the domain is a Mosaic of connected tiles (e.\+g. size tiles of a cube) the global indices (i,j) become meaningless. 