M\+O\+M6 Validation and Verification

In the software engineering world, people talk about validation and verification of their codes. Verification is the confirmation of design specifications, such as\+:

\begin{DoxyItemize}
\item Does it compile on the target platform? \item Is it dimensionally consistent? \item Do answers change with the number of processes? \item Do answers change after a restart?\end{DoxyItemize}
Validation is a little trickier\+:

\begin{DoxyItemize}
\item Does the model meet operational needs? \item Does it produce realistic simulations? \item Are relevant physical features present? \item Can I reproduce my old simulations?\end{DoxyItemize}
There are a number of ways in which M\+O\+M6 is tested before each commit, especially commits to the shared dev/main branch.\hypertarget{Testing_Travis}{}\doxysection{Travis Testing}\label{Testing_Travis}
When pushing code to github, it is possible to set it up so that testing is performed automatically by travis. For M\+O\+M6, the .travis.\+yml file is executed, causing the code to be compiled and then run on all the tests in the .testing directory. It is also possible to run these tests on your local machine, but you might have to do some setup first. See \begin{DoxySeeAlso}{See also}
../../../.testing/\+R\+E\+A\+D\+M\+E.\+md for more information.
\end{DoxySeeAlso}
\hypertarget{Testing_Consortium_testing}{}\doxysection{Consortium Testing}\label{Testing_Consortium_testing}
For commits to the dev/main branch, there is an opportunity for all consortium members to weigh in on proposed updates. A view of the consortium is shown here\+:

 
\begin{DoxyImage}
\includegraphics[width=\textwidth,height=\textheight/2,keepaspectratio=true]{consortium.png}
\doxyfigcaption{The M\+O\+M6 consortium.}
\end{DoxyImage}


Each group is expected to have their own tests and to keep track of expected answers when these tests are run to be compared to prior answers after the code is updated. Answer-\/changing updates have to be evaluated carefully, though there are circumstances in which the new answers may well be \char`\"{}better\char`\"{}.\hypertarget{Testing_Novel_tests}{}\doxysection{Novel Tests}\label{Testing_Novel_tests}
There are two classes of tests which M\+O\+M6 performs within the .testing suite which could be considered unusual, but which can be quite useful for finding bugs.\hypertarget{Testing_Scalings}{}\doxysubsection{Scaling tests}\label{Testing_Scalings}
The equations of motion can be multiplied by factors of two without changing answers. One can use that to scale each of six units by a different factor of two to check for consistent use of units. For instance, the equation\+:

\[ u^{n+1} = u^n + \Delta t \times \cal{F} \]

can be scaled as\+:

\[ {2^{L-T}} u^{n+1} = {2^{L-T}} u^n + {2^T} \Delta t \times {2^{L-2T}} \cal{F} \]

M\+O\+M6 has been recoded to include six different scale factors\+: \hypertarget{Testing_scale_factors}{}
\tabulinesep=1mm
\begin{longtabu}spread 0pt [c]{*{3}{|X[-1]}|}
\caption{Dimensional scale factors}\label{Testing_scale_factors}\\
\hline
\cellcolor{\tableheadbgcolor}\textbf{ Unit }&\cellcolor{\tableheadbgcolor}\textbf{ Scaling }&\cellcolor{\tableheadbgcolor}\textbf{ Name }\\\cline{1-3}
\endfirsthead
\hline
\endfoot
\hline
\cellcolor{\tableheadbgcolor}\textbf{ Unit }&\cellcolor{\tableheadbgcolor}\textbf{ Scaling }&\cellcolor{\tableheadbgcolor}\textbf{ Name }\\\cline{1-3}
\endhead
s &T &Time \\\cline{1-3}
m &L &Horizontal length \\\cline{1-3}
m &H &Layer thickness \\\cline{1-3}
m &Z &Vertical length \\\cline{1-3}
kg/m3 &R &Density \\\cline{1-3}
J/kg &Q &Enthalpy \\\cline{1-3}
\end{longtabu}


You can add these integer scaling factors through the runtime parameters X\+\_\+\+R\+E\+S\+C\+A\+L\+E\+\_\+\+P\+O\+W\+ER, where X is one of T, L, H, Z, R, or Q. The valid range for these is -\/300 to 300.

When adding contributions to M\+O\+M6, this coding style with the scale factors must be maintained. For example, if you add new parameters to read from the input file\+: 
\begin{DoxyCode}{0}
\DoxyCodeLine{call get\_param(..., \textcolor{stringliteral}{"{}DT"{}}, ... , scale=US\%s\_to\_T)}
\end{DoxyCode}


This is also required for explicit contants, though we are trying to move those out of the code\+: 
\begin{DoxyCode}{0}
\DoxyCodeLine{ustar = 0.01 * US\%m\_to\_Z * US\%T\_to\_s}
\end{DoxyCode}


or for adding diagnostics\+: 
\begin{DoxyCode}{0}
\DoxyCodeLine{call \mbox{\hyperlink{namespacemom__diag__mediator_ada491446bf7216fe101e6b0f6ed551e1}{register\_diag\_field}}(..., \textcolor{stringliteral}{"{}u"{}}, ... , \&}
\DoxyCodeLine{      conversion=US\%L\_T\_to\_m\_s)}
\end{DoxyCode}
 \begin{DoxySeeAlso}{See also}
\mbox{\hyperlink{namespacemom__unit__scaling}{mom\+\_\+unit\+\_\+scaling}}
\end{DoxySeeAlso}
\hypertarget{Testing_Rotations}{}\doxysubsection{Rotational tests}\label{Testing_Rotations}
By setting the runtime option R\+O\+T\+A\+T\+E\+\_\+\+I\+N\+D\+EX to True, the model rotates the domain by some number of 90 degree turns. This option can be used to look for bugs in which east-\/west operations do not match north-\/south operations. It changes the order of array elements as shown here\+:

 
\begin{DoxyImage}
\includegraphics[width=\textwidth,height=\textheight/2,keepaspectratio=true]{Rotated_indices.png}
\doxyfigcaption{The original non-\/rotated domain is shown on the left while the right shows the domain rotated counterclockwise by 90 degrees. The array values are shown by the (invariant) colors, while the array indices (and dimensions) change.}
\end{DoxyImage}


It only currently runs in serial mode. One can ask for rotations of 90, 180, or 270 degrees, but only 90 degree turns are supported if there are open boundaries.

Because order matters in numerical computations, care must be taken for four-\/way averages to match between rotated and non-\/rotated runs. Say you want to compute the following quantity\+:

\[ \phi_{i,j}^{(c)} = \frac{1}{4} (\phi_A + \phi_B + \phi_C + \phi_D) \]

as shown in this diagram\+:



You might write this first as\+: \[ \frac{1}{4} ((\phi_A + \phi_B) + (\phi_C + \phi_D)) \] as shown on the left in this figure\+:



However, the round-\/off errors could give differing answers when rotated. Instead, you want to group the terms on the diagonal as shown in the right of the above figure and here\+: \[ \frac{1}{4} ((\phi_A + \phi_D) + (\phi_B + \phi_C)) \] 