Difference between revisions of "Using the UM13 test suites"

Revision as of 13:14, 12 April 2024

Using the UKCA test suites : UM13x

The Rose suites listed here can be executed on Met Office XC40, EX1A and the Monsoon HPCs. Note: For Monsoon suite is launched from the xcsl00/01 login nodes as described in the Rose-Monsoon instructions and not from the PUMA system.

Create a suitable folder for the suite. (default name ~/roses)
cd to the folder and create a copy of the Suite
~/roses $ rosie copy u-df325 (See suites for other versions above)

This will create a copy of the suite (probably as) u-dXNNN and check it out to the ~/roses folder.
To ensure that the base settings work for you:
$ cd ~/roses/u-dXNNN

rose edit
Under suite conf --> jinja2 panel, select Site : as Meto_xc40 ; Meto_ex1a ; Monsoon

Note: Access to Meto_ex1a system is currently restricted to early users.

For Monsoon, this will prompt for choice of the account to be used.
USE_DEFAULT_ACCOUNT: Use the default account on Monsoon (alphabetically 1st account in the `id $USER` list). If set to False:

ACCOUNT_MONSOON : The Monsoon account (project) under which the suite is to be run. (You need to be a part of that project!).

Execution options

Compilation (Build) On/Off: BUILD=true/false

Use Pre-build for Compilation On/Off: USE_PREBUILD=true/false (Check that target HPC has prebuilds installed in ~frum/cylc-run)

StratTrop Chemistry scheme: RUN_STRATTROP=true/false

CRI-Strat Chemistry scheme: RUN_CRI=true/false

Offline Oxidants Chemistry scheme: RUN_OFFLINE=true/false

Run KGO, PE, OpenMP, NRUN-CRUN and Rigorous build Tests: RUN_TESTS=true/false (additional option RUN_RIGOROUS=true/false)

Run Housekeeping to clear model output: HOUSEKEEP=true/false

Setting Offline, StratTrop, CRI-Strat options to false will convert the suite to a compile-only job.

The use of prebuilds reduces the compilation time as well as resource requirement. However, for some code changes, as well as to make use of Dr Hook profiling, this needs to be turned Off

Save and exit

u-dXNNN $ rose suite-run

This will preprocess the configuration and launch the tasks in the background along with opening the gcylc window that displays the status of each task.

Once all the tasks complete (Status = stopped with 'succeeded' or check output in Rose-bush), the Suite can now be modified to test your changes.

Additional Options:

Change the length of the run(s): modify TASKEND='0,Months,Days,Hours,Minutes,0' (Note: Climate meaning and suite-cycling is Off, so the apps can be run only up to one month)

Change the frequency of restart dumps: set DUMPFREQ=number-of-days

Both these options are only available if testing is switched Off, since the tests require a specific run-length and dumping frequency.
Testing your UM and/or UKCA branch
If you are only making changes to the code and not modifying any namelists, the suite only needs to be pointed to the new branch:
u-dXNNN $ Edit app/fcm_make/rose-app.conf and add your UM/ UKCA branch(es) at the line
um_sources=branches/your-branch or um_sources=working-copy (full path)

ukca_sources=branches/your-branch or working-copy

For running on MetO systems, if using a working copy this needs to be prefixed with the machine-id where the source exists

um_sources=vldXXX:<path to working copy> and/or ukca_sources=vldXXX:<path to working copy>

Note that since suites use the mirror of the MOSRS respositories, it takes 5-10 minutes for any recently committed changes to be available to the suite.

Now rerun the suite:
u-dXNNN $ rose suite-run --new

The --new option forces the removal (and fresh creation) of the extracted source, build and working folders thus ensuring that all modifications to the source code made in your branch are included.

If any code changes are expected to change the results for this configuration, failure of the comparison against Known Good Output (KGO) i.e. the rose_ana_atmos_*_kgo tasks can be ignored. However, if the comparison tests fail unexpectedly, this indicates a bug somewhere in the implementation which needs to be traced, corrected and re-tested.
Namelist changes
If there are any modifications to the namelists e.g. addition or deletion of items, moving items between namelists, these need to be applied in u-dXNNN/app/um/rose-app.conf
Note: Since the base app/um/rose-app.conf needs to work for both UKCA configurations, any new settings that are specific to a configuration need to be made in the optional configurations viz app/um/opt/rose-app.offline.conf and app/um/opt/rose-app-stratdiags.conf.

For adding the accompanying metadata changes:

Checkout your branch to an universal location i.e. $HOME or $DATADIR.

Modify the meta-data file : your-working-copy/rose-meta/um-atmos/HEAD/rose-meta.conf to reflect the namelist changes.

Edit the u-dXNNN/app/um/rose-app.conf and replace the line:

meta=um-atmos/vn13.5 (or earlier vn13.X) with meta=your working copy/rose-meta/um-atmos/HEAD

Open the apps with rose edit to ensure that the namelist changes are consistent with the metadata.

Now rerun the suite to make sure the Rose namelist changes are consistent with the Fortran namelist changes.
Making and Testing changes to STASHmaster

This is a bit more complicated as the changes required for Rose (i.e. GUI) to see the new STASHmaster are different to those required so that the 'run' or executable sees this.

Changes for Rose/GUI to see the new STASHmaster = the same as other metadata changes
Edit the u-dXNNN/app/um/rose-app.conf and replace the line:

meta=um-atmos/vn13.5 with meta=your working copy/rose-meta/um-atmos/HEAD

Changes for the model/run to see the new STASHmaster
Make sure to commit your STASHmaster_A changes to the branch

In the app/um/rose-app.conf,in the [env] section add the line:

STASHMASTER=STASHmaster

and at the end of the [file:xxx] section, add the following two lines:

[file:STASHmaster]

source=fcm:um.xm_br/dev/user-id/branch-name/rose-meta/um-atmos/HEAD/etc/stash/STASHmaster@your-revision

Commit the suite-changes and re-run as normal

Warning: If any of the tasks in the workflow fails, the corresponding housekeeping task will not be carried out and all the data generated will remain in $DATADIR/cylc-run/u-dXNNN on the HPC. This may have to be cleaned manually after testing through the command:

u-dXNNN $ rose suite-clean

@@ Line 1: / Line 1: @@
 === Using the UKCA test suites : UM13x ===
-'''13.3: [https://code.metoffice.gov.uk/trac/roses-u/browser/d/a/9/4/0 u-da940]'''<br/>
+'''13.5: [https://code.metoffice.gov.uk/trac/roses-u/browser/d/f/3/2/5 u-df325]'''<br/>
+'''13.4: [https://code.metoffice.gov.uk/trac/roses-u/browser/d/a/9/4/0 u-da940]'''<br/>
 '''13.3: [https://code.metoffice.gov.uk/trac/roses-u/browser/c/y/0/7/9 u-cy079]'''<br/>
@@ Line 10: / Line 13: @@
 '''13.0: [https://code.metoffice.gov.uk/trac/roses-u/browser/c/p/5/5/1 u-cp551]'''<br/>
-All Rose suites listed here are set to be executed on Monsoon as described in the [https://code.metoffice.gov.uk/doc/monsoon2/rose.html#rose-setup Rose-Monsoon] instructions and '''not from the PUMA''' system.<br/>
+The Rose suites listed here can be executed on Met Office XC40, EX1A and the Monsoon HPCs. '''
+''Note: For Monsoon suite is launched from the xcsl00/01 login nodes as described in the [https://code.metoffice.gov.uk/doc/monsoon2/rose.html#rose-setup Rose-Monsoon] instructions and '''not from the PUMA''' system.''<br/>
 # Create a suitable folder for the suite. (default name ~/roses)
 # cd to the folder and create a copy of the Suite
-#: ~/roses $ ''rosie copy u-da940''   (See suites for other versions above)
+#: ~/roses $ ''rosie copy u-df325''   (See suites for other versions above)
 #:This will create a copy of the suite (probably as) ''u-dXNNN'' and check it out to the ~/roses folder.
 # To ensure that the base settings work for you:
@@ Line 21: / Line 25: @@
 #::: Under ''suite conf --> jinja2'' panel, select '''Site :''' as ''Meto_xc40'' ; ''Meto_ex1a'' ; ''Monsoon''
 #:: '''Note:''' Access to ''Meto_ex1a'' system is currently restricted to early users.
 #:: For ''Monsoon'', this will prompt for choice of the account to be used.
 #::: USE_DEFAULT_ACCOUNT: Use the default account on Monsoon (alphabetically 1st account in the `id $USER` list). If set to ''False'':
 #::: ACCOUNT_MONSOON : The Monsoon account (project) under which the suite is to be run. (You need to be a part of that project!).
@@ Line 68: / Line 72: @@
 #::Modify the meta-data file : ''your-working-copy/rose-meta/um-atmos/HEAD/rose-meta.conf'' to reflect the namelist changes.
 #::Edit the ''u-dXNNN/app/um/rose-app.conf'' and replace the line:
-#::''meta=um-atmos/vn13.4'' (or earlier vnXX.X) with ''meta=your working copy/rose-meta/um-atmos/HEAD''
+#::''meta=um-atmos/vn13.5'' (or earlier vn13.X) with ''meta=your working copy/rose-meta/um-atmos/HEAD''
 #:
 #:Open the apps with ''rose edit'' to ensure that the namelist changes are consistent with the metadata.
@@ Line 79: / Line 83: @@
 #: ''Changes for Rose/GUI to see the new STASHmaster'' = the same as other metadata changes
 #::Edit the ''u-dXNNN/app/um/rose-app.conf'' and replace the line:
-#::''meta=um-atmos/vn13.4'' with ''meta=your working copy/rose-meta/um-atmos/HEAD''
+#::''meta=um-atmos/vn13.5'' with ''meta=your working copy/rose-meta/um-atmos/HEAD''
 #:
 #: ''Changes for the model/run to see the new STASHmaster''