Rose-UKCA suites

Rose UKCA suites on MONsooN

This page describes the Rose suites with UKCA that are installed on MONsooN.

Disclaimer: The main purpose of these suites is for model development and overall assessment of performance. Until the suites are assessed properly and released via the UKCA Release Jobs page, they should be considered as not suitable for Science applications.

Pre-requisites

Account for the PUMA Rose repository and access to the MONSooN HPC system.
Basic knowledge about Rose and its usage with UM.

For the initial Rose setup and instructions on using Rose on MONSooN, see this page (requires Collaboration Twiki login).

UM10.2

puma-aa237: This is an UKCA StratTrop/CheST Chemistry +GLOMAP suite at N96L85 resolution. The suite is derived from the UM Global Atmosphere development suite mi-ah615, but using the StraTrop or CheST scheme instead of Offline oxidants.
The suite is set up to archive model output (to the MASS tape drive) and submit the next segment, so it can be used for performing long simulations.

UM10.3

puma-aa262: This is a test suite based on the UKCA configurations in the UM trunk and can simultaneously run the following configurations at N96L85:

• StratTrop(CheST)+GLOMAP+RADAER+Activate
• Offline oxidants+GLOMAP+RADAER+Activate with two PE configurations

Both the configurations carry out a 2-day simulation that requires between 10 - 15 minutes to complete on the Cray. The CheST Configuration uses 16x8 PEs, while the Offline configuration uses 16x8 and 8x12 PEs.

UM10.4

puma-aa322: This is a test suite based on the UKCA configurations in the UM trunk and can simultaneously run the following configurations at N96L85:

• StratTrop(CheST)+GLOMAP+RADAER+Activate
• Offline oxidants+GLOMAP+RADAER+Activate

The Offline oxidants job includes tests to compare between 16x8 vs 8x12 PE configurations (TEST_PE), as well as between runs composed of different segments (NRUN-CRUN, CRUN_TEST)
The main configurations carry out a 2-day simulation that requires between 10 - 15 minutes to complete on the Cray.
A significant change from the UM10.3 suite is that all the tasks are run from the same base app (um), making use of optional configurations : ROSE_APP_OPT_CONF_KEYS in the suite.rc.

Using the UM10.4 Suite

All Rose suites on MONsooN need to be executed from the special Rose server as described in the instructions and not on the puma system.

1. Create a suitable folder for the suite. (default name ~/roses)
2. cd to the folder and create a copy of the Suite

   ~/roses $ rosie copy puma-aa322

   This will create a copy of the suite as puma-aaNNN and check it out to the ~/roses folder.

3. To ensure that the base settings work for you:

   $ cd ~/roses/puma-aaNNN

   puma-aaNNN $ rose suite-run

   This will preprocess the configuration and launch the tasks in the background along with opening the gcylc window that displays the status of each task.

   Once all the tasks complete (Status = stopped with 'succeeded' or check output in Rose-bush), the Suite can now be modified to test your changes.

4. Execution options
   

   The suite can be executed as a combination of the following configurations, by setting values of the related variables in puma-aaNNN/rose-suite.conf:

   Compilation (Build) On/Off: BUILD=true/false

   StratTrop Chemistry scheme: STRATTROP=true/false

   Offline Oxidants Chemistry scheme: OFFLINE=true/false

   PE Test for the Offline scheme: TEST_PE=true/false

   NRUN-CRUN Test for the Offline scheme: TEST_CRUN=true/false

   Setting both Offline & StratTrop options to false will convert the suite to a compile-only job.

   Additional Options:

   Change the length of the run(s): modify TASKEND='0,Months,Days,Hours,Minutes,0' (Note: Meaning is Off, so the apps can be run only upto one month)

   Change the frequency of restart dumps: set DUMPFREQ=number-of-days

5. Testing an UM10.4 branch

   If you are only making changes to the code and not modifying any namelists, the suite only needs to be pointed to the new branch:

   puma-aaNNN $ Edit app/fcm_make/rose-app.conf and add your branch at the line

   um_sources=branches/your-branch or um_sources=working-copy (full path)

   Note that since suites use the mirror of the Ex-UM respository, it takes 5-10 minutes for any recently committed changes to be available to the suite.

   Now rerun the suite:

   puma-aaNNN $ rose suite-run --new

   The --new option forces the removal (and fresh creation) of the extracted source, build and working folders thus ensuring that all modifications to the source code made in your branch are included.

   If the code changes are expected to change results for this configuration, failure of the comparison (rose ana) tasks can be ignored. However, if no change is expected yet the comparison fails, this indicates a bug somewhere in the implementation which needs to be traced, corrected and re-tested.

6. Namelist changes

   If there are any modifications to the namelists e.g. addition or deletion of items, moving items between namelists, these need to be applied in puma-aaNNN/app/um/rose-app.conf

   Note: Since the base app/um/rose-app.conf needs to work for both UKCA configurations, any new settings that are specific to a configuration need to be made in the optional configurations viz app/um/rose-app.offline.conf and app/um/rose-app-stratdiags.conf.

1. For adding the accompanying metadata changes:

   Checkout your branch to an universal location i.e. $HOME or $DATADIR.

   Modify the meta-data file : your-working-copy/rose-meta/um-atmos/HEAD/rose-meta.conf to reflect the namelist changes.

   Edit both puma-aaNNN/app/um/rose-app.conf and replace the line:

   meta=um-atmos/vn10.4 with meta=your working copy/rose-meta/um-atmos/HEAD

   Open the apps with rose edit to ensure that the namelist changes are consistent with the metadata.

   Now rerun the suite to make sure the Rose namelist changes are consistent with the Fortran namelist changes.

Warning: If any of the tasks in the workflow fails, the corresponding housekeeping task will not be carried out and all the data generated will remain in $DATADIR/cylc-run/puma-aaNNN on the HPC. This may have to be cleaned manually after testing through the command:

puma-aaNNN $ rose suite-clean