Rose-UKCA suites

From UKCA

Rose UKCA suites on MONsooN


This page describes the Rose suites with UKCA that are installed on MONsooN.

Disclaimer: The main purpose of these suites is for model development and overall assessment of performance. Until the suites are assessed properly and released via the UKCA Release Jobs page, they should be considered as not suitable for Science applications.

Pre-requisites

Account for the PUMA Rose repository and access to the MONSooN HPC system.
Basic knowledge about Rose and its usage with UM.

For the initial Rose setup and instructions on using Rose on MONSooN, see this page (requires Collaboration Twiki login).

UM10.2

puma-aa237: This is an UKCA StratTrop/CheST Chemistry +GLOMAP suite at N96L85 resolution. The suite is derived from the UM Global Atmosphere development suite mi-ah615, but using the StraTrop or CheST scheme instead of Offline oxidants.
The suite is set up to archive model output (to the MASS tape drive) and submit the next segment, so it can be used for performing long simulations.

UM10.3

puma-aa262: This is a test suite based on the UKCA configurations in the UM trunk and can simultaneously run the following configurations at N96L85:

  • StratTrop(CheST)+GLOMAP+RADAER+Activate
  • Offline oxidants+GLOMAP+RADAER+Activate with two PE configurations

Both the configurations carry out a 2-day simulation that requires between 10 - 15 minutes to complete on the Cray. The CheST Configuration uses 16x8 PEs, while the Offline configuration uses 16x8 and 8x12 PEs.

Using the Suite

The following steps use the UM10.3 suite as an example. All Rose suites on MONsooN need to be executed from the special Rose server as described in the instructions and not on the puma system.

  1. Create a suitable folder for the suite. (default name ~/roses)
  2. cd to the folder and create a copy of the Suite
    ~/roses $ rosie copy puma-aa262
    This will create a copy of the suite as puma-aaNNN and check it out to the ~/roses folder.
  3. To ensure that the base settings work for you:
    I (MarkR) had to change the ACCOUNT in suite-conf jinja2 (bottom of right hand panel) from ukca-meto to ukca-leeds (you need to know which site you belong to on the XC40 projects)
    Use rose edit in the directory to change the ACCOUNT value
    $ cd ~/roses/puma-aaNNN
    puma-aaNNN $ rose suite-run
    This will preprocess the configuration and launch the tasks in the background along with opening the gcylc window that displays the status of each task.
    Once all the tasks complete (Status = stopped with 'succeeded' or check output in Rose-bush), the Suite can now be modified to test your changes.
  4. Execution options
    The suite can be executed as a combination of the following configurations, by setting values of the related variables in puma-aaNNN/rose-suite.conf:
    Compilation (Build) On/Off: BUILD=true/false
    StratTrop Chemistry scheme: STRATTROP=true/false
    Offline Oxidants Chemistry scheme: OFFLINE=true/false
    PE Test for the Offline scheme: TEST_PE=true/false
    Setting both Offline & StratTrop options to false will convert the suite to a compile-only job.
    Additional Options:
    Change the length of the run(s): modify TASKEND='0,Months,Days,Hours,Minutes,0' (Note: Meaning is Off, so the apps can be run only upto one month)
    Change the frequency of restart dumps: set DUMPFREQ=number-of-days
  5. Testing an UM10.3 branch
    If you are only making changes to the code and not modifying any namelists, the suite only needs to be pointed to the new branch:
    puma-aaNNN $ Edit app/fcm_make/rose-app.conf and add your branch at the line
    um_sources=branches/your-branch or um_sources=working-copy (full path)
    Note that since suites use the mirror of the Ex-UM respository, it takes 5-10 minutes for any recently committed changes to be available to the suite.
    Now rerun the suite:
    puma-aaNNN $ rose suite-run --new
    The --new option forces the removal (and fresh creation) of the extracted source, build and working folders thus ensuring that all modifications to the source code made in your branch are included.
    If the code changes are expected to change results for this configuration, failure of the comparison (rose ana) tasks can be ignored. However, if no change is expected yet the comparison fails, this indicates a bug somewhere in the implementation which needs to be traced, corrected and re-tested.
  6. Namelist changes
    If there are any modifications to the namelists e.g. addition or deletion of items, moving items between namelists, these need to be applied in
    puma-aaNNN/app/um/rose-app.conf as well as puma-aaNNN/app/um_offl/rose-app.conf.
    For adding the accompanying metadata changes:
    Checkout your branch to an universal location i.e. $HOME or $DATADIR.
    Modify the meta-data file : your-working-copy/rose-meta/um-atmos/HEAD/rose-meta.conf to reflect the namelist changes.
    Edit both puma-aaNNN/app/um/rose-app.conf and puma-aaNNN/app/um_offl/rose-app.conf and replace the line:
    meta=um-atmos/vn10.3 with meta=your working copy/rose-meta/um-atmos/HEAD
    Open the apps with rose edit to ensure that the namelist changes are consistent with the metadata.
    Now rerun the suite to make sure the Rose namelist changes are consistent with the Fortran namelist changes.

Warning: If any of the tasks in the workflow fails, the corresponding housekeeping task will not be carried out and all the data generated will remain in $DATADIR/cylc-run/puma-aaNNN on the HPC. This may have to be cleaned manually after testing through the command:

puma-aaNNN $ rose suite-clean