Testing Requirements for UKCA Code Changes

When making changes to UKCA code, additional testing will be required compared to the usual UM developments.

Please also see:

• UM Working Practices (MOSRS)
• UM Standard Jobs (MOSRS)

All changes will require a ticket on the UM trac pages on the Met Office Science Repository Service.

rose-stem

All UM developments require rose-stem to be used on a committed branch, using the developer group. For UKCA changes you should also use the ukca keyword, like this:

rose-stem --group=developer,ukca

Climate Configurations

If you are making changes to climate configurations two types of suites are provided for you:

Short testing suite

Details about the UKCA short testing suites are maintained here:

• Rose-UKCA suites (maintained by Mohit Dalvi)

This suite allows you to test your changes with logicals on and includes thread and bit-comparability tests, as well as a rigorous test. These jobs only run for 2 model days (6 hours for rigorous) and are useful for making initial developments.

As there are no StratTrop or CRI-Strat rigorous tests under rose-stem, so it is important that this is done using this suite as it fulfils the requirement to test new changes under rigorous.

You can link to the Rose Bush output on your ticket, and newer suites produce a trac.log file.

20-year AMIP suite

If you are introducing new code (either a science addition or a bugfix) that changes results in some way, you should also test using a 20-year AMIP simulation and assess whether or not your change produces large changes compared to the existing configuration. These changes may be absolutely fine, and significant differences from the control shouldn't prevent code being committed (unless these are unexpected).

Details of available suites can be found here:

• GA7.1 StratTrop suites

20-year controls are run at the Met Office and archived to MASS.

Once your suite has finished you should use the UKCA Evaluation Suite to produce a series of standard plots from your suite and compare these to equivalent plots from the control. A combined PDF document with the suites interleaved and viewed under 2-page view is usually best here.

You may also want to make use of valnote/autoassess or Maverick to produce a set of standard plots as well.

It is best to be in close contact with your SciTech Reviewer as they may request additional diagnostics/plots for the ticket.

Specific testing when adding new Jules surface tiles in the dry deposition

New surface tiles were added as an urgent ticket for UKESM1 work in UM:#1255 and UM:#1761.

However, there are some less common ways of setting up suites that were not fully tested at the time.

Bug fixes behind temporary logical switches were made in tickets UM:#4157 and UM:#4312 and at least one more fix will be required in UM:#4913.

Existing bugs were also fixed in ticket UM:#1250.

We would like to avoid a repeat of the bugs added in UKESM1 for future UKESM releases.

This is quite a niche task, so there are not currently specific tests for this. Please get in touch with Alan Hewitt and Gerd Folberth before working on a ticket to extend surface tiles in the dry deposition scheme. Also cc both of us into the ticket and add keyword dry_deposition please.

Temporary Logicals

If you need to make use of a temporary logical, the testing requirements are discussed here:

• Temporary UM Logicals

Note that you should AVOID using temporary logicals unless it is absolutely necessary.