UKCA Chemistry and Aerosol Tutorials: Things to know before you start the vn11.8 practicals

From UKCA

UKCA Chemistry and Aerosol Tutorials at vn11.8

You will need to have the details of your MOSRS account and VM connection details to hand.

Before you begin the UKCA Tutorials

You should be comfortable using a GNU/Linux operating system, including using and navigating around using a terminal. You should also be familiar with programming, ideally Fortran. You do NOT need to be familiar with using PUMA, Monsoon2, or ARCHER2.

For these tutorials it is helpful if you familiar with FCM, and training on this can be found here (with further information on useful commands below):

http://cms.ncas.ac.uk/wiki/Fcm

The UM FCM tutorial that is linked from this page uses the old user interface (the "UMUI"), so will be less relevant here.

You should also familiarise yourself with Rose & Cylc, and more information on these can be found here:

http://cms.ncas.ac.uk/wiki/RoseCylc

The Met Office provide some on-line UM specific Rose training here:

https://code.metoffice.gov.uk/doc/um/latest/um-training/index.html (MOSRS password required)

Accounts

To run these tutorials you will need


which will then allow you to run the tutorials in a Virtual Machine.

Note that using some special characters in your MOSRS password, particularly the % and £ symbols, have issues on the virtual machine used for training. If you believe your password may have issues you can request a password reset by emailing Scientific_Partnerships@metoffice.gov.uk, although please be aware that it may take some time to process this request.

Developing code for the UM

During this tutorial you will be creating tickets on the Met Office Science Repository Service, and making code changes to UM branches. Please note that this is a live system, used by scientists all over the world to develop their code. However, please don't worry about breaking anything, as there are practices in place to prevent this.

For more information about developing for the UM, please see the Working Practices for UM Development with Rose, FCM and trac.

Useful commands

These fcm commands are useful when working with UM branches and Rose suites. You may need to have cached your MOSRS account for these commands to work correctly, e.g. by running

mosrs-cache-password

which you will need to do usually once per day.

fcm info

This gives you basic information about your suite/branch, which is useful for commands later:

vagrant@vagrant:~/roses/u-ca021$ fcm info
Path: .
Working Copy Root Path: /home/vagrant/roses/u-ca021
URL: https://code.metoffice.gov.uk/svn/roses-u/c/a/0/2/1/trunk
Relative URL: ^/c/a/0/2/1/trunk
Repository Root: https://code.metoffice.gov.uk/svn/roses-u
Repository UUID: 7d42c781-efc1-4492-b88e-e6facad7bc31
Revision: 183843
Node Kind: directory
Schedule: normal
Last Changed Author: lukeabraham
Last Changed Rev: 179292
Last Changed Date: 2020-12-01 09:25:07 +0000 (Tue, 01 Dec 2020)
vagrant@vagrant:~/src/vn11.8_UKCA_Tutorial_Solns$ fcm info
Path: .
Working Copy Root Path: /home/vagrant/src/vn11.8_UKCA_Tutorial_Solns
URL: https://code.metoffice.gov.uk/svn/um/main/branches/dev/lukeabraham/vn11.8_UKCA_Tutorial_Solns
Relative URL: ^/main/branches/dev/lukeabraham/vn11.8_UKCA_Tutorial_Solns
Repository Root: https://code.metoffice.gov.uk/svn/um
Repository UUID: 0462af51-e8cd-401e-9b71-8f0b910fc1c1
Revision: 95777
Node Kind: directory
Schedule: normal
Last Changed Author: lukeabraham
Last Changed Rev: 94453
Last Changed Date: 2021-01-18 14:11:28 +0000 (Mon, 18 Jan 2021)

If you are using a system with an fcm mirror, e.g. ARCHER2 or Monsoon2, you will need to wait a few minutes after committing your changes before trying to compile them. This is not necessary on the VM, but for these HPC systems, the command

[09:51:21 hadlk@xcslc0 ~]$ fcm info fcm:um.xm
Path: main
URL: file:///home/d04/fcm/srv/svn/um.xm/main
Relative URL: ^/main
Repository Root: file:///home/d04/fcm/srv/svn/um.xm
Repository UUID: 2ba80990-f612-11e7-84a5-05deda76c84b
Revision: 95779
Node Kind: directory
Last Changed Author: stevewardle
Last Changed Rev: 95779
Last Changed Date: 2021-02-12 09:47:17 +0000 (Fri, 12 Feb 2021)

will tell you the last time the mirror was synced. You must ensure that the Last Changed Rev is equal to or greater than the revision number of your last commit.

fcm diff

This command can be used to tell you the differences from

  1. Your current working changes and the last committed revision
  2. Two different revisions that have been committed, from either the same suite/branch or different suite/branches

Here we look at changes in a working copy.

vagrant@vagrant:~/src/vn11.8_UKCA_Tutorial_Solns$ fcm diff
Index: src/atmosphere/UKCA/ukca_constants.F90
===================================================================
--- src/atmosphere/UKCA/ukca_constants.F90	(revision 95777)
+++ src/atmosphere/UKCA/ukca_constants.F90	(working copy)
@@ -345,7 +345,7 @@
 
 ! Additions for UKCA Tutorial
 REAL, PARAMETER :: C_ALICE      = 1.0000
-REAL, PARAMETER :: C_BOB        = 1.0000
+REAL, PARAMETER :: C_BOB        = 2.0000
 
 !     molecular masses in g/mol of emitted species,
 !     for budget calculations

You can also add the -g flag which will open the differences in a separate graphical viewer, usually tkdiff on the VM, although it may be xxdiff on other platforms.

Here we look at differences between two different revisions of the same branch. Note that the URL used is the same as that found with the fcm info command.

vagrant@vagrant:~/src/vn11.8_UKCA_Tutorial_Solns$ fcm diff https://code.metoffice.gov.uk/svn/um/main/branches/dev/lukeabraham/vn11.8_UKCA_Tutorial_Solns@94072 https://code.metoffice.gov.uk/svn/um/main/branches/dev/lukeabraham/vn11.8_UKCA_Tutorial_Solns@94120
Index: src/atmosphere/UKCA/ukca_chem_master.F90
===================================================================
--- src/atmosphere/UKCA/ukca_chem_master.F90	(revision 94072)
+++ src/atmosphere/UKCA/ukca_chem_master.F90	(revision 94120)
@@ -97,7 +97,7 @@
 INTEGER, PARAMETER :: n_chch_master = 329 ! number of known species
 INTEGER, PARAMETER :: n_het_master  =  18 ! number of heterogeneous reactions
 INTEGER, PARAMETER :: n_dry_master  = 150 ! number of dry deposition reactions
-INTEGER, PARAMETER :: n_wet_master  = 106 ! number of wet deposition reactions
+INTEGER, PARAMETER :: n_wet_master  = 107 ! number of wet deposition reactions
 INTEGER, PARAMETER :: n_bimol_master= 855 ! number of bimolecular reactions
 INTEGER, PARAMETER :: n_ratj_master = 158 ! number of photolysis reactions
 INTEGER, PARAMETER :: n_ratt_master =  76 ! number of termolecular reactions
@@ -847,7 +847,7 @@
 ! 272
 chch_t1(272,'ALICE     ',1,'TR        ','          ',1,0,st,0,0,111),          &
 ! 273
-chch_t1(273,'BOB       ',1,'TR        ','          ',0,0,st,0,0,111)           &
+chch_t1(273,'BOB       ',1,'TR        ','          ',0,1,st,0,0,111)           &
  /)
 
 ! ----------------------------------------------------------------------
@@ -2058,7 +2058,10 @@
 (/2.30e+02,4.90e+03,0.00e+00,0.00e+00,0.00e+00,0.00e+00/),cs,0,0,107),         &
 ! WD: 96. Using MeOH Henry's Law Coeffs
 wetdep( 96, 'ARNOH17   ',                                                      &
-(/2.30e+02,4.90e+03,0.00e+00,0.00e+00,0.00e+00,0.00e+00/),cs,0,0,107)          &
+(/2.30e+02,4.90e+03,0.00e+00,0.00e+00,0.00e+00,0.00e+00/),cs,0,0,107),         &
+!    UKCA Tutorial: wet deposition of BOB
+wetdep(97,'BOB       ',                                                        &
+(/0.21e+06,0.87e+04,0.20e+02,0.00e+00,0.00e+00,0.00e+00/),st,0,0,107)          &
 /)
 
! ---------------------------------------------------------------------

As Rose suites are also held in fcm branches, the same commands work with these. Here we diff the most recent revision of u-ca021 with a particular revision of u-ca024.

vagrant@vagrant:~/roses/u-ca021$ fcm diff https://code.metoffice.gov.uk/svn/roses-u/c/a/0/2/1/trunk https://code.metoffice.gov.uk/svn/roses-u/c/a/0/2/4/trunk@180902
Index: app/um/rose-app.conf
===================================================================
--- app/um/rose-app.conf	(.../1/trunk)	(revision 186431)
+++ app/um/rose-app.conf	(.../4/trunk)	(revision 180902)
@@ -3634,6 +3634,14 @@
 tim_name='T3HMN'
 use_name='UPA'
 
+[namelist:umstash_streq(34010_c4b8f1ad)]
+dom_name='DALLTH'
+isec=34
+item=10
+package=
+tim_name='T3HMN'
+use_name='UPA'
+
 [namelist:umstash_streq(34106_ebb7bb67)]
 dom_name='DALLTH'
 isec=34
@@ -3754,6 +3762,14 @@
 tim_name='T3HMN'
 use_name='UPA'
 
+[namelist:umstash_streq(51010_5e3d3bef)]
+dom_name='DP27CCM'
+isec=51
+item=10
+package=
+tim_name='T3HMN'
+use_name='UPA'
+
 [namelist:umstash_streq(51999_8b73e62f)]
 dom_name='DP27CCM'
 isec=51
Index: rose-suite.info
===================================================================
--- rose-suite.info	(.../1/trunk)	(revision 186431)
+++ rose-suite.info	(.../4/trunk)	(revision 180902)
@@ -1,4 +1,4 @@
-description=Copy of u-bz223/trunk@177854
+description=Copy of u-ca021/trunk@179292
 owner=lukeabraham
 project=ukca
-title=vn11.8 N48L38 GA7-Strattrop GNU/VM 1x2: UKCA Training
+title=vn11.8 N48L38 GA7-Strattrop GNU/VM 1x2: UKCA Training Solutions

fcm status

This command tells you the difference files that have been changed in your working copy compared to the last committed revision.

vagrant@vagrant:~/src/vn11.8_UKCA_Tutorial_Solns$ fcm status
M       src/atmosphere/UKCA/ukca_constants.F90

fcm revert

USE THIS COMMAND WITH CARE

You can undo changes in a working copy file, putting them back to the last committed version, with this command:

vagrant@vagrant:~/src/vn11.8_UKCA_Tutorial_Solns$ fcm revert src/atmosphere/UKCA/ukca_constants.F90
Reverted 'src/atmosphere/UKCA/ukca_constants.F90'
vagrant@vagrant:~/src/vn11.8_UKCA_Tutorial_Solns$ fcm diff
vagrant@vagrant:~/src/vn11.8_UKCA_Tutorial_Solns$ 

Be careful with this command, as it will remove all the changes that you have made in that file.

UKCA Training Suite

Machine UM Version/Configuration Suite ID Owner Comments
vm vn11.8 N48L38 GNU Compiler u-ca021 lukeabraham 1x2 decomposition

These suites are based on the um_n48_ukca_eg app, which is available in the MetUM trunk (in rose-stem/app/). A ticket giving more details on these (and the code changes) can be found at: um:#3578.

NCAS-provided VMs

If you have been provided a VM for the NCAS training course, Iris will have been installed for you using miniconda, and can be accessed by first using the command

pyterm

The command

pylab 

has also been provided to give you an iPython environment which has been defined with the following alias

ipython --pylab --logfile=ipython-`date +\"%Y%m%d-%H%M%S\"`.py"

so that the standard pylab additions have been included, along with logging of your python commands. You can then use

import iris

within this ipython environment.

Xconv has also been installed, and can be accessed with the

xconv

or

convsh

commands as required to open UM output files, e.g.

xconv atmosa.pa19810901_00

etc.

Worked solutions and sample output can be found in the

Tutorials

directory that is in your home/ directory.

Additional Requirements when using a personal Virtual Machine

YOU ONLY NEED THESE ADDITIONAL REQUIREMENTS IF YOU ARE SETTING UP THE VIRTUAL MACHINE YOURSELF. DO NOT RUN ANY OF THESE COMMANDS IF YOU HAVE BEEN PROVIDED A VM AS PART OF A TRAINING EVENT.

You will be using a version of the Met Office provided Virtual Machine to run the VM, provided here:

https://github.com/metomi/metomi-vms

It will have been set-up for you, but if you are making your own VM you should follow the instructions in UMDP X10. There is also some (possibly out of date) information that can be found here. These tutorials have been documented on the VM in Abraham et al. (2018)[1].

To to run a simple UKCA suite in the VM you need at least 6GB of RAM to work successfully.

References

  1. Abraham, N. L., Archibald, A. T., Cresswell, P., Cusworth, S., Dalvi, M., Matthews, D., Wardle, S., and Whitehouse, S.: Using a virtual machine environment for developing, testing, and training for the UM-UKCA composition-climate model, using Unified Model version 10.9 and above, Geosci. Model Dev., 11, 3647–3657, https://doi.org/10.5194/gmd-11-3647-2018, 2018.

Tutorial Data

To be able to do Tutorials 5 and 10 you will need emissions data and python scripts. These files, as well as worked solutions for all the Rose and code changes can be obtained by running the following command from the home/ directory of your VM:

wget -q --show-progress http://gws-access.ceda.ac.uk/public/ukca/UKCA_Tutorial_vn118.tgz

There is 915MB to download, so it might take some time.

Once the download has finished, you can extract the archive by running

tar -xzvf UKCA_Tutorial_vn118.tgz

This will make a directory called Tutorials/ containing 1.5GB of files. If you cd into this you can check that the files have not been corrupted by running the following command:

md5sum --status -c vn118.md5

Xconv

You will need to download Xconv (xconv1.94) from here:


You can download this by

wget https://gws-access.jasmin.ac.uk/public/ukca/xconv1.94_linux_x86_64.tar.gz

Download it to your $HOME/bin on the VM, cd into this directory, tar -zxvf the tar-ball, and then

ln -s xconv1.94 xconv
ln -s xconv1.94 convsh

Iris

There is an install-iris script provided, but you will need to set-up modules yourself to be able to use it properly. The anaconda install breaks Rose if put in your PATH, however, there is an alias

pyterm

which will open a new terminal with all the anaconda python packages in its $PATH. This will allow you to use Rose in one terminal and Iris in another. You should edit the install-iris to have the line

conda install -y -c conda-forge python=3.6 iris=1.13 ipython mo_pack numpy=1.15

instead of the existing line, otherwise the scripts will not work correctly.

I have found a handy way to use python is to use ipython with the following arguments

ipython --pylab --logfile=ipython-`date +\"%Y%m%d-%H%M%S\"`.py

I have aliased this to

pylab

in my .bashrc. The

--pylab 

sets-up a MatLab-type environment (numpy, scipy, matplotlib all loaded using standard shortcuts), and

--logfile=ipython-`date +\"%Y%m%d-%H%M%S\"`.py

means that all commands are saved to a file of the format

ipython-YYMMDD-HHMMSS.py

Scripts

There are a number of scripts provided for these tutorials. These can be found on GitHub here:


Written by Luke Abraham 2020.