Difference between revisions of "UKCA Chemistry and Aerosol vn11.8 Tutorial 1"

From UKCA
Line 50: Line 50:
 
The suite will now copy and checkout to your <code>/home/'''$USER'''/roses</code> directory, into a sub-directory that is the same as the SUITE-ID. You should change into this directory now using '''cd''', e.g.
 
The suite will now copy and checkout to your <code>/home/'''$USER'''/roses</code> directory, into a sub-directory that is the same as the SUITE-ID. You should change into this directory now using '''cd''', e.g.
   
cd /home/vagrant/roses/u-ab123
+
cd roses/u-ab123
   
 
then open the suite by typing
 
then open the suite by typing
Line 62: Line 62:
 
in the terminal.
 
in the terminal.
   
If you are using your own VM, times will vary depending on the specifications of the host. It could take about 10 minutes to compile the code (''fcm_make'') for the first time (and about 1-2 minutes when recompiling), followed by about 1 minute to run the reconfiguration step (''recon''), and then about 12 minutes to run the UM itself (''atmos''). If you have been provided a VM the compile should take 1-2 minutes.
+
If you are using your own VM, times will vary depending on the specifications of the host. It could take about 10 minutes to compile the code (''fcm_make'') for the first time (and about 1-2 minutes when recompiling), followed by about 1 minute to run the reconfiguration step (''recon''), and then around 10 minutes to run the UM itself (''atmos''). If you have been provided a VM the compile should take 1-2 minutes.
   
 
When the suite has finished successfully it will then become blank with the message '''stopped with 'succeeded'''' in the bottom-left corner.
 
When the suite has finished successfully it will then become blank with the message '''stopped with 'succeeded'''' in the bottom-left corner.

Revision as of 10:35, 28 January 2021

UKCA Chemistry and Aerosol Tutorials at vn11.8

Difficulty EASY
Time to Complete Under 1 hour

Remember to run mosrs-cache-password in your Rose suite and UM branch terminals.

Copying and Running an Existing Rose Suite

Machine UM Version/Configuration Suite ID
vm vn11.8 N48L38 GNU Compiler u-ca021

If you are using your own Met Office Virtual Machine, you will need to login to the VM, e.g.

vagrant ssh

If you are using a VM provided for you, you should connect via X2Go or via a Terminal.

You should then be asked for your Met Office Science Repository Service (MOSRS) password. Once you enter this, if this is the first time you are logging in, you will also be asked for your MOSRS username. This will likely be your name in lower case, and is not your username on the VM.

Then copy the rose suite by:

rosie copy u-ca021
GCylc running on the VM via X2Go.

This will open an editor containing text similar to this:

description=Copy of u-ca021/trunk@179292
owner=lukeabraham
project=ukca
title=vn11.8 N48L38 GA7-Strattrop GNU/VM 1x2: UKCA Training

# Make changes ABOVE these lines.
# The "owner", "project" and "title" fields are compulsory.
# Any KEY=VALUE pairs can be added. Known fields include:
# "access-list", "description" and "sub-project".

You can edit the title to make it more descriptive if you like. Then save (e.g. Ctrl-s) and close the file (e.g. Ctrl-q). The press y on the keyboard when prompted and press return.

The suite will now copy and checkout to your /home/$USER/roses directory, into a sub-directory that is the same as the SUITE-ID. You should change into this directory now using cd, e.g.

cd roses/u-ab123

then open the suite by typing

rose edit &

and pressing return. You can either run the suite by pressing the play button that looks like a triangle to the top right of the options bar, or by typing

rose suite-run

in the terminal.

If you are using your own VM, times will vary depending on the specifications of the host. It could take about 10 minutes to compile the code (fcm_make) for the first time (and about 1-2 minutes when recompiling), followed by about 1 minute to run the reconfiguration step (recon), and then around 10 minutes to run the UM itself (atmos). If you have been provided a VM the compile should take 1-2 minutes.

When the suite has finished successfully it will then become blank with the message stopped with 'succeeded' in the bottom-left corner.

Version Control

Rose suites are all held under version control, using fcm. When making changes to a suite, you will need to save it before you can run the suite. Once you are happy with the settings, you can also commit these changes back to the repository - to do this change directory to the

/home/$USER/roses/[SUITE-ID]

and then type

fcm commit

a text editor will then open, and you should type a short message describing what the changes you have made do. You should then close the editor and type y in the terminal. It is recommended that you commit frequently (even on configurations that aren't working) as this protects you against mistakes and accidental deletions etc.

These suites can be viewed on the SRS here: https://code.metoffice.gov.uk/trac/roses-u (password required)

It is recommended that you commit your suites regularly.

Output Directory Structure

The output directory structure of rose suites are rather complex. The schematic on the right gives a broad overview of the general structure. Everything can be found within the [SUITE-ID] (e.g. u-ca021 etc.) directory, which can be found within your $HOME/cylc-run directory. On the VM, everything in held in this single place, but things are more complicated when using other machines, such as PUMA & ARCHER2, as some files are held on the ARCHER /home filesystem, some on the /work filesystem, and some files are synced back to PUMA as well. On Monsoon2, everything is held on the same machine, but with some files in /projects and some in /home.

Within this directory there are several directories, including:

  • log: a symbolic link to a directory called something like log.20171214T141332Z, which contains all the job.out (containing output from UM WRITE statements from the umPrint subroutine) and job.err files etc., as well as the script used to run the job.
  • work: this contains the directories used when the job actually runs. Real-time output will be sent to files here (held in a pe_output directory. Sometimes model output will also be here. On ARCHER this directory exists on the /work filesystem.
  • share: This isn't used by output from the UKCA Training Suite, but for climate jobs model output will usually be held in this directory. On ARCHER this directory exists on the /work filesystem.

Within these directories there will be many sub-directories. Some of these will be named from the cycle-point (labelled [CYCLE] in the graphic. For the UKCA Training Suite this will be 1, but under normal climate operation this will be a date string, e.g. 19880901T0000Z. The sub-directories will (eventually) be named after the app (labelled by [JOB NAME] in the table below) that the output is from, e.g. fcm_make, recon, atmos etc.

Files General Path Example
Most recent job.out files vm: /home/vagrant/cylc-run/[SUITE-ID]/log/job/1/[JOB NAME]/NN /home/vagrant/cylc-run/u-as297/log/job/1/atmos/NN
Processor output (while running) vm: /home/vagrant/cylc-run/[SUITE-ID]/work/1/[JOB NAME]/pe_output /home/vagrant/cylc-run/u-as297/work/1/atmos/pe_output
Output files in 64-bit fieldsfile format vm: /home/vagrant/cylc-run/[SUITE-ID]/work/1/atmos /home/vagrant/cylc-run/u-as297/work/1/atmos

Notes

  • The job.out files can also be viewed through the Gcylc GUI right-click menu from each job.
  • A handy command to check the progress of the atmos job is
vm: tail -1000f atmos.fort6.pe0 | grep Atm_Step
  • Other suites may put the output fieldsfiles into a different location, e.g.
/work/n02/n02/$USER/cylc-run/[SUITE-ID]/share/data/History_Data
and there are also post-processing settings that can be used to copy data to the RDF or JASMIN as the suite runs.
  • Output fieldsfiles can have various naming conventions. For these suites, they will be atmosa.pa19810901_00, but more generally they are likely to be of the form
[SHORT SUITE-ID]a.p[abcdefghijkmsyx]YYYYMMDD
e.g. ag308a.pk19880901
[SHORT SUITE-ID]a.p[abcdefghijkm]YYYYmon
e.g. ak468a.pe1989oct
  • Climate-mean files (.pm) will often also have seasonal (.ps), annual (.py), and decadal (.px) equivalents.

Whilst this suite is running, take a look at Tutorial 2: Exploring Rose.

Viewing Output

Example output from the UKCA training suite.

For more detailed plotting, tools such as the Iris and cf-python libraries can be used to view UM file formats directly. However, for quick viewing, Xconv is a very useful tool. Information on how to install Xconv on the VM can be found here.

To view these files, do

xconv atmosa.pa19810901_00

As well as viewing files, you can use Xconv to convert these files to netCDF, by filling in the Output file name: box (e.g. foo.nc), and then clicking convert. If no path is defined, this will save the file in the same directory that you opened Xconv from.

Example output from the UKCA training suite can be found at

Checklist

List suites using rosie go
Copy suites using the right-click menu
Run suites using the play button


Tutorial 2


Written by Luke Abraham 2020