Evaluation suite2

UKCA Evaluation Suite - Version 2

The UKCA Evaluation suite is a collection of basic assessment methods being used at various partner institutions. The package can be divided largely into two categories:

• Tropospheric chemistry evaluation
• Stratospheric chemistry and dynamics evaluation

The evaluation suite can currently analyse the outputs from UM-UKCA configurations against specific observation datasets. To use the evaluation suite, the model output needs to be in the form of 12 x monthly mean pp files (UM pm stream) conforming to the UM pp file naming format (jobida.somename.pp). The Stratospheric as well as Tropospheric Chemistry suites can carry out multi-annual meaning if data for more than one year are specified as input. Each type of analysis requires a specified list of fields to be present in the pp files (See list below). These fields should be available in the Release UKCA and UKESM suites (tagged as package='UKCA diagnostics').

The UKCA Evaluation suite V2 is available on the Met Office Desktop, JASMIN (lotus) and ARCHER2 (under development) systems.

---

Tropospheric Chemistry package

This currently performs the following analysis/ evaluations:

• CO against CMDL obs
• ClO against MLS data
• CO, HNO3, NOx against Emmons et al dataset
• OH using Lawrence etal (1991) method, vs ACCMIP and Parta et al Reference values.
• O3 against Tilmes ozonesonde data at multiple locations
• Ox budget (sources/sinks, deposition)
• OH/CH4 ratio and CH4 lifetime vs ACCMIP multi-model values
• Profiles of O3,HNO3,NO2,H2O2,water vap against ACE,UARS,.. data
• Age of Air against SF6 obs
• Tropospheric O3 and NO2 columns
• Lightning NOx : total N and zonal mean

The tool will automatically carry out multi-annual meaning on-the-fly if more than 12 (and a multiple of 12) files are detected.
Usage:

A. Met Office desktop:
/home/users/mohit.dalvi/ukca_eval/camb_chem/eval_tropchem_spice.py -i <ppfiles> [-s STASHlist] [-m trmap]

Options

-h, --help	show this help message and exit
-i	Required: ppfiles (12) from the year to analyse -full path-
-s STASHLIST	Optional: STASHcodes list, e.g. if using pre-vn8.5 output (diags in Section 34 vs 50)
-m TRMAPS	Optional :Var<->STASH mapping file, e.g. if using pre-vn8.5 output
-f SCALE_FAC	Optional : Flux multiplication factor, to account for difference in UM:UKCA call frequency (default=3.0 for 1 UM: 3 UKCA timesteps)

B. JASMIN
On jasmin-sciX servers (run on Lotus)

/home/users/mcdalvi/ukca_eval/camb_chem/eval_tropchem_jas.py -a <account> -i <ppfiles> [-s STASHlist] [-m trmap]

-a (account) This is a required option for JASMIN/Lotus2 through which the account to use for the run is specified. The accounts/ projects available to an user can be listed by running useraccounts command on the sciX servers.

This will launch a job on the Lotus2 system.

The plots will be created in: ./Plots_TC/<jobid>/

Pre-vn8.5 Model output
Add arguments: -s <mohit home>/eval_v2/camb_chem/stash_pre85.lst -m <mohit home>/eval_v2/camb_chem/ukca_stdname_pre85

---

Stratospheric Chemistry (toms_haloe) package

The suite currently evaluates:

• CH4, N2O, O3, against HALOE dataset
• O3 against a dataset compiled at NIWA for CCMVal
• Column O3 against TOMS

The HALOE/ NIWA comparisons are currently for the months of January and July, while the Column data is based on the whole year.

Usage:

Met Office Desktop:- /home/users/mohit.dalvi/ukca_eval/toms_haloe/comp_toms_haloe_spice.py < model pp files >
The names of all the model output files need to be passed, i.e. at least 12 for an year (note: wild cards can be used).
The tool will automatically carry out multi-annual meaning on-the-fly if more than 12 (and a multiple of 12) files are detected.

JASMIN: (launch from sciX servers onto Lotus) /home/users/mcdalvi/ukca_eval/toms_haloe/comp_toms_haloe_jas.py -a <account> < model pp files >.

The -a (account) argument has to be the first one, followed by list of files

Outputs will be created in : ./Plots_TH/<jobid>/

---

Combining and Comparing plots

The Stratospheric Chemistry package currently produces plots in postscript format, while the Tropospheric package produces plots as separate PDF files. To make it easier to review all the plots, and to compare two runs (e.g. Control and Expt) side-by-side a script is made available as part of the UKCA evaluation suite.

The script is expected to be run in the directory that contains the Plots_TC and Plots_TH output folders from above runs.

(mohit-home)/ukca_eval/bin/merge_eval_outputs.sh <job1> [job2] [--dir1 jobdir1] [--dir2 jobdir2]

Options

job1	Required
job2	Optional - if comparing two runs
--dir1[--dir2]	Optional: The path(s) containing the Plots_TC and Plots_TH folders from job1 (dir2 for job2 output)

On completion, the script produces an ukca_eval_(job1).pdf for single job run, or a
ukca_eval_job1_job2.pdf file with each plot from job2 following the corresponding plot from job1, so the plots can be perused by displaying the PDF in Dual, (continous, Odd pages left) mode.
---

ARCHER

The scripts have been modified slightly to work on ARCHER. They are designed such that output from a UM vn10.6 job or above will not require any additional arguments, other than the location of the *.pp files. Also, all Stratospheric output is saved as .pdf.

While you can use the login nodes for this, you can also log-in to the post-processing nodes by

ssh -Y espp1

Required modules

To be able to use Iris (required for both Stratospheric and Tropospheric packages), you will need to:

module load anaconda/2.2.0-python2

To run the Tropospheric chemistry package you will need to load R by

module load R/3.2.3

and load the required netCDF libraries by

module load cray-netcdf-hdf5parallel/4.4.1.1

Running the packages

For UM versions vn10.6 and above you will just need to:

• Tropospheric Chemistry: /work/n02/n02/ukca/Eval/eval_v2/camb_chem/eval_tropchem.py -i /path/to/pp/files/*.pp [--eval_only]
• Stratospheric Chemistry: /work/n02/n02/ukca/Eval/eval_v2/toms_haloe/compare_toms_haloe.py /path/to/pp/files/*.pp

You can use evince to view the outputted .pdf files.

If you need to run these on pre-vn10.6 versions, the equivalent to <mohit home> is /work/n02/n02/ukca/Eval for the paths to particular STASH maps etc.

The Stratospheric chemistry package takes a few minutes for a single years-worth of data. The Tropospheric chemistry package will take about 50 minutes to extract the data to netCDF and then an additional 15-20 minutes to produce the plots.

Example Data

Example data from vn10.9 can be found at

/work/n02/n02/ukca/Eval/ExampleData/u-as022

and an empty directory with the data already extracted to netCDF can be found here:

/work/n02/n02/ukca/Eval/WorkedExample/u-as022

This directory can be rsync-d to a working directory and the evaluation suite can be run using the --eval_only command which will save some time.

Required Chemistry Diagnostics

The UKCA chemistry evaluation packages require the following diagnostics (STASH section/item numbers from vn10.3 onwards):

STASH Section	STASH Item	STASH Name
0	010	SPECIFIC HUMIDITY AFTER TIMESTEP
0	408	PRESSURE AT THETA LEVELS AFTER TS
16	004	TEMPERATURE ON THETA LEVELS
30	451	Pressure at Tropopause Level
30	453	Height at Tropopause Level
34	001	O3 MASS MIXING RATIO AFTER TIMESTEP
34	002	NO MASS MIXING RATIO AFTER TIMESTEP
34	004	NO2 MASS MIXING RATIO AFTER TIMESTEP*
34	007	HONO2 MASS MIXING RATIO AFTER TSTEP
34	009	CH4 MASS MIXING RATIO AFTER TSTEP
34	010	CO MASS MIXING RATIO AFTER TSTEP
34	042	ClO MASS MIXING RATIO AFTER TSTEP
34	049	N2O MASS MIXING RATIO AFTER TIMESTEP
34	081	OH MASS MIXING RATIO AFTER TIMESTEP
34	150	AGE OF AIR IN SECONDS
34	992	HCl MASS MIXING RATIO AFTER TSTEP**
34	996	NO2 MASS MIXING RATIO AFTER TIMESTEP**
50	001	Ox PROD: HO2+NO
50	002	Ox PROD: MeOO+NO
50	003	Ox PROD: NO+RO2
50	004	Ox PROD: OH+INORGANIC ACID
50	005	Ox PROD: OH+ORGANIC NITRATE
50	006	Ox PROD: ORGANIC NITRATE PHOTOLYSIS
50	007	Ox PROD: OH + PAN-TYPE REACTIONS
50	011	Ox LOSS: O(1D)+H2O
50	012	Ox LOSS: MINOR LOSS REACTIONS
50	013	Ox LOSS: HO2+O3
50	014	Ox LOSS: OH+O3
50	015	Ox LOSS: O3+ALKENE
50	016	Ox LOSS: N2O5+H2O
50	017	Ox LOSS: NO3 CHEMICAL LOSS
50	021	Ox BUDGET: O3 DRY DEPOSITION (3D)
50	022	Ox BUDGET: NOy DRY DEPOSITION (3D)
50	031	Ox BUDGET: NOy WET DEPOSITION (3D)
50	041	RXN FLUX: OH+CH4 (CH4 LIFETIME) TROP
50	051	STE: O3
50	061	AIR MASS DIAGNOSTIC (TROP ONLY)
50	062	TROPOSPHERIC MASK
50	063	AIR MASS DIAGNOSTIC (WHOLE ATMOS)
50	081	Lightning NOx Emissions (3D)
50	219	Ozone column in Dobson Units

Notes:

* NO2 is not available in s34i004 in StratTrop/CheST

** HCl is only available in s34i992 in StratTrop/CheST

** NO2 is only available in s34i996 in StratTrop/CheST

If you are extracting these from MASS, you can use the following query file with the moo select command:

begin
   year=[1999..2008]
   stash=(10, 408, 16004, 30451, 30453, 34001, 34002, 34004, 34007, 34009, 34010, 34042, 34049, 34081, 34150, 34992, 
     34996, 50001, 50002, 50003, 50004, 50005, 50006, 50007, 50011, 50012, 50013, 50014, 50015, 50016, 50017, 50021, 
     50022, 50031, 50041, 50051, 50061, 50062, 50063, 50081, 50219)
end

Usage : moo select <query file> moose:crum/<suite-id>/apm.pp [output path or ./]

Note: If you copy the above and paste into you query file directly, align all the stashcodes in a single continuous line as line-breaks are not permitted for stash=. You might also need to run dos2unix <filename>.