Difference between revisions of "Evaluation suite2"

From UKCA
Line 1: Line 1:
 
=== UKCA Evaluation Suite - Version 2 ===
 
=== UKCA Evaluation Suite - Version 2 ===
   
==== This is the description for Version 2. Instructions for Version-2 can be found [[Evaluation_Suite | here]] ====
+
==== This is the description for Version 2. Instructions for Version-1 can be found [[Evaluation_Suite | here]] ====
   
 
The UKCA Evaluation suite is a collection of basic assessment methods being used at various partner institutions. The package can be divided largely into three categories:<br/>
 
The UKCA Evaluation suite is a collection of basic assessment methods being used at various partner institutions. The package can be divided largely into three categories:<br/>

Revision as of 12:24, 2 June 2017

UKCA Evaluation Suite - Version 2

This is the description for Version 2. Instructions for Version-1 can be found here

The UKCA Evaluation suite is a collection of basic assessment methods being used at various partner institutions. The package can be divided largely into three categories:

  • Tropospheric chemistry evaluation
  • Stratospheric chemistry and dynamics evaluation
  • Aerosol chemistry and processes evaluation --Not yet available for V2

The evaluation suite can currently analyse the outputs from UM-UKCA configurations against specific observation datasets. To use the evaluation suite, the model output needs to be in the form of 12 x monthly mean pp files (UM pm stream) conforming to the UM pp file naming format (jobida.somename.pp). The Stratospheric as well as Tropospheric Chemistry suites can carry out multi-annual meaning if data for more than one year are specified as input. Each type of analysis requires a specified list of fields to be present in the pp files and there are UM STASH macro handedits to ensure that these fields are requested in the output.

For the Tropospheric and Stratospheric Chemistry part of evaluation, these fields can be added to your job by using the hand_edit:

  • /home/h02/hadzm/umui_jobs/hand_edits/vn8.2/add_ukca_eval1_diags_lxx.ed (where xx is the number of vertical levels) on the Met Office systems and
  • /home/mdalvi/umui_jobs/hand_edits/vn8.2/add_ukca_eval1_diags_lxx.ed on the Puma server.

For UM versions vn8.5-8.6, use the hand_edits:

  • /home/h02/hadzm/umui_jobs/hand_edits/vn8.5/add_ukca_eval_diags_vn85_l85.ed on the Met Office systems and
  • /home/mdalvi/umui_jobs/hand_edits/vn8.5/add_ukca_eval_diags_vn85_l85.ed on the Puma server.

For Rose (Um9.0+) configurations, the requests will have to be added manually or inherited from a base/ Release suite.

The UKCA Evaluation suite V2 is available on the Met Office Desktop, MONSooN postprocessor and JASMIN (sci1,lotus).


Tropospheric Chemistry package

This currently performs the following analysis/ evaluations:

  • CO against CMDL obs
  • ClO against MLS data
  • CO, HNO3, NOx against Emmons et al dataset
  • OH using Lawrence etal (1991) method, vs ACCMIP and Parta et al Reference values.
  • O3 against Tilmes ozonesonde data at multiple locations
  • Ox budget (sources/sinks, deposition)
  • OH/CH4 ratio and CH4 lifetime vs ACCMIP multi-model values
  • Profiles of O3,HNO3,NO2,H2O2,water vap against ACE,UARS,.. data
  • Age of Air against SF6 obs


The tool will automatically carry out multi-annual meaning on-the-fly if more than 12 (and a multiple of 12) files are detected.
Usage:

A. Met Office desktop:
/home/h02/hadzm/eval_v2/camb_chem/eval_tropchem_spice.py -i <ppfiles> [-s STASHlist] [-m trmap] [--eval_only] [--noclean] [--nocopy]

Options
-h, --help show this help message and exit
-i Required: ppfiles (12) from the year to analyse -full path-
-s STASHLIST Optional: STASHcodes list, e.g. if using pre-vn8.5 output (diags in Section 34 vs 50)
-m TRMAPS Optional :Var<->STASH mapping file, e.g. if using pre-vn8.5 output
-f SCALE_FAC Optional : Flux multiplication factor, to account for difference in UM:UKCA call frequency (default=3.0 for 1 UM: 3 UKCA timesteps)
--eval_only Optional: Only carry out Evaluation, skipping the extraction.
Useful when extract is ok but evaluation has previously failed.
--noclean Optional: Do not delete extracted NetCDF data after completion
--nocopy By default the input files will be copied temporarily to /scratch for processing. Use this option only if model output is in $DATADIR

B. MONSooN Postprocessor:
/home/mdalvi/eval_v2/camb_chem/eval_tropchem.py -i <ppfiles> [-s STASHlist] [-m trmap] [--eval_only] [--noclean]

Note: The performance on postproc01 varies, depending on the machine load.

C. JASMIN
On jasmin-sci1/sci2 or lotus

/home/users/mcdalvi/eval_v2/camb_chem/eval_tropchem_top.py -i <ppfiles> [-s STASHlist] [-m trmap] [--eval_only] [--noclean]

This will launch a job on the Lotus system.

The plots will be created in: ./Plots_TC/<jobid>/


Pre-vn8.5 Model output
Add arguments: -s <mohit home>/eval_v2/camb_chem/stash_pre85.lst -m <mohit home>/eval_v2/camb_chem/ukca_stdname_pre85


vn10.3 onwards
Some of the required fields are being output via different stash codes, hence to analyse output from UM10.3+ runs add the arguments:

-s <mohit home>/eval_v2/camb_chem/stash_eval_v103.lst -m <mohit home>/eval_v2/etc/ukca_stdname_vn103

vn10.6 onwards
In addition to stash changes above, an additional option needs to be set while analysing output from suites at vn10.6 and above, use the arguments:

-s <mohit home>/eval_v2/camb_chem/stash_eval_v103.lst -m <mohit home>/eval_v2/etc/ukca_stdname_vn103 -f 1.0

This is because the scaling factor for UM:UKCA timestep ratio no longer needs to be applied to reaction fluxes, as this is now implicitly handled by Stash while storing/meaning the data.



Stratospheric Chemistry (toms_haloe) package

The suite currently evaluates:

  • CH4, N2O, O3, against HALOE dataset
  • O3 against a dataset compiled at NIWA for CCMVal
  • Column O3 against TOMS

The HALOE/ NIWA comparisons are currently for the months of January and July, while the Column data is based on the whole year.

Usage:

MONSooN - /home/mdalvi/eval_v2/toms_haloe/compare_toms_haloe.py < model pp files -atleast 12>

The names of all the model output files need to be passed, i.e. at least 12 for an year (note: wild cards can be used).
The tool will automatically carry out multi-annual meaning on-the-fly if more than 12 (and a multiple of 12) files are detected.


Met Office Desktop:- /home/h02/hadzmeval_v2/toms_haloe/compare_toms_haloe.py < model pp files >
JASMIN: (sci1,2/lotus) /home/users/mcdalvi/eval_v2/toms_haloe/compare_toms_haloe.py < model pp files >.


Outputs will be created in : ./Plots_TH/<jobid>/