Evaluation Suite


UKCA Evaluation suite

This is the description for Version 1. Instructions for Version-2 can be found here

The UKCA Evaluation suite is a collection of basic assessment methods being used at various partner institutions. The package can be divided largely into three categories: - Tropospheric chemistry evaluation - Stratospheric chemistry and dynamics evaluation - Aerosol chemistry and processes evaluation

The evaluation suite can currently analyse the outputs from UM-UKCA (N48 or N96) configurations against specific observation datasets. To use the evaluation suite, the model output needs to be in the form of 12 x monthly pp files (single year or multi-year means) conforming to the UM pp file naming format (jobida.somename.pp). Each type of analysis requires a specified list of fields to be present in the pp files and there are UM STASH macro handedits to ensure that these fields are requested in the output.

For the Tropospheric and Stratospheric Chemistry part of evaluation, these fields can be added to your job by using the hand_edit: /home/h02/hadzm/umui_jobs/hand_edits/vn8.2/add_ukca_eval1_diags_lxx.ed (where xx is the number of vertical levels) on the Met Office systems and /home/mdalvi/umui_jobs/hand_edits/vn8.2/add_ukca_eval1_diags_lxx.ed on the Puma server.

The UKCA Evaluation suite is currently available only on the Met Office Desktop and the MONSooN postprocessor.

(A) The Tropospheric Chemistry package currently performs the following analysis/ evaluations:

  • CO against CMDL obs
  • ClO against MLS data
  • CO, HNO3, NOx against Emmons et al dataset
  • OH using Lawrence etal (1991) method
  • O3 against Tilmes ozonesonde data at multiple locations
  • Ox budget (sources/sinks, deposition)
  • OH/CH4 ratio and CH4 lifetime
  • Profiles of O3,HNO3,NO2,H2O2,water vap against ACE,UARS,.. data
  • Age of Air against SF6 obs

Top-level script:/home/mdalvi/eval_v1/camb_chem/eval_tropchem.py. (Replace with /home/h02/hadzm for Linux desktop)

This can be launched by supplying the top-level script with path+name of any monthly mean pp file (from a set of 12), e.g.

/home/mdalvi/eval_v1/camb_chem/eval_tropchem.py /nerc/ukca/mdalvi/xhmag/xhmaga.pm2000feb.pp

  • Outputs will be created in: ./Plots_TC/<jobid>/

(B) The Stratospheric Chemistry (toms_haloe) package currently includes:

  • CH4, N2O, O3, against HALOE dataset
  • O3 against a dataset compiled at NIWA for CCMVal
  • Column O3 against TOMS

The HALOE/ NIWA comparisons are currently for the months of January and July, while the Column data is based on the whole year.

This is launched in a similar way i.e. by passing address of one monthly pp file from an annual set:

/home/mdalvi/eval_v1/toms_haloe/compare_toms_haloe.py /nerc/ukca/mdalvi/xhmaj/xhmaja.pm2000feb.pp

For comparing two models against observations, use: /home/mdalvi/eval_v1/toms_haloe_2mod/compare2_toms_haloe.py <ppfile-run1> <ppfile-run2>

  • Outputs will be created in : ./Plots_TH/<jobid>/

(C) The Aerosols evaluation package Assessments based on those being carried out for the AEROCOM project. This requires specific fields including GLOMAP-MODE tracers to be present in the monthly pp files. These fields can be added to the job output by using the hand_edit: ~hadzm/umui_jobs/hand_edits/vn8.2/add_aero_eval1_diags_l85.ed (for L85 runs).

  • To launch the analysis:

/home/h02/hadzm/eval_v1/AeroCom/eval_aero.py <one pp file from 12 monthly set> [output dir (default ./)]

  • The outputs will be generated in : <outdir>/EvalAero/<jobid>/

  • Note: (A) and (C) require more than 5 Gbytes memory and will slow down your Desktop or even fail to execute. For the Met Office Linux environment, it is preferable to launch these on one of the els* servers (requiring the pp files to be on that server or in a common area) that has atleast 7-8 Gbytes free memory as indicated from the top command.