Minutes of the code management group meeting 2018-09-05

From UKCA
Jump to: navigation, search

Code management group minutes

Teleconference Details

Wednesday 5th September 2018, 10am - 12noon BST

  • UK Landline: +44 (0)1223 761376 (x61376 from an internal University of Cambridge phone)
  • Participant PIN: 323180

Present

Met Office: John Hemmings (JH), Mohit Dalvi (MD), Fiona O’Connor (FOC), Lucy Neal (LN), Alan Hewitt (AJH), Marc Stringer (MS), Adrian Hill (AH), Jane Mulcahy (JM), Maff Glover (MG)

Cambridge: Luke Abraham (LA), Alex Archibald (ATA)

Leeds: Chris Dearden (CD), Kirsty Pringle (KP), Mark Richardson (MR)

Reading:

Lancaster: Ryan Hossaini (RH), Ewa Bednarz (EB)

Edinburgh:

UEA:

NIWA: Olaf Morgenstern (OM), Guang Zeng (GZ)

CSIRO:

Apologies: GM, NB, JK, SAN

Actions from this meeting

One-off actions

  1. (ongoing) MY to check GLOMAP-mode code and report any missing bug fixes on the bugfixes page.
  2. (ongoing) LA to liaise with MY, and to chase MY and GM as to when this would be done.
  3. (ongoing) LA to migrate PUMA UM tickets to MOSRS - see table below.
  4. (ongoing) LA to open tickets for all changes that still require a ticket.
  5. (ongoing) FOC to recommend at next Exec meeting a policy of allowing only CMG to (certify and) distribute any UKCA configurations for science purpose.
  6. (ongoing) MD to collate all UKCA parameters into ukca_constants in advance of removing duplicates already available in the UM and circulate details.
  7. (ongoing) LA to put C_SPECIES definitions in the Redundant Code table (short-term), and switch to use M_SPECIES instead. Talk with JH about AQUM tests and requirements. Note that there will be problems with exoplanets and >100km top configs where M_air not constant.
  8. (new) AJH to open ticket and assign to himself regarding loops over imode+n and imode-n.
  9. (ongoing) MD/LA to talk about releases offline, and set up free-running and nudged GA7.1+StratTrop TS2000ish timeslice, with instructions to make it transient.
  10. (new) NLA/MD make script for perpetual nudging (symbolic linking to existing files).
  11. (new) NLA to port suite to ARCHER.
  12. (ongoing) LA to create Wiki page on UKCA website for users to add details of configurations and input files.
  13. (ongoing) ALL to review output from vn11.0 release candidate(s).
  14. (new) MD to provide vn11.0 release candidate output again.
  15. (ongoing) FOC to circulate doodle poll for further emissions discussion to CMG for all to fill-in. That meeting to report to CMG
  16. (ongoing) MR to discuss with KP implications for GLOMAP in regards to a separate UKCA repository
  17. (new) NLA to contact CEJ (cc JM) about trace-gas convective plume scavenging and how this will be impacted (or impact) the repository work
  18. (ongoing) LA to contact FOC, AH, MR, KP, ATA, Ken Carslaw, Philip Stier regarding taking the proposal for a separate code repository forward.
  19. (ongoing) FOC to see what action can be taken from MASS, then put out an email for POCs from institutes.
  20. (new) FOC/AH/MD to develop a strategy on MASS for discussion at next CMG
  21. (new) FOC to contact Garry Hayman to highlight bug #35.
  22. (new) JM to take ownership of redundant code item #6 (Wet oxidation of SO2 in GLOMAP).
  23. (new) LN to document redundant code item #9 (Deprecated UKCA netcdf attribute options).
  24. (new) JM to do a catch-call ticket for GMED
  25. (new) LN to l_fix_nh4no3_equilibrium in rose-stem and jobs and remove if necessary.
  26. (new) NLA to do doodle poll for 90 minute meeting in first 2 weeks in November.

Standing actions

  1. (ongoing - standing item) CMG to monitor list of bugs and circulate between group for owners.
  2. (ongoing - standing item) At CMG future meetings there should be included an appendix listing the open bugs. If a ticket is included it is not an open bug (as the owner of ticket is person doing lodging).
  3. (ongoing - standing item) ALL with access to keep monitoring ukca_bugs email list.
  4. (ongoing - standing item) ALL to think about if code changes will change KGO and let systems team know in good time, and remember to include the UKCA & SC0138 keywords to all UKCA tickets. This should be a standard part of our working practice. Note also the required keywords.
  5. (ongoing - standing item) ALL to note, review, and remove obsolete/redundant code
  6. (ongoing - standing item) ALL closed ticket owners to produce a summary of their change for the UKCA newsletter
  7. (ongoing - standing item) Code Owner to produce draft newsletter for CMG to approve.

Minutes

Actions from the Last Meeting (LA)

Minutes of the code management group meeting 2018-06-19

One-off actions

  1. (ongoing) MY to check GLOMAP-mode code and report any missing bug fixes on the bugfixes page.
    Keep ongoing. This should be done later in the year.
  2. (ongoing) LA to liaise with MY.
    Keep ongoing. LA to chase MY and GM as to when this would be done.
  3. (ongoing) LA to migrate PUMA UM tickets to MOSRS - see table below.
    Keep ongoing.
  4. (ongoing) LA to open tickets for all changes that still require a ticket.
    Keep ongoing.
  5. (ongoing) FOC to recommend at next Exec meeting a policy of allowing only CMG to (certify and) distribute any UKCA configurations for science purpose.
    Keep ongoing.
  6. (ongoing) MD to collate all UKCA parameters into ukca_constants in advance of removing duplicates already available in the UM and circulate details.
    The UM is moving its constants to SHUMlib, but this changes results. Keep ongoing.
  7. (new) LA to put C_SPECIES definitions in the Redundant Code table (short-term), and switch to use M_SPECIES instead. Talk with JH about AQUM tests and requirements.
    Note that there will be problems with exoplanets and >100km top configs where M_air not constant. Keep as ongoing.
  8. (ongoing) AJH to scope out reducing repetition of mode definitions in GLOMAP. See UM:#4097
    Close. Ticket under review.
    AJH has since found some parts of code with loops around imode-n and imode+n, and this will probably need to be done in a more clever way. (e.g. mask with soluble modes etc.).
    ACTION: AJH to open ticket and assign to himself.
  9. (ongoing) MD/LA to talk about releases offline, and set up free-running and nudged GA7.1+StratTrop TS2000ish timeslice, with instructions to make it transient.
    Keep ongoing.
    MD has a nudged job on long run at the moment, but it is not possible to define perpetual year in the nudging within Rose.
    ACTION: NLA/MD make script for perpetual nudging (symbolic linking to existing files).
    ACTION: NLA to port suite to ARCHER.
    MD has a ticket details page and will move to UKCA website. Costs estimates for ARCHER and Monsoon2/NEXCS should be include as done for vn8.4.
    Have ready by start of October.
  10. (ongoing) LA to create Wiki page on UKCA website for users to add details of configurations and input files.
    Keep ongoing.
  11. (ongoing) ALL to review output from vn11.0 release candidate(s).
    Keep ongoing.
    ATA highlighted that nothing is out of the ordinary, but STE is low.
    AOD diagnostics and auto-assess output is missing.
    ACTION: MD to provide output
  12. (ongoing) MG to progress ticket UM:#3909 after doing further tests.
    Close
    This will be superseded by new Cray compiler - MG to link to ticket for new compiler.
    OM highlighted that the NIWA XC50 has to use ifort as the CCE on XC50 doesn’t compile the UM.
    It was noted that NCAS-CMS are having similar issues on ARCHER.
  13. (new) ALL to feedback to AJH by 2nd July regarding UKCA Rose panel changes
    Close
  14. (new) FOC to ask Paul Agnew to send AQUM representative to CMG
    Close (LN)
  15. (new) FOC to circulate doodle poll for further emissions discussion to CMG for all to fill-in. That meeting to report to CMG
    Keep ongoing.
  16. (new) MR to discuss with KP implications for GLOMAP.
    Keep ongoing - this is in regards to a separate UKCA repository
    ACTION: NLA to contact CEJ about trace-gas convective plume scavenging, cc JM
  17. (new) LA to contact FOC, AH, MR, KP, ATA, Ken Carslaw, Philip Stier regarding taking the proposal for a separate code repository forward.
    ongoing, NLA to circulate again. Feedback has been positive from those contacted.
  18. (new) FOC to see what action can be taken from MASS, then put out an email for POCs from institutes.
    Keep ongoing.
    FOC and AH have agreed to be joint owners for users outside of MO and will split institutes between them. They need to talk to MASS team.
    ACTION: FOC/AH/MD to develop a strategy on MASS for discussion at next CMG
  19. (new) LA to do doodle poll for weeks 1-2 of September 2018.
    Close

Standing actions

  1. (ongoing - standing item) CMG to monitor list of bugs and circulate between group for owners.
  2. (ongoing - standing item) At CMG future meetings there should be included an appendix listing the open bugs. If a ticket is included it is not an open bug (as the owner of ticket is person doing lodging).
  3. (ongoing - standing item) ALL with access to keep monitoring ukca_bugs email list.
  4. (ongoing - standing item) ALL to think about if code changes will change KGO and let systems team know in good time, and remember to include the UKCA & SC0138 keywords to all UKCA tickets. This should be a standard part of our working practice. Note also the required keywords.
  5. (ongoing - standing item) ALL to note, review, and remove obsolete/redundant code
  6. (ongoing - standing item) ALL closed ticket owners to produce a summary of their change for the UKCA newsletter
  7. (ongoing - standing item) Code Owner to produce draft newsletter for CMG to approve.

Bugs/Bugfixes (LA)

Open Bugs

# Description Ticket Owner Status/ Updates Comments from previous meeting Comments from this meeting
35 Error in labelling: Surface resistance <--> stomatal resistance ? UM:#1409 Fiona O'Connor Leave open. Garry Hayman started work in offline JULES. Still ongoing. See action.
74 Bug in ukca_tropopause with definition of PV tropopause UM:#1283
New ticket UM:#1367
Mohit Dalvi Fix to be combined with Bug 75 below MOSRS: vn10.6_ukca_tropopause_amendment Should make 11.2
75 Bug in STE fluxes UM:#1367 Mohit Dalvi Fix branch: vn11.1_ukca_fix_o3_ste
will also include tropopause fix
Should make 11.2
87 Possible issues with YSTORE in UKCA_CHEMISTRY_CTL. UM:#3245 Colin Johnson / Luke Abraham Leave open. Leave open. NLA to have another look. MS may have changed things with OMP changes.
96 Output drydiam diagnostics where l_ukca_arg_act but not l_ukca_radaer UM:#3962 Alan J Hewitt The beginnings of a solution are found in vn11.0_ukca_drydiam_activate_wo_radaer. Leave open. AJH to take ownership - may be affected by reorganisation. Leave open.
98 Incorrect usage of npft integer when npft == 9 OR 13 UM:#4157 Alan J Hewitt / Garry Hayman To be lodged at vn11.2 Follow-on ticket to remove temporary logical (l_fix_ukca_h2dd_x) UM:#4245 NA Under SciTech review. Logical will be removed after GA9.

ACTION: FOC to contact Garry Hayman to highlight bug #35.

Redundant Code (LA)

Code for deletion

PLEASE REMEMBER TO EDIT THE ORIGINAL REDUNDANT CODE TABLE AND NOT THIS PAGE

# Description Ticket Remarks Status Timeframe Owner Comments from this meeting
2 Old dry deposition scheme UM:#2356 None PROPOSED Medium-term Nick Savage
4 ukca_strat_update.F90 - relax_ozone UM:#1505 None PROPOSED Medium-term Luke Abraham
5 Backward Euler Standard Tropospheric Chemistry UM:#1507 None PROPOSED Medium-term Fiona O'Connor
6 Wet oxidation of SO2 in GLOMAP None None PROPOSED Medium-term Colin Johnson JM to look at this re: UKESM1 c.f. offline GLOMAP. JM to take ownership.
7 Updraft velocity - l_conv_tke UM:#3668 Proposed in December 2017. Approval being sought for GA series (likely GA8). PROPOSED Medium-term Alan J Hewitt / Ian Boutle
8 All backward Euler chemistry schemes None Note that this is only for the “hard-coded” BE schemes. Quite a lot of work would be involved, but good as a longer-term aim. Note that problems for AQUM would need to be addressed first. PROPOSED Long-term Luke Abraham
9 Deprecated UKCA netcdf attribute options UM:#2513 'emission_type', string type attributes no longer recommeneded. Need some time for most new code to adopt these. Still used in AQUM files. DEPRECATED Medium-term Nick Savage AQUM are addressing the files. Have gone from operational jobs, have ticket to remove from rose-stem. LN to document.
10 ASAD families chemistry None Families chemistry has not been used for many years (since at least UM6.1. It is likely that it no longer works if turned on, and it complicates the code making other developments harder DEPRECATED Short-term Luke Abraham
11 Dry deposition 9 PFT option None The 13 PFT option has been adopted for UKESM1.
The 9 PFT option is possibly not being used at all.
Proposed in June 2018.
PROPOSED Long-term Alan J Hewitt
12 Remove explicit C_SPECIES constants and calculate these from M_SPECIES and the mass of air None Will have wide-ranging rose-stem KGO breakages PROPOSED Short-term Luke Abraham
13 Remove temporary logical l_fix_nacl_density
Introduced at UM:#2405 by Colin Johnson
UM:#3490 Requires AMIP testing and a GMED ticket
Could be accepted to GA9
PROPOSED Medium-term None See JM GMED action
14 Remove temporary logical l_fix_nh4no3_equilibrium
Introduced at UM:#646 by Carlos Ordonez
None Requires AMIP testing and a GMED ticket
Could be accepted to GA9
PROPOSED Medium-term None See LN temporary logical action
15 Remove temporary logical l_fix_ukca_impscav
Introduced at UM:#1421
UM:#4145 Approval being sought for GA series (likely GA9)
GMED:#402
PROPOSED Medium-term Alan J Hewitt See JM GMED action
16 Remove temporary logical l_fix_improve_drydep
Introduced at UM:#1250
UM:#3997 Approval being sought for GA series (likely GA9)
GMED:#393
PROPOSED Medium-term Alan J Hewitt See JM GMED action
17 Remove temporary logical l_fix_ukca_h2dd_x
Introduced at UM:#4157
UM:#4245 Approval being sought for GA series (likely GA9)
GMED:#410
PROPOSED Medium-term Alan J Hewitt See JM GMED action

ACTION: JM to take ownership of redundant code item #6 (Wet oxidation of SO2 in GLOMAP).

ACTION: LN to document redundant code item #9 (Deprecated UKCA netcdf attribute options).

ACTION: JM to do a catch-call ticket for GMED

ACTION: LN to l_fix_nh4no3_equilibrium in rose-stem and jobs and remove if necessary.

UKCA Release Job (LA/MD)

See previous comments under actions from the last meeting.

Targets for vn11.2 (and beyond) (LA)

PLEASE REMEMBER TO INCLUDE THE KEYWORDS: UKCA & SC0138

Tickets targeting vn11.2

UM vn11.2 code review submission deadline is 10th Oct 2018. SOCRATES/JULES/GCOM/KGO by 3rd Oct 2018.

# Description
UM:#1367 Fix bug in Stratosphere-Troposphere Exchange fluxes of UKCA tracers
UM:#2689 Enable coupling of GLOMAP-MODE Aerosol to FAST-JX photolysis.
UM:#3127 Investigate KGO differences in um_n48_ukca_eg app using GNU on the Cray
UM:#3292 Consolidate constant variables used by UKCA
UM:#3342 Update of UKCA heterogeneous chemistry routine
UM:#3367 Removing transport of peroxy radical tracers in Strattrop (CHeST)
UM:#3673 Fail-if tests for CH4 and N2O in radiation panel for certain configurations of UKCA
UM:#3769 Time updating for glomap climatology
UM:#3779 Improvements to the inline comments in ukca_option_mod
UM:#3959 RO2-Permutation Chemistry
UM:#3970 Tidy the UKCA gui panel
UM:#4001 Magic numbers in ukca_setup_chem_mod.F90
UM:#4020 Improved calculation of ozone dry deposition to ocean - surface resistance term
UM:#4032 Addition of new diagnostics for the wet and dry deposition flux of oxidised nitrogen
UM:#4097 Remove repetition of mode definitions and some magic numbers
UM:#4131 Enable use of ASAD Backward-Euler solver with UKCA offline oxidants chemistry
UM:#4157 Bug in surface resistance on tiles (H2) for npft/=5 in ukca_surfddr
UM:#4202 Introduce UKCA variable 'nrsteps' as a Rose namelist variable
UM:#4212 Add new local variables to prevent memory striding in call to asad_cdrive

Tickets targeting vn11.3

UM vn11.3 code review submission deadline is 7th Feb 2019. SOCRATES/JULES/GCOM/CASIM/KGO by 31st Jan 2019.

# Description
UM:#3962 Output dry diameter diagnostic in niche case where l_ukca_arg_act but not l_ukca_radaer
UM:#3997 Retire temporary logical l_fix_improve_drydep
UM:#4024 Triggering of i_ukca_dms_flux by l_sulpc_so2
UM:#4145 Retire temporary logical l_fix_ukca_impscav
UM:#4201 Verify use of qsmin in UKCA plume scavenging diagnostics
UM:#4221 Optimisation of GLOMAP_CLIM
UM:#4231 Adding Common Representatives Intermediate (CRI) mechanism
UM:#4245 Retire temporary logical l_fix_ukca_h2dd_x

"Work-stealing" between day and night points (MS)

Michele Guidolin from the HPC Optimisation team at the MO is implementing work stealing into the SW radiation in the UM, and I think it's something which might work for UKCA. From memory, I think Michele's work on SW radiation has produced around a 5% speed-up to an AMIP job.

As I understand for the SW radiation, the following is done for work-stealing

  1. Pair up PEs on opposite sides of the globe, so you always have a ‘PE in light’ and a ‘PE in dark’
  2. The ‘PE in light’ has more segments of SW radiation work to do than the ‘PE in dark’ and so you determine a number of segments of work to be sent from the ‘PE in light’ to the ‘PE in dark’ (this is the ‘work-stealing’).
  3. The segments are passed between the ‘PE in light’ and ‘PE in dark’ using MPI

Now for the chemistry, I have in my head that it’s a general rule that much more chemistry is done in the light than in the dark – as is the case for the SW radiation. However, it if turns out that there’s always a lot of chemistry in say the south pole then we might have to re-think in what direction the work stealing is done. We would need some better analysis of the imbalance in the chemistry work, than I currently have, before starting anything like this. But for now let’s assume that I’m broadly right that there’s more chemistry done in the light than in the dark.

Unlike the SW radiation, where it seems that the ‘PE in light’ has more segments of work to do than the ‘PE in dark’, the segments of work for ‘PE in light’ and ‘PE in dark’ are the same. The difference is that the work for ‘PE in dark’ can be done much quicker, because the chemistry reactions turn out to be simpler and quicker.

The work stealing might work for both chemistry and aerosol, but I’ll continue with the chemistry solver for now as it’s more expensive and I know more about it. The top routine for the chemistry solver, which UKESM uses, is UKCA_CHEMISTRY_CTL. This applies OpenMP over the vertical levels. Here we might consider sending say 10 levels from the ‘PE in light’ to the ‘PE in dark’ to calculate, so that the ‘PE in light’ would calculate 75 levels and the ‘PE in dark’ would calculate 95 levels. However, sending and receiving all the information from the ‘PE in light’ to the ‘PE in dark’ in this routine is probably going to be very complicated and require a lot of code. It might be better to use the alternative to UKCA_CHEMISTRY_CTL which is UKCA_CHEMISTRY_CTL_COL - especially if Chris adds segments into this.

Discussion

FOC highlighted that a similar scheme to this was done for FastJ on the old NEC machine by a consultant. This code could be revived.

MR believes that following CDs work on chunking a lot of these numbers could be quantified.

It was agreed for CD to continue with chunking, and re-visit work stealing when that work is completed and do an assessment then.

AOB/DNM (LA)

Date of next meeting (LA)

ACTION: NLA to do doodle poll for 90 minute meeting in first 2 weeks in November.