Minutes of the code management group meeting 2018-09-05
Teleconference Details
Wednesday 5th September 2018, 10am - 12noon BST
- UK Landline: +44 (0)1223 761376 (x61376 from an internal University of Cambridge phone)
- Participant PIN: 323180
Present
Met Office: John Hemmings (JH), Mohit Dalvi (MD), Fiona O’Connor (FOC), Lucy Neal (LN), Alan Hewitt (AJH), Marc Stringer (MS), Adrian Hill (AH), Jane Mulcahy (JM), Maff Glover (MG)
Cambridge: Luke Abraham (LA), Alex Archibald (ATA)
Leeds: Chris Dearden (CD), Kirsty Pringle (KP), Mark Richardson (MR)
Reading:
Lancaster: Ryan Hossaini (RH), Ewa Bednarz (EB)
Edinburgh:
UEA:
NIWA: Olaf Morgenstern (OM), Guang Zeng (GZ)
CSIRO:
Apologies: GM, NB, JK, SAN
Actions from this meeting
One-off actions
- (ongoing) MY to check GLOMAP-mode code and report any missing bug fixes on the bugfixes page.
- (ongoing) LA to liaise with MY, and to chase MY and GM as to when this would be done.
- (ongoing) LA to migrate PUMA UM tickets to MOSRS - see table below.
- (ongoing) LA to open tickets for all changes that still require a ticket.
- (ongoing) FOC to recommend at next Exec meeting a policy of allowing only CMG to (certify and) distribute any UKCA configurations for science purpose.
- (ongoing) MD to collate all UKCA parameters into
ukca_constants
in advance of removing duplicates already available in the UM and circulate details. - (ongoing) LA to put
C_SPECIES
definitions in the Redundant Code table (short-term), and switch to useM_SPECIES
instead. Talk with JH about AQUM tests and requirements. Note that there will be problems with exoplanets and >100km top configs where M_air not constant. - (new) AJH to open ticket and assign to himself regarding loops over imode+n and imode-n.
- (ongoing) MD/LA to talk about releases offline, and set up free-running and nudged GA7.1+StratTrop TS2000ish timeslice, with instructions to make it transient.
- (new) NLA/MD make script for perpetual nudging (symbolic linking to existing files).
- (new) NLA to port suite to ARCHER.
- (ongoing) LA to create Wiki page on UKCA website for users to add details of configurations and input files.
- (ongoing) ALL to review output from vn11.0 release candidate(s).
- (new) MD to provide vn11.0 release candidate output again.
- (ongoing) FOC to circulate doodle poll for further emissions discussion to CMG for all to fill-in. That meeting to report to CMG
- (ongoing) MR to discuss with KP implications for GLOMAP in regards to a separate UKCA repository
- (new) NLA to contact CEJ (cc JM) about trace-gas convective plume scavenging and how this will be impacted (or impact) the repository work
- (ongoing) LA to contact FOC, AH, MR, KP, ATA, Ken Carslaw, Philip Stier regarding taking the proposal for a separate code repository forward.
- (ongoing) FOC to see what action can be taken from MASS, then put out an email for POCs from institutes.
- (new) FOC/AH/MD to develop a strategy on MASS for discussion at next CMG
- (new) FOC to contact Garry Hayman to highlight bug #35.
- (new) JM to take ownership of redundant code item #6 (Wet oxidation of SO2 in GLOMAP).
- (new) LN to document redundant code item #9 (Deprecated UKCA netcdf attribute options).
- (new) JM to do a catch-call ticket for GMED
- (new) LN to l_fix_nh4no3_equilibrium in rose-stem and jobs and remove if necessary.
- (new) NLA to do doodle poll for 90 minute meeting in first 2 weeks in November.
Standing actions
- (ongoing - standing item) CMG to monitor list of bugs and circulate between group for owners.
- (ongoing - standing item) At CMG future meetings there should be included an appendix listing the open bugs. If a ticket is included it is not an open bug (as the owner of ticket is person doing lodging).
- (ongoing - standing item) ALL with access to keep monitoring ukca_bugs email list.
- (ongoing - standing item) ALL to think about if code changes will change KGO and let systems team know in good time, and remember to include the UKCA & SC0138 keywords to all UKCA tickets. This should be a standard part of our working practice. Note also the required keywords.
- (ongoing - standing item) ALL to note, review, and remove obsolete/redundant code
- (ongoing - standing item) ALL closed ticket owners to produce a summary of their change for the UKCA newsletter
- (ongoing - standing item) Code Owner to produce draft newsletter for CMG to approve.
Minutes
Actions from the Last Meeting (LA)
Minutes of the code management group meeting 2018-06-19
One-off actions
- (ongoing) MY to check GLOMAP-mode code and report any missing bug fixes on the bugfixes page.
- Keep ongoing. This should be done later in the year.
- (ongoing) LA to liaise with MY.
- Keep ongoing. LA to chase MY and GM as to when this would be done.
- (ongoing) LA to migrate PUMA UM tickets to MOSRS - see table below.
- Keep ongoing.
- (ongoing) LA to open tickets for all changes that still require a ticket.
- Keep ongoing.
- (ongoing) FOC to recommend at next Exec meeting a policy of allowing only CMG to (certify and) distribute any UKCA configurations for science purpose.
- Keep ongoing.
- (ongoing) MD to collate all UKCA parameters into
ukca_constants
in advance of removing duplicates already available in the UM and circulate details.- The UM is moving its constants to SHUMlib, but this changes results. Keep ongoing.
- (new) LA to put
C_SPECIES
definitions in the Redundant Code table (short-term), and switch to useM_SPECIES
instead. Talk with JH about AQUM tests and requirements.- Note that there will be problems with exoplanets and >100km top configs where M_air not constant. Keep as ongoing.
- (ongoing) AJH to scope out reducing repetition of mode definitions in GLOMAP. See UM:#4097
- Close. Ticket under review.
- AJH has since found some parts of code with loops around imode-n and imode+n, and this will probably need to be done in a more clever way. (e.g. mask with soluble modes etc.).
- ACTION: AJH to open ticket and assign to himself.
- (ongoing) MD/LA to talk about releases offline, and set up free-running and nudged GA7.1+StratTrop TS2000ish timeslice, with instructions to make it transient.
- Keep ongoing.
- MD has a nudged job on long run at the moment, but it is not possible to define perpetual year in the nudging within Rose.
- ACTION: NLA/MD make script for perpetual nudging (symbolic linking to existing files).
- ACTION: NLA to port suite to ARCHER.
- MD has a ticket details page and will move to UKCA website. Costs estimates for ARCHER and Monsoon2/NEXCS should be include as done for vn8.4.
- Have ready by start of October.
- (ongoing) LA to create Wiki page on UKCA website for users to add details of configurations and input files.
- Keep ongoing.
- (ongoing) ALL to review output from vn11.0 release candidate(s).
- Keep ongoing.
- ATA highlighted that nothing is out of the ordinary, but STE is low.
- AOD diagnostics and auto-assess output is missing.
- ACTION: MD to provide output
- (ongoing) MG to progress ticket UM:#3909 after doing further tests.
- Close
- This will be superseded by new Cray compiler - MG to link to ticket for new compiler.
- OM highlighted that the NIWA XC50 has to use ifort as the CCE on XC50 doesn’t compile the UM.
- It was noted that NCAS-CMS are having similar issues on ARCHER.
- (new) ALL to feedback to AJH by 2nd July regarding UKCA Rose panel changes
- Close
- (new) FOC to ask Paul Agnew to send AQUM representative to CMG
- Close (LN)
- (new) FOC to circulate doodle poll for further emissions discussion to CMG for all to fill-in. That meeting to report to CMG
- Keep ongoing.
- (new) MR to discuss with KP implications for GLOMAP.
- Keep ongoing - this is in regards to a separate UKCA repository
- ACTION: NLA to contact CEJ about trace-gas convective plume scavenging, cc JM
- (new) LA to contact FOC, AH, MR, KP, ATA, Ken Carslaw, Philip Stier regarding taking the proposal for a separate code repository forward.
- ongoing, NLA to circulate again. Feedback has been positive from those contacted.
- (new) FOC to see what action can be taken from MASS, then put out an email for POCs from institutes.
- Keep ongoing.
- FOC and AH have agreed to be joint owners for users outside of MO and will split institutes between them. They need to talk to MASS team.
- ACTION: FOC/AH/MD to develop a strategy on MASS for discussion at next CMG
- (new) LA to do doodle poll for weeks 1-2 of September 2018.
- Close
Standing actions
- (ongoing - standing item) CMG to monitor list of bugs and circulate between group for owners.
- (ongoing - standing item) At CMG future meetings there should be included an appendix listing the open bugs. If a ticket is included it is not an open bug (as the owner of ticket is person doing lodging).
- (ongoing - standing item) ALL with access to keep monitoring ukca_bugs email list.
- (ongoing - standing item) ALL to think about if code changes will change KGO and let systems team know in good time, and remember to include the UKCA & SC0138 keywords to all UKCA tickets. This should be a standard part of our working practice. Note also the required keywords.
- (ongoing - standing item) ALL to note, review, and remove obsolete/redundant code
- (ongoing - standing item) ALL closed ticket owners to produce a summary of their change for the UKCA newsletter
- (ongoing - standing item) Code Owner to produce draft newsletter for CMG to approve.
Bugs/Bugfixes (LA)
Open Bugs
# | Description | Ticket | Owner | Status/ Updates | Comments from previous meeting | Comments from this meeting |
---|---|---|---|---|---|---|
35 | Error in labelling: Surface resistance <--> stomatal resistance ? | UM:#1409 | Fiona O'Connor | Leave open. Garry Hayman started work in offline JULES. | Still ongoing. See action. | |
74 | Bug in ukca_tropopause with definition of PV tropopause | UM:#1283 New ticket UM:#1367 |
Mohit Dalvi | Fix to be combined with Bug 75 below MOSRS: vn10.6_ukca_tropopause_amendment | Should make 11.2 | |
75 | Bug in STE fluxes | UM:#1367 | Mohit Dalvi | Fix branch: vn11.1_ukca_fix_o3_ste will also include tropopause fix |
Should make 11.2 | |
87 | Possible issues with YSTORE in UKCA_CHEMISTRY_CTL. | UM:#3245 | Colin Johnson / Luke Abraham | Leave open. | Leave open. NLA to have another look. MS may have changed things with OMP changes. | |
96 | Output drydiam diagnostics where l_ukca_arg_act but not l_ukca_radaer | UM:#3962 | Alan J Hewitt | The beginnings of a solution are found in vn11.0_ukca_drydiam_activate_wo_radaer. | Leave open. AJH to take ownership - may be affected by reorganisation. | Leave open. |
98 | Incorrect usage of npft integer when npft == 9 OR 13 |
UM:#4157 | Alan J Hewitt / Garry Hayman | To be lodged at vn11.2 Follow-on ticket to remove temporary logical (l_fix_ukca_h2dd_x) UM:#4245 | NA | Under SciTech review. Logical will be removed after GA9. |
ACTION: FOC to contact Garry Hayman to highlight bug #35.
Redundant Code (LA)
Code for deletion
PLEASE REMEMBER TO EDIT THE ORIGINAL REDUNDANT CODE TABLE AND NOT THIS PAGE
# | Description | Ticket | Remarks | Status | Timeframe | Owner | Comments from this meeting |
---|---|---|---|---|---|---|---|
2 | Old dry deposition scheme | UM:#2356 | None | PROPOSED | Medium-term | Nick Savage | |
4 | ukca_strat_update.F90 - relax_ozone | UM:#1505 | None | PROPOSED | Medium-term | Luke Abraham | |
5 | Backward Euler Standard Tropospheric Chemistry | UM:#1507 | None | PROPOSED | Medium-term | Fiona O'Connor | |
6 | Wet oxidation of SO2 in GLOMAP | None | None | PROPOSED | Medium-term | Colin Johnson | JM to look at this re: UKESM1 c.f. offline GLOMAP. JM to take ownership. |
7 | Updraft velocity - l_conv_tke | UM:#3668 | Proposed in December 2017. Approval being sought for GA series (likely GA8). | PROPOSED | Medium-term | Alan J Hewitt / Ian Boutle | |
8 | All backward Euler chemistry schemes | None | Note that this is only for the “hard-coded” BE schemes. Quite a lot of work would be involved, but good as a longer-term aim. Note that problems for AQUM would need to be addressed first. | PROPOSED | Long-term | Luke Abraham | |
9 | Deprecated UKCA netcdf attribute options | UM:#2513 | 'emission_type', string type attributes no longer recommeneded. Need some time for most new code to adopt these. Still used in AQUM files. | DEPRECATED | Medium-term | Nick Savage | AQUM are addressing the files. Have gone from operational jobs, have ticket to remove from rose-stem. LN to document. |
10 | ASAD families chemistry | None | Families chemistry has not been used for many years (since at least UM6.1. It is likely that it no longer works if turned on, and it complicates the code making other developments harder | DEPRECATED | Short-term | Luke Abraham | |
11 | Dry deposition 9 PFT option | None | The 13 PFT option has been adopted for UKESM1. The 9 PFT option is possibly not being used at all. Proposed in June 2018. |
PROPOSED | Long-term | Alan J Hewitt | |
12 | Remove explicit C_SPECIES constants and calculate these from M_SPECIES and the mass of air |
None | Will have wide-ranging rose-stem KGO breakages | PROPOSED | Short-term | Luke Abraham | |
13 | Remove temporary logical l_fix_nacl_density Introduced at UM:#2405 by Colin Johnson |
UM:#3490 | Requires AMIP testing and a GMED ticket Could be accepted to GA9 |
PROPOSED | Medium-term | None | See JM GMED action |
14 | Remove temporary logical l_fix_nh4no3_equilibrium Introduced at UM:#646 by Carlos Ordonez |
None | Requires AMIP testing and a GMED ticket Could be accepted to GA9 |
PROPOSED | Medium-term | None | See LN temporary logical action |
15 | Remove temporary logical l_fix_ukca_impscav Introduced at UM:#1421 |
UM:#4145 | Approval being sought for GA series (likely GA9) GMED:#402 |
PROPOSED | Medium-term | Alan J Hewitt | See JM GMED action |
16 | Remove temporary logical l_fix_improve_drydep Introduced at UM:#1250 |
UM:#3997 | Approval being sought for GA series (likely GA9) GMED:#393 |
PROPOSED | Medium-term | Alan J Hewitt | See JM GMED action |
17 | Remove temporary logical l_fix_ukca_h2dd_x Introduced at UM:#4157 |
UM:#4245 | Approval being sought for GA series (likely GA9) GMED:#410 |
PROPOSED | Medium-term | Alan J Hewitt | See JM GMED action |
ACTION: JM to take ownership of redundant code item #6 (Wet oxidation of SO2 in GLOMAP).
ACTION: LN to document redundant code item #9 (Deprecated UKCA netcdf attribute options).
ACTION: JM to do a catch-call ticket for GMED
ACTION: LN to l_fix_nh4no3_equilibrium in rose-stem and jobs and remove if necessary.
UKCA Release Job (LA/MD)
See previous comments under actions from the last meeting.
Targets for vn11.2 (and beyond) (LA)
PLEASE REMEMBER TO INCLUDE THE KEYWORDS: UKCA & SC0138
Tickets targeting vn11.2
UM vn11.2 code review submission deadline is 10th Oct 2018. SOCRATES/JULES/GCOM/KGO by 3rd Oct 2018.
# | Description |
---|---|
UM:#1367 | Fix bug in Stratosphere-Troposphere Exchange fluxes of UKCA tracers |
UM:#2689 | Enable coupling of GLOMAP-MODE Aerosol to FAST-JX photolysis. |
UM:#3127 | Investigate KGO differences in um_n48_ukca_eg app using GNU on the Cray |
UM:#3292 | Consolidate constant variables used by UKCA |
UM:#3342 | Update of UKCA heterogeneous chemistry routine |
UM:#3367 | Removing transport of peroxy radical tracers in Strattrop (CHeST) |
UM:#3673 | Fail-if tests for CH4 and N2O in radiation panel for certain configurations of UKCA |
UM:#3769 | Time updating for glomap climatology |
UM:#3779 | Improvements to the inline comments in ukca_option_mod |
UM:#3959 | RO2-Permutation Chemistry |
UM:#3970 | Tidy the UKCA gui panel |
UM:#4001 | Magic numbers in ukca_setup_chem_mod.F90 |
UM:#4020 | Improved calculation of ozone dry deposition to ocean - surface resistance term |
UM:#4032 | Addition of new diagnostics for the wet and dry deposition flux of oxidised nitrogen |
UM:#4097 | Remove repetition of mode definitions and some magic numbers |
UM:#4131 | Enable use of ASAD Backward-Euler solver with UKCA offline oxidants chemistry |
UM:#4157 | Bug in surface resistance on tiles (H2) for npft/=5 in ukca_surfddr |
UM:#4202 | Introduce UKCA variable 'nrsteps' as a Rose namelist variable |
UM:#4212 | Add new local variables to prevent memory striding in call to asad_cdrive |
Tickets targeting vn11.3
UM vn11.3 code review submission deadline is 7th Feb 2019. SOCRATES/JULES/GCOM/CASIM/KGO by 31st Jan 2019.
# | Description |
---|---|
UM:#3962 | Output dry diameter diagnostic in niche case where l_ukca_arg_act but not l_ukca_radaer |
UM:#3997 | Retire temporary logical l_fix_improve_drydep |
UM:#4024 | Triggering of i_ukca_dms_flux by l_sulpc_so2 |
UM:#4145 | Retire temporary logical l_fix_ukca_impscav |
UM:#4201 | Verify use of qsmin in UKCA plume scavenging diagnostics |
UM:#4221 | Optimisation of GLOMAP_CLIM |
UM:#4231 | Adding Common Representatives Intermediate (CRI) mechanism |
UM:#4245 | Retire temporary logical l_fix_ukca_h2dd_x |
"Work-stealing" between day and night points (MS)
Michele Guidolin from the HPC Optimisation team at the MO is implementing work stealing into the SW radiation in the UM, and I think it's something which might work for UKCA. From memory, I think Michele's work on SW radiation has produced around a 5% speed-up to an AMIP job.
As I understand for the SW radiation, the following is done for work-stealing
- Pair up PEs on opposite sides of the globe, so you always have a ‘PE in light’ and a ‘PE in dark’
- The ‘PE in light’ has more segments of SW radiation work to do than the ‘PE in dark’ and so you determine a number of segments of work to be sent from the ‘PE in light’ to the ‘PE in dark’ (this is the ‘work-stealing’).
- The segments are passed between the ‘PE in light’ and ‘PE in dark’ using MPI
Now for the chemistry, I have in my head that it’s a general rule that much more chemistry is done in the light than in the dark – as is the case for the SW radiation. However, it if turns out that there’s always a lot of chemistry in say the south pole then we might have to re-think in what direction the work stealing is done. We would need some better analysis of the imbalance in the chemistry work, than I currently have, before starting anything like this. But for now let’s assume that I’m broadly right that there’s more chemistry done in the light than in the dark.
Unlike the SW radiation, where it seems that the ‘PE in light’ has more segments of work to do than the ‘PE in dark’, the segments of work for ‘PE in light’ and ‘PE in dark’ are the same. The difference is that the work for ‘PE in dark’ can be done much quicker, because the chemistry reactions turn out to be simpler and quicker.
The work stealing might work for both chemistry and aerosol, but I’ll continue with the chemistry solver for now as it’s more expensive and I know more about it. The top routine for the chemistry solver, which UKESM uses, is UKCA_CHEMISTRY_CTL. This applies OpenMP over the vertical levels. Here we might consider sending say 10 levels from the ‘PE in light’ to the ‘PE in dark’ to calculate, so that the ‘PE in light’ would calculate 75 levels and the ‘PE in dark’ would calculate 95 levels. However, sending and receiving all the information from the ‘PE in light’ to the ‘PE in dark’ in this routine is probably going to be very complicated and require a lot of code. It might be better to use the alternative to UKCA_CHEMISTRY_CTL which is UKCA_CHEMISTRY_CTL_COL - especially if Chris adds segments into this.
Discussion
FOC highlighted that a similar scheme to this was done for FastJ on the old NEC machine by a consultant. This code could be revived.
MR believes that following CDs work on chunking a lot of these numbers could be quantified.
It was agreed for CD to continue with chunking, and re-visit work stealing when that work is completed and do an assessment then.
AOB/DNM (LA)
Date of next meeting (LA)
ACTION: NLA to do doodle poll for 90 minute meeting in first 2 weeks in November.