NAG compiler error in PRTAllometricCNPMod.F90

[jedwards4b]

Using NAG Fortran Compiler Release 6.2(Chiyoda) Build 6207 we are getting the following error:
(FATES tag sci.1.46.2_api.16.1.0)

mpif90 -c -I. -I/scratch/cluster/jedwards/ERS_Ld5_Vnuopc.f19_g17.ETEST.izumi_nag.cice-default.20210816_103800_z2pt9k/bld/nag/mvapich2/nodebug/nothreads/nuopc/CDEPS/fox/include -I/scratch/cluster/jedwards/ERS_Ld5_Vnuopc.f19_g17.ETEST.izumi_nag.cice-default.20210816_103800_z2pt9k/bld/nag/mvapich2/nodebug/nothreads/nuopc/CDEPS/dshr -I/scratch/cluster/jedwards/ERS_Ld5_Vnuopc.f19_g17.ETEST.izumi_nag.cice-default.20210816_103800_z2pt9k/bld/nag/mvapich2/nodebug/nothreads/nuopc/include -I/scratch/cluster/jedwards/ERS_Ld5_Vnuopc.f19_g17.ETEST.izumi_nag.cice-default.20210816_103800_z2pt9k/bld/nag/mvapich2/nodebug/nothreads/nuopc/nuopc/esmf/c1a1l1i1o1r1/include -I/scratch/cluster/jedwards/ERS_Ld5_Vnuopc.f19_g17.ETEST.izumi_nag.cice-default.20210816_103800_z2pt9k/bld/nag/mvapich2/nodebug/nothreads/nuopc/finclude -I/usr/local/netcdf-c-4.6.1-f-4.4.4-gcc-g++-8.1.0-nag-6.2/include -I/cluster/mvapich2-2.3.3-gcc-8.1.0-nag-6.2/include -I/scratch/cluster/jedwards/ERS_Ld5_Vnuopc.f19_g17.ETEST.izumi_nag.cice-default.20210816_103800_z2pt9k/bld/nag/mvapich2/nodebug/nothreads/nuopc/include -I/scratch/cluster/jedwards/ERS_Ld5_Vnuopc.f19_g17.ETEST.izumi_nag.cice-default.20210816_103800_z2pt9k/bld/nag/mvapich2/nodebug/nothreads/nuopc/nuopc/esmf/include -I/scratch/cluster/jedwards/ERS_Ld5_Vnuopc.f19_g17.ETEST.izumi_nag.cice-default.20210816_103800_z2pt9k/bld/nag/mvapich2/nodebug/nothreads/nuopc/nuopc/esmf/clm/obj -I. -I/scratch/cluster/jedwards/ERS_Ld5_Vnuopc.f19_g17.ETEST.izumi_nag.cice-default.20210816_103800_z2pt9k/SourceMods/src.clm -I/home/jedwards/cesm2_x_alpha/components/clm/src/cpl/nuopc -I/home/jedwards/cesm2_x_alpha/components/clm/src/main -I/home/jedwards/cesm2_x_alpha/components/clm/src/biogeophys -I/home/jedwards/cesm2_x_alpha/components/clm/src/biogeochem -I/home/jedwards/cesm2_x_alpha/components/clm/src/soilbiogeochem -I/home/jedwards/cesm2_x_alpha/components/clm/src/dyn_subgrid -I/home/jedwards/cesm2_x_alpha/components/clm/src/init_interp -I/home/jedwards/cesm2_x_alpha/components/clm/src/self_tests -I/home/jedwards/cesm2_x_alpha/components/clm/src/fates -I/home/jedwards/cesm2_x_alpha/components/clm/src/fates/main -I/home/jedwards/cesm2_x_alpha/components/clm/src/fates/biogeophys -I/home/jedwards/cesm2_x_alpha/components/clm/src/fates/biogeochem -I/home/jedwards/cesm2_x_alpha/components/clm/src/fates/fire -I/home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh -I/home/jedwards/cesm2_x_alpha/components/clm/src/utils -I/home/jedwards/cesm2_x_alpha/components/clm/src/cpl -I/home/jedwards/cesm2_x_alpha/components/clm/src/cpl/utils -I/home/jedwards/cesm2_x_alpha/components/clm/src/cpl/share_esmf -I/scratch/cluster/jedwards/ERS_Ld5_Vnuopc.f19_g17.ETEST.izumi_nag.cice-default.20210816_103800_z2pt9k/bld/lib/include -Wp,-macro=no_com -convert=BIG_ENDIAN -indirect /home/jedwards/cesm2_x_alpha/cime/config/cesm/machines/nag_mpi_argument.txt -ieee=full -O2 -I/project/esmf/PROGS/esmf/8.2.0b13/mvapich2/2.3.3/nag/6.2/mod/modO/Linux.nag.64.mvapich2.default -I/project/esmf/PROGS/esmf/8.2.0b13/mvapich2/2.3.3/nag/6.2/include -I/usr/local/netcdf-c-4.6.1-f-4.4.4-gcc-g++-8.1.0-nag-6.2/include -I/usr/local/netcdf-c-4.6.1-f-4.4.4-gcc-g++-8.1.0-nag-6.2/include -DLINUX -DCESMCOUPLED -DFORTRANUNDERSCORE -DNO_CRAY_POINTERS -DNO_SHR_VMATH -DCPRNAG -DNDEBUG -DUSE_ESMF_LIB -DHAVE_MPI -DNUOPC_INTERFACE -DPIO2 -DHAVE_SLASHPROC -DESMF_VERSION_MAJOR=8 -DESMF_VERSION_MINOR=2 -DATM_PRESENT -DICE_PRESENT -DLND_PRESENT -DOCN_PRESENT -DROF_PRESENT -DGLC_PRESENT -DWAV_PRESENT -DESP_PRESENT -DMED_PRESENT -DPIO2 -free -DUSE_CONTIGUOUS= /home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh/PRTAllometricCNPMod.F90
NAG Fortran Compiler Release 6.2(Chiyoda) Build 6207
Questionable: /home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh/PRTAllometricCNPMod.F90, line 987: Variable CANOPY_TRIM set but never referenced
Questionable: /home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh/PRTAllometricCNPMod.F90, line 987: Variable C_GAIN0 set but never referenced
Warning: /home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh/PRTAllometricCNPMod.F90, line 987: Unused local variable DBH
Warning: /home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh/PRTAllometricCNPMod.F90, line 987: Unused dummy variable MAINT_R_DEFICIT
Warning: /home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh/PRTAllometricCNPMod.F90, line 1533: Unused local variable MAINT_R_DEFICIT
Warning: /home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh/PRTAllometricCNPMod.F90, line 1650: Unused local variable GROWTH_R_FLUX
Warning: /home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh/PRTAllometricCNPMod.F90, line 1650: Unused dummy variable N_EFFLUX
Warning: /home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh/PRTAllometricCNPMod.F90, line 1650: Unused dummy variable P_EFFLUX
Warning: /home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh/PRTAllometricCNPMod.F90, line 1650: Unused local variable STORE_C_FLUX
Warning: /home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh/PRTAllometricCNPMod.F90, line 1664: Unused dummy variable THIS
Warning: /home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh/PRTAllometricCNPMod.F90, line 1939: Unused local variable GROW_C
Warning: /home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh/PRTAllometricCNPMod.F90, line 1939: Unused dummy variable IPFT
Questionable: /home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh/PRTAllometricCNPMod.F90, line 2051: Variable C_VAR_ID set but never referenced
Warning: /home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh/PRTAllometricCNPMod.F90, line 2221: Unused dummy variable CBALANCE
Warning: /home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh/PRTAllometricCNPMod.F90, line 2293: ALL_ORGANS explicitly imported into PRTALLOMETRICCNPMOD but not used
Warning: /home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh/PRTAllometricCNPMod.F90, line 2293: FATES_INT explicitly imported into PRTALLOMETRICCNPMOD (as I4) but not used
Warning: /home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh/PRTAllometricCNPMod.F90, line 2293: H_ALLOM explicitly imported into PRTALLOMETRICCNPMOD but not used
Warning: /home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh/PRTAllometricCNPMod.F90, line 2293: LEAVES_ON explicitly imported into PRTALLOMETRICCNPMOD but not used
Extension: /home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh/PRTAllometricCNPMod.F90, line 1625: Argument A2 (no. 2) data type DOUBLE PRECISION inconsistent with previous argument with data type REAL in reference to intrinsic MAX
Panic: /home/jedwards/cesm2_x_alpha/components/clm/src/fates/parteh/PRTAllometricCNPMod.F90: Unexpected expr node type 432
Internal Error -- please report this bug
Abort

[rgknox]

@jedwards4b do you know if this error cropping up because of a new compiler/machine being tested, or is this associated with a change to the code?

[jedwards4b]

@ekluzek may be able to answer - I think that we've been using this compiler machine combination for a while, but I'm not sure when the fates code was last updated in clm. I get this error for hash 66f3d4c and newer of the file. Older hashes present a different error.

[ekluzek]

This is using ctsm5.1.dev050 and the last fates update was ctsm5.1.dev046 (and dev036 before that). This compiler/machine has been used for awhile. But, this test is likely only done for prebeta testing maybe? Or it's a new test added to CAM? We haven't been seeing this error in our own aux_clm and fates testing, but there must be something different in this test. I'll look into it more.

[rgknox]

I've seen internal compiler errors in the past that were just problems with the compiler and/or bad luck. But its possible internal compiler errors could be trying to tell me something is wrong, but it just doesn't know how to express itself?

[ekluzek]

We had a previous problem similar to this (that was an internal compiler error) that was a legit code error.

ESCOMP/CTSM#1129

That had to do with polymorphism in FATES. So yeah it can be pointing out legit problems in the code, but can't characterize what the problem is.

[ekluzek]

OK, so I think the "problem" is that this test is running without DEBUG mode off. We have a DEBUG mode test that's been working ERI_D_Ld9_P48x1_Vnuopc.f10_f10_mg37.I2000Clm50Sp.izumi_nag.clm-SNICARFRC. I ran ERS_Ld5_Vnuopc.f19_g17.I2000Clm50Sp.izumi_nag.clm-default on izumi with ctsm5.1.dev050 and get the same kind of problem.

@jedwards4b can we change this test to be running with DEBUG on? Our workaround has been to run with DEBUG mode on for all our nag testing on izumi. We keep forgetting about this problem and then remember it again when beta testing is done again. I can add it for discussion at the CSEG meeting tomorrow as well.

[glemieux]

Closing as stale.