2019-05-03 13:13:31: xmlchange success ./xmlchange DOUT_S=TRUE --------------------------------------------------- 2019-05-03 13:13:32: xmlchange success ./xmlchange RESUBMIT=7 --------------------------------------------------- 2019-05-03 13:13:32: xmlchange success ./xmlchange STOP_N=10 --------------------------------------------------- 2019-05-03 13:13:32: xmlchange success ./xmlchange STOP_OPTION=nyears --------------------------------------------------- 2019-05-03 13:13:38: case.setup starting --------------------------------------------------- 2019-05-03 13:13:39: case.setup success --------------------------------------------------- 2019-05-03 13:14:27: xmlchange success ./xmlchange RESUBMIT=0 --------------------------------------------------- 2019-05-03 13:31:59: setup.clean starting --------------------------------------------------- 2019-05-03 13:31:59: setup.clean success --------------------------------------------------- 2019-05-03 13:32:11: case.setup starting --------------------------------------------------- 2019-05-03 13:32:12: build.clean starting --------------------------------------------------- 2019-05-03 13:32:28: build.clean success --------------------------------------------------- 2019-05-03 13:32:28: build.clean starting --------------------------------------------------- 2019-05-03 13:32:44: build.clean success --------------------------------------------------- 2019-05-03 13:32:45: case.setup success --------------------------------------------------- 2019-05-03 13:32:55: build.clean starting --------------------------------------------------- 2019-05-03 13:35:45: build.clean success --------------------------------------------------- 2019-05-03 14:58:20: case.build starting --------------------------------------------------- CESM version is cesm2.1.1-exp14 Processing externals description file : Externals.cfg Processing externals description file : Externals_CLM.cfg Processing externals description file : Externals_POP.cfg Processing externals description file : Externals_CISM.cfg Checking status of externals: clm, fates, ptclm, mosart, ww3, cime, cice, pop, cvmix, marbl, cism, source_cism, rtm, cam, ./cime clean sandbox, on cime5.6.15 ./components/cam clean sandbox, on cam1/release_tags/cam_cesm2_1_rel_26/components/cam ./components/cice clean sandbox, on cice5_cesm2_1_1_20190321 ./components/cism clean sandbox, on release-cesm2.0.04 ./components/cism/source_cism clean sandbox, on release-cism2.1.03 ./components/clm clean sandbox, on branch_tags/relcisofix.n01_release-clm5.0.20 ./components/clm/src/fates clean sandbox, on fates_s1.21.0_a7.0.0 ./components/clm/tools/PTCLM clean sandbox, on PTCLM2_180611 ./components/mosart clean sandbox, on release-cesm2.0.03 ./components/pop clean sandbox, on pop2/release_tags/pop2_cesm2_1_rel_n04 ./components/pop/externals/CVMix clean sandbox, on v0.93-beta ./components/pop/externals/MARBL clean sandbox, on cesm2.1-n00 ./components/rtm clean sandbox, on release-cesm2.0.02 ./components/ww3 clean sandbox, on ww3/release_tags/ww3_cesm2_1_rel_01 2019-05-03 15:09:09: case.build success --------------------------------------------------- 2019-05-04 09:05:38: case.run starting --------------------------------------------------- 2019-05-04 09:05:43: model execution starting --------------------------------------------------- 2019-05-04 19:10:50: model execution success --------------------------------------------------- 2019-05-04 19:10:50: case.run success --------------------------------------------------- 2019-05-04 19:11:04: xmlchange success ./xmlchange CONTINUE_RUN=TRUE --------------------------------------------------- 2019-05-10 08:51:01: st_archive starting --------------------------------------------------- 2019-05-10 08:51:26: st_archive success --------------------------------------------------- 2019-05-10 08:51:55: case.run starting --------------------------------------------------- 2019-05-10 08:52:01: model execution starting --------------------------------------------------- 2019-05-10 18:47:24: model execution success --------------------------------------------------- 2019-05-10 18:47:24: case.run success --------------------------------------------------- 2019-05-10 18:47:38: xmlchange success ./xmlchange CONTINUE_RUN=TRUE --------------------------------------------------- 2019-05-13 08:06:41: st_archive starting --------------------------------------------------- 2019-05-13 08:07:06: st_archive success --------------------------------------------------- 2019-05-13 08:07:25: case.run starting --------------------------------------------------- 2019-05-13 08:07:31: model execution starting --------------------------------------------------- 2019-05-13 17:53:31: model execution success --------------------------------------------------- 2019-05-13 17:53:31: case.run success --------------------------------------------------- 2019-05-13 17:53:44: xmlchange success ./xmlchange CONTINUE_RUN=TRUE --------------------------------------------------- 2019-05-14 08:03:09: st_archive starting --------------------------------------------------- 2019-05-14 08:03:35: st_archive success --------------------------------------------------- 2019-05-14 08:04:14: case.run starting --------------------------------------------------- 2019-05-14 08:04:21: model execution starting --------------------------------------------------- 2019-05-14 17:24:57: model execution success --------------------------------------------------- 2019-05-14 17:24:57: case.run success --------------------------------------------------- 2019-05-14 17:25:10: xmlchange success ./xmlchange CONTINUE_RUN=TRUE --------------------------------------------------- 2019-05-14 17:25:52: st_archive starting --------------------------------------------------- 2019-05-14 17:26:16: st_archive success --------------------------------------------------- 2019-05-14 19:36:20: case.run starting --------------------------------------------------- 2019-05-14 19:36:26: model execution starting --------------------------------------------------- 2019-05-15 05:28:37: model execution success --------------------------------------------------- 2019-05-15 05:28:37: case.run success --------------------------------------------------- 2019-05-15 05:28:50: xmlchange success ./xmlchange CONTINUE_RUN=TRUE --------------------------------------------------- 2019-05-15 05:29:16: st_archive starting --------------------------------------------------- 2019-05-15 05:29:43: st_archive success --------------------------------------------------- 2019-05-15 05:32:37: case.run starting --------------------------------------------------- 2019-05-15 05:32:45: model execution starting --------------------------------------------------- 2019-05-15 16:13:59: model execution success --------------------------------------------------- 2019-05-15 16:13:59: case.run success --------------------------------------------------- 2019-05-15 16:14:12: xmlchange success ./xmlchange CONTINUE_RUN=TRUE --------------------------------------------------- 2019-05-17 08:47:55: st_archive starting --------------------------------------------------- 2019-05-17 08:48:26: st_archive success --------------------------------------------------- 2019-05-17 08:48:59: case.run starting --------------------------------------------------- 2019-05-17 08:49:03: model execution starting --------------------------------------------------- 2019-05-17 19:00:48: model execution success --------------------------------------------------- 2019-05-17 19:00:48: case.run success --------------------------------------------------- 2019-05-17 19:01:02: xmlchange success ./xmlchange CONTINUE_RUN=TRUE --------------------------------------------------- 2019-05-18 20:14:13: st_archive starting --------------------------------------------------- 2019-05-18 20:14:41: st_archive success --------------------------------------------------- 2019-05-18 20:15:03: case.run starting --------------------------------------------------- 2019-05-18 20:15:09: model execution starting --------------------------------------------------- 2019-05-19 05:47:48: model execution success --------------------------------------------------- 2019-05-19 05:47:48: case.run success --------------------------------------------------- 2019-05-19 05:48:01: xmlchange success ./xmlchange CONTINUE_RUN=TRUE --------------------------------------------------- 2019-05-19 05:48:20: st_archive starting --------------------------------------------------- 2019-05-19 05:48:48: st_archive success --------------------------------------------------- 2019-05-20 08:30:40: xmlchange success ./xmlchange STOP_N=6 --------------------------------------------------- 2019-05-20 08:30:40: case.run starting --------------------------------------------------- 2019-05-20 08:30:47: model execution starting --------------------------------------------------- 2019-05-20 14:42:55: model execution success --------------------------------------------------- 2019-05-20 14:42:55: case.run error ERROR: RUN FAIL: Command 'mpiexec_mpt -p "%g:" -np 1692 omplace -tm open64 /glade/scratch/cmip6/b.e21.BSSP370cmip6.f09_g17.CMIP6-SSP3-7.0-SSP1-2.6Lu.001/bld/cesm.exe >> cesm.log.$LID 2>&1 ' failed See log file for details: /glade/scratch/cmip6/b.e21.BSSP370cmip6.f09_g17.CMIP6-SSP3-7.0-SSP1-2.6Lu.001/run/cesm.log.5982651.chadmin1.190520-083040 --------------------------------------------------- 2019-05-20 15:22:07: xmlchange success ./xmlchange STOP_N=6 --------------------------------------------------- 2019-05-20 15:22:08: case.run starting --------------------------------------------------- 2019-05-20 15:22:13: model execution starting --------------------------------------------------- 2019-05-20 21:38:22: model execution success --------------------------------------------------- 2019-05-20 21:38:22: case.run error ERROR: RUN FAIL: Command 'mpiexec_mpt -p "%g:" -np 1692 omplace -tm open64 /glade/scratch/cmip6/b.e21.BSSP370cmip6.f09_g17.CMIP6-SSP3-7.0-SSP1-2.6Lu.001/bld/cesm.exe >> cesm.log.$LID 2>&1 ' failed See log file for details: /glade/scratch/cmip6/b.e21.BSSP370cmip6.f09_g17.CMIP6-SSP3-7.0-SSP1-2.6Lu.001/run/cesm.log.5995670.chadmin1.190520-152208 --------------------------------------------------- 2019-05-21 09:00:29: xmlchange success ./xmlchange STOP_N=6 --------------------------------------------------- 2019-05-21 09:00:29: case.run starting --------------------------------------------------- 2019-05-21 09:00:37: model execution starting --------------------------------------------------- 2019-05-21 14:58:10: model execution success --------------------------------------------------- 2019-05-21 14:58:10: case.run success --------------------------------------------------- 2019-05-21 14:58:18: xmlchange success ./xmlchange CONTINUE_RUN=TRUE --------------------------------------------------- 2019-05-21 14:58:52: st_archive starting --------------------------------------------------- 2019-05-21 14:59:20: st_archive success ---------------------------------------------------