A Robustness Analysis of CMIP5 Models over the East Asia-Western North Pacific Domain

Article history: Received 18 February 2016 Revised 19 July 2016 Accepted 3 February 2017 Available online 31 October 2017 The Coupled Model Intercomparison Project (CMIP) is an international community-based infrastructure that supports climate model intercomparison, climate variability, climate prediction, and climate projection. Improving the performance of climate models over East Asia and the western North Pacific has been a challenge for the climate-modeling community. In this paper, we provide a synthesis robustness analysis of the climate models participating in CMIP-Phase 5 (CMIP5). The strengths and weaknesses of the CMIP5 models are assessed from the perspective of climate mean state, interannual variability, past climate change during the mid-Pliocene (MP) and the last millennium, and climate projection. The added values of regional climate models relative to the driving global climate models are also assessed. Although an encouraging increase in credibility and an improvement in the simulation of mean states, interannual variability, and past climate changes are visible in the progression from CMIP3 to CMIP5, some previously noticed biases such as the ridge position of the western North Pacific subtropical high and the associated rainfall bias are still evident in CMIP5 models. Weaknesses are also evident in simulations of the interannual amplitude, such as El NiñoSouthern Oscillation (ENSO)-monsoon relationships. Coupled models generally show better results than standalone atmospheric models in simulating both mean states and interannual variability. Multi-model intercomparison indicates significant uncertainties in the future projection of climate change, although precipitation increases consistently across models constrained by the Clausius-Clapeyron relation. Regional ocean-atmosphere coupled models are recommended for the dynamical downscaling of climate change projections over the East Asia-western North Pacific domain. © 2017 THE AUTHORS. Published by Elsevier LTD on behalf of the Chinese Academy of Engineering and Higher Education Press Limited Company. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).


Introduction
Climate models are useful tools in understanding the mechanisms of climate variability and in predicting and projecting future climate change. Due to the limitations of current state-of-the-art climate models in representing physical processes, uncertainties exist in the simulation results of climate models. A multi-model ensemble is a useful way to reduce the uncertainties of individual models. The Coupled Model Intercomparison Project (CMIP) is an international community-based infrastructure in support of climate model inter-comparison and climate change projection. Its most recent phase is CMIP-Phase 5 (CMIP5) [1,2]. The robustness of analysis of CMIP5 models has attracted an increasing amount of attention in recent years. The objective of the current study is to provide an overview of the performance of CMIP5 models over the East Asia-western North Pacific (EA-WNP) domain based on the published literature. The strengths and weaknesses of the models are assessed. This robust analysis aims to provide a useful reference on the creditability of CMIP5 models, and should assist in model development and improvement.

Climate mean state simulation
Benefiting from improvements in climate models in the past five years, the basic climatological features over the EA-WNP domain are reasonably simulated by the CMIP5 models [3]. For example, both the summer monsoon rainfall pattern and 850 hPa wind fields are reproduced well by the CMIP5 atmospheric models. The skill score of precipitation simulation has slightly improved, from 0.75 (CMIP3) to 0.77 (CMIP5), in the multi-model ensemble (MME) [4]. The improvements in precipitation simulation are closely related to the improvements in atmospheric circulation simulation. The CMIP5 atmospheric models successfully reproduce the climatological low-level southerly wind and high-level westerly jet over East Asia [5,6]. A large spread is seen among models in the simulation of large-scale circulations [7]. Nearly all CMIP5 models show a northward shift of the western North Pacific subtropical high (WNPSH), which leads to bias in the East Asian summer monsoon (EASM) rainfall band simulation [4,8].
The CMIP5 models are more skillful than the CMIP3 models in the simulation of the EA-WNP climate [9], although both CMIP3 and CMIP5 models produce slightly less precipitation than the observed [10]. This is evident in the pattern of summer monsoon rainfall, circulation, moisture transportation, and mid-tropospheric horizontal temperature advection [11], as well as in simulations of the onset of the summer monsoon [12,13]. The extension of the monsoon rain band over East Asia is underestimated, whereas the rainfall over the subtropical western/central Pacific Ocean is overestimated. Although the bias of the northward shift of the WNPSH ridgeline is also evident in CMIP3 models [14], coupled models general show better results than the standalone atmospheric models in the CMIP5 [6].

Interannual climate variability
The WNPSH is an important part of the EASM system. It has two dominant modes on the interannual time scale [15]. Both modes correspond to an anomalous anticyclone over the western North Pacific, but the anticyclone associated with the second mode is shifted northward relative to the one associated with the first mode [16]. He and Zhou [8] showed that the first mode is associated with sea surface temperature (SST) anomalies in the tropical Indian Ocean and central-eastern Pacific, while the second mode is associated with local SST anomalies. The first mode can be reproduced well by the CMIP5 Atmospheric Model Intercomparison Project (CMIP5-AMIP) simulations, indicating that it is a forced mode. However, the second mode cannot be perfectly simulated by the CMIP5-AMIP simulations. The simulated anomalous anticyclone in the MME or most individual models is far weaker than what is observed. This suggests that the second mode is associated with air-sea interactions over the tropical western North Pacific. There is substantial covariability between the WNPSH and the North Pacific subtropical high [17].
The EA-WNP monsoon is tightly linked with El Niño through a key low-level anomalous anticyclone over the tropical western North Pacific (the western North Pacific anomalous anticyclone, or WNPAC) [18]. The WNPAC is maintained over three consecutive seasons, from the El Niño mature winter to decaying summer [19]. Model performances in simulating the El Niño-Southern Oscillation (ENSO)-monsoon relationship are determined by their skill in simulating the WNPAC [20].
During an El Niño mature winter, the southwesterly anomalies to the northwestern flank of the WNPAC weaken the mean northeasterly wind of the East Asian winter monsoon [21]. Meanwhile, they transport moisture northward to Southeast China, greatly increasing the precipitation there [22]. About half of the CMIP5 coupled general circulation models (CGCMs) can reasonably simulate the WNPAC during an El Niño mature winter. However, nearly all models underestimate the positive precipitation anomalies over Southeast China [20].
WNPAC not only influences the East Asian winter monsoon (EAWM), but also modulates the temporal evolution of El Niño. The easterly anomalies on the southern flank of the WNPAC tend to stimulate oceanic upwelling Kelvin waves and thus accelerate the decay of El Niño [23,24]. The western North Pacific anomalous cyclone (WNPC), the counterpart of WNPAC during a La Niña winter, tends to be shifted westward relative to the WNPAC. Correspondingly, the westerly anomalies over the equatorial western Pacific during a La Niña winter are far weaker than the easterly anomalies during an El Niño winter. As a result, La Niña tends to decay much more slowly than El Niño [24]. This mechanism is supported by the results of the CMIP5 CGCMs. If a model can (or cannot) simulate the asymmetry between WNPAC and WNPC, it can (or cannot) simulate the asymmetry in evolution between El Niño and La Niña [20].
During an El Niño decaying summer, the WNPAC is maintained by the combined effects of local cold SST anomalies and remote forcing from the tropical Indian Ocean [25]. The CMIP5-CGCM MME supports the argument that the WNPAC and local cold SST anomalies form a damping coupled mode. The cold SST anomalies can only suppress local convection and thus maintain the WNPAC in the early summer, before they are damped completely by local negative feedbacks [20]. During the late summer, the maintenance of the WNPAC is primarily related to remote forcing from the tropical Indian Ocean through atmospheric Kelvin wave dynamics [25][26][27]. The CMIP5-CGCM MME indicates that the remote-forcing effect of the tropical Indian Ocean on the WNPAC is gradually enhanced with the establishment of the climatological monsoon trough from July to August [20].
Song and Zhou [4] systematically compared the CMIP5-AMIPs with the CMIP3-AMIPs in their simulations of the WNPAC and of the associated negative precipitation anomalies over the western North Pacific and positive precipitation anomalies extending from the middle and lower reaches of the Yangzi River to Japan. The CMIP5-AMIPs show higher capability and thus an improved simulation of the interannual pattern of the EASM (Fig. 1) [4]. The CMIP5-CGCMs have higher capability than the corresponding AMIP runs in their simulations of the WNPAC due to stronger remote forcing from the tropical Indian Ocean in the coupled models; this suggests that airsea interactions are essential in simulating the interannual variability of the EASM [6].

Past climate change simulation
Past climates provide an opportunity to establish constraints for East Asian monsoon (EAM) evolution and dynamics. Looking at geological analogues, the most recent warm climate associated with carbon dioxide (CO 2 ) values ((405 ± 50) ppm) that are higher than those of the modern climate is the mid-Pliocene (MP); therefore, the MP is considered as the potential analogue for understanding the future warming climate. For example, the MP Hadley circulation is regarded as a potential analogue for the future scenario [28][29][30]. For North China, the simulated EAM in the MP is demonstrated as an intensified EASM and as a weakened EAWM [31][32][33]. Both the simulated EASM and EAWM in the MP agreed reasonably well with geological reconstructions. The enhancement of the land-sea thermal contrast contributes to the intensified EASM in the MP [33]. The intensified EASM circulation brings stronger moisture transport into the East Asian domain, by increasing the local convergence of the stationary meridional velocity, resulting in an increase of the MP EASM precipitation in both the atmospheric general circulation model (AGCM) and the CGCM simulations [33].
The decadal-centennial variations of the EASM during the last forcing can cause changes over the tropical Pacific and North Polar regions, which play an important role in regulating the EAWM in the post-eruption winters [39].

Climate change projection
Much effort has been devoted to project the EN-WNP climate from the coming several decades until the end of the 21st century, using both global and regional coupled climate models [40][41][42][43][44][45][46]. For two typical scenarios of CMIP5, the RCP4.5 and RCP8.5, the annual mean surface temperature over China is projected to increase by (2.58 ± 0.78) °C and by (5.19 ± 1.10) °C, respectively, by the end of the 21st century. The summer rainfall, which is mostly contributed by the monsoon system, also evidently increases in most of the EN-WNP region, mainly due to increased moisture under warmer conditions, based on the Clausius-Clapeyron relation. The increase of annual rainfall over China during the end of the 21st century is projected at about (0.17 ± 0.10) mm·d −1 and (0.25 ± 0.12) mm·d −1 under the RCP4.5 and RCP8.5 scenarios based on CMIP5 multi-model results, which is about 8% and 11% of the millennium have been successfully simulated [34]. The EASM is generally strong during the Medieval Warm Period (MWP) and weak during the Little Ice Age (LIA). This result is consistent with the reconstruction performed from a stalagmite record in the Wanxiang Cave, China [35]. A comparison of the interannual variability mode of the EASM during the MWP, LIA, and 20th century global warming (20CW) reveals a similar rainfall anomaly pattern. However, the power spectra of the leading interannual variability modes during the three typical periods are different, and the biannual oscillation is most evident during the warm period [36].
Volcanic eruptions provide a good opportunity to observe the EASM response to external radiative forcing [36][37][38]. The East Asian continent is dominated by northerly wind anomalies, and the corresponding summer rainfall exhibits a reduction over East China following large volcanic eruptions (Fig. 2(a)) [36]. The cooling over the middle-high latitudes of the East Asian continent is stronger than that over the tropical ocean after such eruptions (Fig. 2(b)), which suggests a reduced land-sea thermal contrast, and which is favorable for a weak EASM circulation. It is found that strong tropical volcanic eruptions also have an important influence on the EAWM. Volcanic present-day amount, respectively.
The projected circulation is less sensitive than surface temperature and rainfall to greenhouse gas (GHG) forcing. Only a slight strengthening of the EASM southerly wind is observed in the MME due to the amplified land-sea thermal contrast near the surface and to faster warming of the northern hemisphere than of the southern [45,[47][48][49]], but the low-level wind convergence in the monsoon region is weakened [50]. The WNPSH is projected to be weakened and to retreat eastward in the mid-troposphere, based on the multimodel projected changes in 500 hPa wind, eddy geopotential height, and eddy stream function (Fig. 3) [51]. The eastward retreat of the WNPSH is accompanied by an eastward expansion of the East Asian subtropical rain belt [51]. The low-level circulation associated with the WNPSH seems to be unchanged based on the multi-model mean [41,45,51,52]. However, low-level monsoon westerlies in subtropical regions may accelerate under warmer conditions [53]. For both the EASM circulation and rainfall, interannual variability is projected to increase in the 21st century [54].
Extreme climate, including heat waves and heavy rainfall, shows a similar trend as the mean state of the surface temperature rises and as rainfall increases, except for some sub-regions [42,43,[55][56][57][58]. Even under the moderate scenario, the risk of a hot summer in China is projected to increase significantly [59,60]. Extreme precipitation in Northeast China and the Tibetan Plateau is shown to increase while decreasing precipitation is found in Southeast China; this finding is evident in both regional and global climate models [43]. The Palmer Drought Severity Index under the RCP8.5 scenario shows a similar trend in these regions [61].
The projected changes in the EA-WNP climates have large uncertainties, coming from the unknown emission scenarios, internal variability, model climate sensitivity, and parameter uncertainties. The land-warming magnitude over East Asia is mainly determined by the global mean warming, which is controlled by different feedback processes [62]. The contribution of internal variability to the uncertainty can be up to 30% in regional-scale extreme rainfall, but varies from region to region [63]. Different parameterized convective schemes may shift the link between temperature and precipitation away from the Clausius-Clapeyron relation by modulating the thermodynamic constraint, resulting in larger uncertainties [64]. The large uncertainty found in the projected WNPSH may be associated with the zonal gradient of SST change between the Indian Ocean and the western Pacific [52]. Different Pacific warming may be critical to the large uncertainty in the projected onset of the EASM [13,65].

Added values of regional climate models
High-resolution regional climate change information is necessary in order to assess the impacts of climate change on human and natural systems. The international Coordinated Regional Climate Downscaling Experiment (CORDEX) project is a multi-model CMIPlike effort with the goal of delivering useful regional climate change information † . Regarding the regional climate models (RCMs) participating in the CORDEX with a focus on East Asia ‡ (hereafter CORDEX-EA), nearly all were prescribed by the SST derived directly from the driving general circulation models (GCMs) [66][67][68]. This result indicated that the regional air-sea coupling processes were not included during dynamical downscaling. Recent progress suggested that compared with the standalone RCM, the RCMs that include regional air-sea coupling yield a significantly improved simulation of rainfall and circulation over the Asian summer monsoon domain in terms of both climatology and interannual variability [69][70][71][72][73][74][75][76][77].
Therefore, Zou et al. [78] applied a flexible regional oceanatmosphere-land system (FROALS) model to the CORDEX-EA domain. This model is driven by the output of the historical simulations and future climate projections derived from GCMs. A validation of the present-day climate simulated using the FROALS model showed that the performance of the FROALS model is better than that of the corresponding standalone RCM for simulating the climatology and interannual variability of summer rainfall over East China for the period of 1981-2005 [78]. For projected climate change under the RCP8.5 scenario, the spatial patterns of the projected rainfall changes by the FROALS model were generally consistent with those from the driving GCMs at a broad scale, due to similar projected circulation changes. The enhanced southerlies over East China increased the moisture divergences over South China and enhanced the moisture advection over North China. However, the atmosphereonly RCM exhibited responses to the underlying SST warming anomalies that were too strong, which induced an anomalous cyclone over the northern region of the South China Sea, followed by increases (or decreases) of total and extreme rainfall over South China (or central China) [79]. These results demonstrated that the regional atmospheric model was in better equilibrium with the underlying sea surface forcing in the regional ocean-atmosphere coupled model than in the SST-prescribed RCM, indicating the advantages of the inclusion of regional air-sea coupling in the dynamical downscaling of present and future climate changes over the CORDEX-EA domain [78,79]. The results also suggested that the regional oceanatmosphere coupled model is a useful tool for the dynamical downscaling of climate change over the CORDEX-EA domain [78,79].

Summaries
The CMIP5 provides an international community-based infrastructure in support of climate change and climate variability studies. This review provides a robustness analysis of CMIP5 models over the EA-WNP domain. Major findings are summarized below: (1) CMIP5 models are more skillful than CMIP3 models in the simulation of EA-WNP mean climate in the context of both circulations and rainfall. Coupled models generally show better results Fig. 3. Projected future change of the summertime 500 hPa mean state over the western North Pacific. The shading is the change in geopotential height, and the vector is the change in wind. Changes in wind that are agreed upon by more than 75% of the models are stippled. The boundary of the subtropical high is indicated by the zero contour of the eddy geopotential height (solid line) and by the zero contour of the eddy stream function (dashed line), for the 20th century (blue line) and 21st century (red line), respectively. Hist: historical scenario. (After Ref. [51]) † http://www.cordex.org/ ‡ https://cordex-ea.climate.go.kr/main/modelsPage.do than standalone atmospheric models. Some systematic bias, such as the northward shift of the WNPSH and the related bias of the rainfall pattern, are still evident and pose a challenge for the climatemodeling community.
(2) The interannual variability of the EA-WNP monsoon is tightly linked with El Niño through a low-level anomalous anticyclone over the tropical western North Pacific (i.e., WNPAC). About half CMIP5-CGCMs simulate the WNPAC during an El Niño mature winter reasonably well, but nearly all models underestimate the positive precipitation anomalies over Southeast China. During an El Niño decaying summer, the WNPAC is maintained by the combined effects of the local cold SST anomalies and remote forcing from the tropical Indian Ocean. The CMIP5-CGCM MME supports the argument that the WNPAC and local cold SST anomalies form a damping coupled mode. The CMIP5-CGCM MME shows better skills than the CMIP5-AGCM MME in the simulation of the WNPAC, indicating the crucial role of air-sea interaction.
(3) Both the MP and the last millennial climate are useful metrics for gauging climate models' performance. The intensified EASM and weakened EAWM in North China that were simulated as occurring during the MP, based on the Pliocene Model Intercomparison Project (PlioMIP), agreed reasonably well with geological reconstructions. The stronger EASM that was simulated as occurring during the MWP and the weaker EASM simulated during the LIA are consistent with the reconstruction from a stalagmite record.
(4) Although uncertainties exist, the rainfall under two CMIP5 scenarios, RCP4.5 and RCP8.5, is projected to increase in most of the EA-WNP region due to increased moisture under warmer conditions, based on the Clausius-Clapeyron relation. The WNPSH is projected to be weakened in the mid-troposphere, whereas the circulation near the surface seems to be unchanged, based on the multi-model mean. Extreme events, including heat waves and heavy rainfall, show a similar trend as the mean state of surface temperature rises and as rainfall increases.
(5) An analysis of CORDEX-EA models shows significant added values in the simulation of extreme rainfall events. The evidence highlights the necessity of employing regional air-sea coupling in the dynamical downscaling of present and future climate change over the CORDEX-EA domain.