From data compilation to model validation: a comprehensive analysis of a full deep-sea ecosystem model of the Chatham Rise

The Chatham Rise is a highly productive deep-sea ecosystem that supports numerous substantial commercial fisheries, and is a likely candidate for an ecosystem based approach to fisheries management in New Zealand. We present the first end-to-end ecosystem model of the Chatham Rise, which is also to the best of our knowledge, the first end-to-end ecosystem model of any deep-sea ecosystem. We describe the process of data compilation through to model validation and analyse the importance of knowledge gaps with respect to model dynamics and results. The model produces very similar results to fisheries stock assessment models for key fisheries species, and the population dynamics and system interactions are realistic. Confidence intervals based on bootstrapping oceanographic variables are produced. The model components that have knowledge gaps and are most likely to influence model results were oceanographic variables, and the aggregate species groups ‘seabird’ and ‘cetacean other’. We recommend applications of the model, such as forecasting biomasses under various fishing regimes, include alternatives that vary these components.


32
The goal of incorporating a holistic approach to understanding the system-wide repercussions  The process of developing this model was not linear, but rather iterative and incremental. 116 There were five main stages to the development, each of which was re-visited until we were    An Atlantis model requires the modelled region to be split into polygons and depth layers. 147 Each polygon/depth layer is referred to as a cell. The intention of the splits is to capture 148 important aspects of the region but at a simplified level such that modelling the region over 149 many years becomes possible. If we were modelling a smaller temporal scale, we may have       can be seen in Figure 4. When biomass pools are tracked in the model, they are done so in 207 mg N m −3 . When a fish (for example) eats another fish, it is nitrogen that is transferred up 208 the food chain, with some nitrogen going to detritus and carrion, thus providing nitrogen to 209 micro-organisms and filter feeders to fuel the cycle over again.      scenarios explored using this model should consider sensitivities for these.    Atlantis simulates growth rates of age-structured groups as a function of consump-313 tion. If growth is too slow, there may be insufficient food available, the feeding search 314 rate could be too low or handling time too high, and the reverse of these when growth 315 is too fast. Simulated growth rates of age-structured species groups were assessed by 316 comparing the simulated size-at-age with those expected based on growth curve esti-317 mates from the literature ( Table 6). The overlaid simulated and 'observed' figures were 318 generally very similar (Appendix B). For each species group, we estimated CVs required 319 to satisfy the hypothesis that the modelled size-at-age were not significantly different 320 from the 'observed' with probability of 0.95. The required CVs were all less than 30% 321 except for epibenthic fish (deep and shallow), invertebrate herbivore (commercial), in-322 vertebrate scavenger (commercial), ling, rock lobster and small pelagic fishes. For all 323 these groups, the first age class, and sometimes the first few, were larger in size than 324 expected. Deep epibenthic fish were larger than expected at all age classes, but for all 325 other groups the characteristic of larger than expected size at age had been remedied 326 by the time they were adults.

328
Natural mortality in the model consists of mortality intrinsic within the model from 329 predation, starvation, and light, oxygen or nutrient deprivation, and additional forced 330 mortality. The latter was applied for modelled species groups that would not otherwise suffer sufficient natural mortality within the model, such as those that have little 332 known predation. Age-structured simulated natural mortality rates from the stable base 333 model were compared to estimates of M from the literature where available (Table 6)

397
Understanding which species groups are most influential or responsive in the model is  (1) There were four species groups that stood out as having more effect than the other 433 groups: orange roughy, hoki, pelagic fish small (primarily myctophids) and spiny dog-434 fish. These remain the top four for keystoneness, but the order changes due to the 435 proportional biomasses (Figure 9). 436 We calculated responsiveness in a similar way to keystoneness, but from the perspec-437 tive of the response group (Equation 5).
R g responsiveness of group g to increased mortality in all other groups The most responsive group was pelagic fish small (primarily myctophids), followed      The fisheries were modelled with six fleets, defined in Table 7. The demersal line  Manuscript to be reviewed  Table 7.      The orange bars are the skill metrics with respect to the trawl survey 95% confidence intervals.

Comparison with fisheries CPUE and Stock Assessment in-
The grey horizontal lines in the MEF and RI figures mark the value for a perfect fit, which is 1 for both of these.
8 Bringing it together 540 We qualitatively graded the species groups by how well they performed in the model 541 and how well informed they were by data, information and other research (referred to 542 as 'informance'). We compared these gradings with the keystone and responsiveness 543 from Section 5.2. Figure 15 gives a visual guide for how well the most influential or 544 responsive species groups did for informance and performance. While poor knowledge 545 may not be concerning if paired with high responsiveness providing keystoneness is low 546 (since the effects may be more limited to this single species group), the triple of highly 547 responsive, a keystone species, and poorly defined may need consideration for future 548 scenarios.

550
The groups that were highest for keystoneness and highest for informance and perfor-551 mance were hoki, orange roughy, benthic fish shallow (primarily oblique banded rattail), 552 and hake. These all have abundance indices available, biological parameters, diet infor-553 mation, and all perform well with respect to these in the model. Hoki, orange roughy 554 and hake (groups 1, 2, and 10 for keystoneness) have full stock assessments, which the 555 model matches well. These are important groups for fisheries and will likely feature 556 strongly in any fisheries scenarios explored with this model.

Manuscript to be reviewed
Species groups Pelagic fish small (primarily myctophids) and Pelagic fish medium 559 (primarily barracouta) were both high with respect to keystoneness and responsiveness, 560 and while both were fairly well defined, these had some areas of poor model performance 561 and do not have abundance indices to compare. The estimated length at age 1 from 562 CRAM for small pelagic fish is larger than expected. This may be due to the size of 563 recruits being larger than they should be, or the fish eating (and hence growing) more 564 than they should in this first year. They are not so big that the effect transfers to the 565 age-2's, as the age-2's are the correct size (Appendix B), so this is probably not influ-566 ential on the model overall. Medium pelagics have slightly less natural mortality in the 567 model than they should (Appendix C), and may be less responsive to fishing mortality 568 as a result. As they are 7 th with respect to keystoneness and high for responsiveness, 569 they could affect scenario outcomes and are worth considering when analysing results.

570
They make up approximately 1% of the age-structured biomass.  (Table 2). Scenarios 586 explored in the future may benefit from sensitivity analysis with respect to these two 587 groups to understand their effect on the outcomes, or perhaps some more work to better 588 define them. No data gaps, performed well, abundance index available Slight data gaps and/or poor performance Some data gaps and/or poor performance Poorly specified  gives confidence that the model can respond to fishing in a way that is realistic, and that 605 the ecosystem effects relative to these species are realistic. The stock assessment models 606 fit data such as proportions at length and biomass indices with the help of between-607 year recruitment deviates, which are not present in the Chatham Rise Atlantis model.

608
Conversely, the stock assessment models do not have time-varying natural mortality or 609 growth rates, which are present in the Chatham Rise Atlantis model. As such, both 610 modelling approaches achieve similar results but in very different ways. It is possible 611 that the recruitment deviates in the stock assessments are proxy's for the other ecosys-612 tem dynamics that the Atlantis model is able to capture (or vice versa). However, the 613 Atlantis model is too complex to fit comprehensively to data and is entirely determin-614 istic. Hence, the Chatham Rise Atlantis model's ability to achieve the same results as 615 the stock assessment models, that were fitted to data, is the best outcome. Rise Atlantis model has realistic diet summaries for all species groups, and the top 627 keystone species groups were all those we would expect to be most influential within 628 this ecosystem. This is not to say the model could not benefit from further future work 629 examining the realised diets at a finer scale -spatially, temporally, and by age-class.

631
Exploring the models sensitivity to initial conditions, while not an insignificant 632 amount of work, may be worth doing at some stage in the future to add to our un-633 derstanding of the models stability and persistence of dynamics. This has not, to the 634 best of our knowledge, been done for Atlantis or OSMOSE models, likely due to the 635 enormous complexity and computing resources required for the task. Sensitivities to ini-636 tial conditions have been explored using Ecopath (Essington, 2007) and Ecopath with       Ben fish deep  Lookdown dory  (Table 6) where available (orange 924 shaded shows 95% confidence intervals using CV 10%) and from CRAM simulated years