Mathematical modelling for variations of inbreeding populations fitness with single and polygenic traits

Inbreeding mating has been widely accepted as the key mechanism to enhance homozygosity which normally will decrease the fitness of the population. Although this result has been validated by a large amount of biological data from the natural populations, a mathematical proof of these experimental discoveries is still not complete. A related question is whether we can extend the well-established result regarding the mean fitness from a randomly mating population to inbreeding populations. A confirmative answer may provide insights into the frequent occurrence of self-fertilization populations. This work presents a theoretic proof of the result that, for a large inbreeding population with directional relative genotype fitness, the mean fitness of population increases monotonically. However, it cannot be extended to the case with over-dominant genotype fitness. In addition, by employing multiplicative intersection hypothesis, we prove that inbreeding mating does decrease the mean fitness of polygenic population in general, but does not decrease the mean fitness with mixed dominant-recessive genotypes. We also prove a novel result that inbreeding depression depends on not only the mating pattern but also genetic structure of population. For natural inbreeding populations without serious inbreeding depression, our theoretical analysis suggests the majority of its genotypes should be additive or dominant-recessive genotypes. This result gives a reason to explain why many hermaphroditism populations do not show severe inbreeding depression. In addition, the calculated purging rate shows that inbreeding mating purges the deleterious mutants more efficiently than randomly mating does.


Background
In genetic terminology, inbreeding is the breeding of two individuals who are related to each other. Here inbred is defined as the individual that is generated from inbreeding. The closeness of individuals is important in inbreeding, which has substantial influence on the fitness of offspring individuals. Inbreeding depression refers to the reduced survival and fertility of offspring of related individuals. An important mechanism that leads to inbreeding depression is overdomain in which the heterozygote has *Correspondence: tianhai.tian@monash.edu School of Mathematical Sciences, Monash University, VIC 3800 Melbourne, Australia greater phenotype value and perhaps is more fit than the homozygous state for either of the alleles that it comprises. Therefore there are two major approaches to increase homozygosity but decrease fitness. The first method is to increase homozygosity for partially recessive detrimental mutations; while the second approach is to increase homozygosity for alleles at loci with heterozygote advantage (overdominance). However, it is still under debate which mechanism is more dominat in nature [1][2][3][4][5]. This is an important question in genetics because two differnet mechanisms will lead to different theories regarding the trait values of the crossbred progeny. If the overdominance theory is valid, the mean trait value of crossed lines will return to the equivalence of the outbred population, as heterozygosity will be restored. On the other hand, the partial dominance theory predicts that the mean trait value will exceed that of the outbred population. In this case, in addition to the restore of heterozygosity, crossbred individuals will be purged from their genetic load [6,7].
Genetic purging is the process for frequency reduction of a deleterious allele. It has been accepted that purging was achieved when inbreeding depression is caused by deleterious recessive alleles. A fitness rebound in inbred populations provides evidence to support the partial dominance mechanism [8,9]. The effect of genetic purging have been observed in a number of experimental studies [10,11], but overall, the evidence for purging in plant and animal populations is still limited. In addition, the combined effect of both inbreeding and purging is important for evaluating the evolutionary consequence of inbreeding and for making recommendations in conservation [12]. Although the inbreeding-purging model has been proposed to predict the evolution of the fitness and inbreeding load [13], a number of model parameters should be estimated from observation data. Thus it is high demanded to explore the role of purging in restoring fitness and calculation of purging speed.
The number of phenotypes produced for a given trait depends on how many genes control the trait. There are two types of trait. The single gene trait is controlled by a single gene that has two alleles. On the other hand, polygenic trait is controlled by two or more genes and has 2 or more alleles. Unlike monogenic traits, polygenic traits do not follow patterns of Mendelian inheritance (discrete categories). Instead, their phonotypes vary along a continuous gradient, such as the height of animals and the colour of human skin. Although we have conducted theoretical study for the single trait [14], the study of inbreeding depression for polygenic traits is more challenging and there is limited theoretical result regarding the variation of mean fitness under inbreeding.
In this work we will conduct novel theoretical studies for inbreeding depression in large population with Mendelian and polygenic traits. First, for single locus, we will explore the conditions for the monotonical increase of the mean fitness. We also derive a formula to calculate the purging rate. In addition, for multiple loci, we will find conditions for inbreeding mating reducing the mean fitness of the population. Furthermore, this work will show mechanisms to determine inbreeding depression. As an application, we present additional reasons for the frequent occurrence of self-fertilization populations. The well-known reason is that self-pollination does not need to be visited by animals to produce seed and hence have ecological advantageous under some circumstances [15].

Inbreeding of Mendelian traits
A Mendelian trait is a single trait that is regulated by a single locus and shows a simple discrete inheritance pattern. According to Wright's formula [5,16], at one autosomal locus of two alleles A and a with frequency x and 1 − x, the three diploid genotypes AA, Aa and aa have frequency respectively, where f is the inbreeding coefficient that is defined as the probability that two homologous alleles in an individual are identical by descent (IBD) [5,17]. In addition, the fitness of AA, Aa and aa is denoted as w 11 , w 12 , w 22 , respectively. Thus the change of frequency where x is the frequency of A in the next generation, and the mean fitness is given by For the difference w between the mean fitnesses of two successive generations, we have shown in [14] that where w = w − w, w is the mean fitness of the next generation and Equation (2) suggests that a sufficient condition for Thus, for random mating (f = 0), the mean fitness increases monotonically. Similar observation can be found for the case w 11 = w 22 even if f = 1. For example, using a different approach, Ziehe and Roberds [18] has shown that, when caused by symmetric homozygous disadvantage at single locus, inbreeding depression is always less than one-third [16]. Here "symmetric" means w 11 = w 22 . Thus for the populations considered in [18], the mean fitness increases monotonically.

Inbreeding depression
Now we discuss the variation of mean fitness due to the inbreeding mating. To be consistent with the published literature, from now on, we always make the three genotypes of a population as (1, 1 − hs, 1 − s) in the sequel. Thus the mean fitness w o with an initial frequency x of a randomly mating population is and the mean fitness w s of x for the inbreeding mating with inbreeding coefficient f is The inbreeding depression is defined by Based on this definition the inbreeding coefficient is a function of inbreeding coefficient f, frequency x, degree of dominance h and selection coefficient s, namely δ = δ(x, f , h, s). We first have the following theory for the changes of mean fitness.

Proposition 1
Let the genotypes of an inbreeding population be (1, 1 − hs, 1 − s). Then inbreeding mating reduces the mean fitness for the following population: 1. the population with overdominance genotype, or with a partial dominant genotype. That is, inbreeding depression occurs. 2. the population with under-dominance genotype, or with a partial recessive genotype. In this case, no inbreeding depression occurs.
It is well known that an overdominant population has an internal equilibrium. But it is not true for inbreeding populations. In fact, a population with genotypes (1, 1−hs, 1−s) has an internal equilibrium if and only if the equation has a solution in (0, 1). Thus the following Proposition gives the condition for the non-existence of the internal equilibrium.

Proposition 2 An inbreeding population with overdominant genotypes
For example, if the degree of dominance satisfies −1 < h < 0, there would not be any internal equilibrium for a population of self-fertilization plants with f = 0.5.

Multiplicative interactions for polygenic traits
Genes may interact to or regulate each other in a number of different ways. However, genes normally interact multiplicatively for the fitness-reducing effects of homozygosity for deleterious alleles, namely mutant alleles or alleles at loci with overdominance [19,20,24]. Accordingly the relative viability w of an individual over all loci is assumed to be the product of the viability value w A for individual locus A, namely w = w A w B w C · · · . All the loci are assumed to assort independently, namely they are unlinked. It is also assumed that there is no mutation at these loci over the period of inbreeding. In addition, we assume that the selective advantage s and initial frequency are the same for all loci. Then each w A can be written as a function of h(≤ 1/s), given by Then the overall inbreeding mean fitness is and that for randomly mating is given by To proceed an theoretic analysis, we write the values of these mean fitness as

Increase of mean fitness for Mendelian traits
We have described the methods for measuring the fitness of inbreeding population in the previous section. The following theorem gives the condition for the increase of mean fitness.

the mean fitness increases monotonically for any inbreeding coefficient
Proof For the difference between the mean fitness of two successive generations (2), we have To prove w > 0, it suffices to show If 0.5 ≤ h ≤ 1, then the inequality (4) holds since f fh+1−h ≤ 1. On the other hand, when 0 ≤ h ≤ 0.5, using the identity we can check the inequality (4) still holds. Thus we have showed that w ≥ 0 for any initial frequency x and any value of h satisfying 0 ≤ h ≤ 1. The proof is completed. Figure 1 clearly shows that the mean fitness of both randomly mating and inbreeding populations increase monotonically although at each generation, the mean fitness of randomly mating population is higher than that of inbreeding one. This result is consistent with the wellestablished result for a random-mating population [17]. It also gives the reason for the existence of self-fertilization plants, even though the most extreme form of inbreeding may reduce the heterozygosity of these plants by 50% per generation. Classically, it was known that selfing has two primary advantages over outcrossing [21], namely the reproductive assurance to ensure seed production and transmission advantage by serving as pollen donors for other individuals and for themselves [15]. Since most genotypes of a self-fertilization plant are either partial dominance or partial recessive [8,[22][23][24], Theorem 1 presents an additional explanation: self-fertilization population increases its mean fitness of population by generating steadily. However, Theorem 1 cannot be extended to the inbreeding population with h > 1 (overdominance) or h < 0 (underdominance). The counter examples are given in Table 1, reference [14] or Fig. 2 below.
On the other hand, the monotonically increasing for mean fitness of a population is not the necessary condition for the population avoiding extinct. Our next goal is to define a quantity which characterize the condition  whether a population becomes extinct. To this regard, the mean fitness is denoted as w(x) to emphasize the importance of the initial frequency x. Then we consider the sum (2). The averaged mean fitness (AMF) of a population is defined by The biological meaning of AMF is that AMF > 0 holds if and only if the population does not become extinct. Thus Theorem 1 in fact shows that AMF for directional inbreeding population is always positive because in this case w(x) > 0 for each x = 0. The following theorem gives the value of AMP without any additional condition.

Theorem 2 The AMF of any inbreeding population is positive.
Proof The proof is given in Appendix section.
Figure 2b and 2d schematically shows this point, where the genotype fitness is (1, 1.4, 0.8) and inbreeding coefficient f = 0.25. From Fig. 2b, when the population goes to the 8th generation, the frequency x becomes 0.6338 and the mean fitness is 1.0831. Then the mean fitness starts to decrease (that is, w(x) < 0) until x reaches to the equilibrium (x * ≈ 0.6667 in this case). On other hand, Fig. 2d clearly shows that the positive area which is above the xaxis (and below the curve) is much larger than the negative area which is below the x-axis and above the curve. These figures show that AMF > 0 intuitively.

Inbreeding depression
For inbreeding depression, we give the theorem which provides insights into inbreeding mating.  Remark 1 This result may provide an explanation to the puzzle for the classical strategy of cultivation. When choosing the offspring (i.e. seeds) of a favourable mutation and then inbreeding between the improved offspring, our results suggest that this inbreeding mating greatly increases the mean fitness of the population. This interesting result explains biological observations but is contradict to the belief that inbreeding mating decreases mean fitness. The key point is that in this case the initial frequency of the favourable mutation normally is very low. However, if the first generation of the population still uses the same selection strategy with the favourable mutation, the initial frequency will be very high, because they all already have the favorable mutation. However, the following Theorem 5 shows that the inbreeding mating for the second generation actually reduces the mean fitness.

Purging rate
Although selection against highly recessive alleles become more effective due to the rise of selfing rates [25], the amplified selfing rates have little impact on the alleles that are more nearly additive (i.e. h = 0.5). It was wellknown that, for a population with directional selection (i.e., 0 ≤ h ≤ 1), inbreeding mating purges deleterious mutants. Here we propose a formula to determine the value of purging rate. For an inbreeding population with genotype (1, 1 − hs, 1 − s) and inbreeding coefficient f, the average time t(x, x t ) required for allele to change the frequency from x to x t is dx. Figure 3 gives the calculated time t (x, x t ) based on different genotypes and different frequency x t . Numerical results show that the random mating population need more time than the inbreeding population from a fixed initial frequency x = 0.9 to a new frequency, which suggests that inbreeding mating purges deleterious mutants more effectively than randomly mating. In particular, Fig. 3(c) suggests that the purging strength of the inbreeding population is substantial higher than that of the random mating population. In addition, Comparing the three cases with h = 0, 0.3, 0.8, Fig. 3 suggests that a small value of h leads to a large purging rate. These results may provide a reason for the existence of so many selfing fertilisation plants, though these plants have various degree of inbreeding depression. This may be one of the advantages of selfing fertilisation plants.

Multiplicative interactions for polygenic traits
We first give an example to show that the theory established in the previous section may not be able to explain some experimental observations. For example, for an selfing population with inbreeding coefficient f = 0.5, nature selection pressure s = 0.05 and h = 0 (i.e. a completely dominant mutant), the inbreeding depression at a single locus (3) is δ = 0.006. However, the observation in natural populations shows that a completely recessive mutant usually will cost around 20 − 30% inbreeding depression [9,23,24]. This difference suggests that the trait may be determined by multi-genes.
In our previous study, we proposed a theorem for the inbreeding mating without proof. The following result is an updated version of Theorem 3.1 in [14].

Theorem 4
Assume that a polygenic trait is determined by the genotypes whose relative fitness is (1, 1 − hs, 1 − s), where the selective advantage coefficient s is fixed and the degree of dominance h is uniformly distributed over [ 0, 1] (i.e., the trait is determined by the dominant-recessive genotypes). Then inbreeding mating does not decrease the mean fitness of population.
Note that the inbreeding depression δ is an allelespecific property. However Theorem 4 shows a genotypespecific property which is independent of the choice of alleles.
From Proposition 1, we immediately have the following results.

Proposition 3 If a polygenic trait is determined by partial dominant or over-dominant genotypes, inbreeding mating reduces the mean fitness of the population.
Theorem 5 Assume that a polygenic trait is determined by the genotypes whose relative fitnesses are (1, 1 − hs, 1 − s), where the selective advantage coefficient s is fixed and the degree of the dominance h is uniformly distributed over [−1, 1]. Then inbreeding mating reduces the mean fitness of the population for all initial frequency x satisfying Proof The proof is given in Appendix section. Recent studies have showed that there are many overdominant loci in rice [3,4,26]. In addition, overdominance and epistasis might play an important role as the genetic basis of heterosis in Brassica rapa [27]. The following theorem investigates the relationship between the number of genotypes and the mean fitness.

Theorem 6
If the set of genotypes which determine a polygenic trait contains enough over-dominant genotypes, inbreeding mating reduces the mean fitness of the population for any given initial frequency x = 0, that is, Proof The proof is given in Appendix section.
However, in natural populations, there may not be many overdominance genotypes [28]. Another restriction for over dominance (1, 1 − hs, 1 − s) is that h may not be very small. A well-known example is Sickle cell anaemia which occurs when a particular pair of genes carry the 'sicklecell trait' which has not been eliminated from the human population by selection. The reason is that there is only one pair of genes carrying the sickle-cell trait and individuals ("carrier") is highly resistant to malaria. On the other hand, a person whose genes do not carry the sickle-cell trait is susceptible to malaria. For example, according to the data from the World Health Organization, the carrier frequency ranges from 10 to 40% across equatorial Africa. The genotype's fitness is 1, 1 − h and 0 with −1.35 ≤ h ≤ −1.05. Thus h (≥ −2s, s = 1) is not very small.

Discussions
We now discuss other factors for determining inbreeding depression. Suppose that initially we have two homozygous, inbred lines P 1 and P 2 which are used as parents. LetP 1 >P 2 , whereP is the mean fitness or fitnessrelated trait of P. The genotypes of P 1 and P 2 for this gene are A + A + and A − A − while their hybrid is A + A − . Let the average phenotype of the two parents be m and the additive and dominance genetic component of means (the averaged phenotypic value) be a A =P 1 − m and d A =F 1 − m , respectively, where F 1 is the first generation of hybrid offsprings of P 1 and P 2 andF 1 is the mean of the trait of F 1 . Suppose d A > a A > 0, that is, P 1 is overdominant to P 2 . If inbreeding mating, the mean value is P 1 +P 2 /2 = m. However, for cross mating, the mean value isF 1 which is greater than m. Consider a natural population consisting of equal number of P 1 and P 2 . Then inbreeding reduces the mean value of fitness or fitness-related, namely this population exhibits inbreeding depression. On the other hand, Theorem 4 shows that a natural population with few over-dominant genotypes does not exhibit severe inbreeding depression. The next result provides a reason to explain this difference.

Theorem 7 If a natural inbreeding population does not show a high level of inbreeding depression (i.e., δ ≤ 0.2), then most fitness or fitness related loci (QTL) of it exhibits additive and dominant-recessive genotypes.
We conclude that inbreeding depression is determined by not only the inbreeding coefficient but also the genetical structure of the population. This result is consistent with the observation that, in many mixed-mating plant populations, selfing (≈ 0.2), outcrossing (≈ 0.4) and the interm1diate (≈ 0.4) reaches at an equilibrium [29].
Let us consider an example. Goodwillie and Knight [2] measured inbreeding depression in three populations, namely Lake Hennessey (LH), Wantrup Researve (WR) and Ida Clayton Road (IC) of Leptosiphon, whose mean outcrossing rate are 0.06, 0.37 and 0.69, respectively. Significant inbreeding depression was observed for the proportion of fertilized ovules that developed into seeds only occurred in the more outcrossing WR and IC populations. Theorem 7 predicts that most genes in population LH are additive and dominant-recessive and both LH and WR have many over-dominant genes.
Another novel contribution of this work is the study for inbreeing depression with polygenic traits. Our theoretical results have shown that inbreeding depression also depends on the structure of the genotypes. The theoretical results support the belief that inbreeding depression is caused more like by over-dominant mechanism rather than dominance-recessive mechanism. An application of these results is to explain why Caenorhabditis remanei has demonstrated to suffer severely from inbreeding depression, while its hermaphroditic relative C. elegans has not [22,30]. In addition, Theorem 5 shows that inbreeding depression is also determined by the initial frequency, which, for cultivation of improved varieties with a favorable mutation, explains why inbreeding mating does not reduce the mean fitness of the first generation, but does reduce the mean fitness of the second generation. These results may have potential application for cultivation of improved varieties.

Conclusions
In this work we developed mathematical approaches to investigate the the variations of inbreeding population fitness under various conditions. For inbreeding population with a single locus, we proved that the mean fitness increases monotonically for directional selection, which extends the existing result for the random mating substantially [17]. Our results showed that, if most genotypes are additive, inbreeding mating (selfing) does not produce inbreeding depression. In addition, we defined the averaged mean fitness (AMF). Using this concept, we have successfully shown monotonical increase of the averaged mean fitness for any inbreeding population. These results can be applied to study the Mendelian trait or unlinked polygenic trait, and provide the reason to explain why inbreeding mating does not have serious impact on the mean fitness of population. For inbreeding depression with single trait, we have presented an explicit formula to calculate purging rate. Computing results suggest that inbreeding mating does purge deleterious mutants more efficiently than randomly mating does.
In addition, let t ≥ s and write Then P(f ) is a decreasing function of t since Hence we have, for f ≤ 0.8, Also note that P f t=1 is a decreasing function of f and P f t=1 f =0.8 > 0 for s ≥ 0.89. Thus we further have For f ≥ 0.8, we have and note that the left hand side function of above equation is increasing for f ≥ 0.8 and However, for f ≥ 0.85, we have By repeating this procedure, we can prove S f > 0 for f < 1. Finally, if w 11 = w 22 , it is easy to check that Note that the unique equilibrium and the mean fitness reaches its maximum value at In addition, w(x) < 0 holds if and only if x ∈ (x, x * ) withx .
A similar argument can be applied to the case of underdominance populations. Thus the proof of Theorem 2 is completed.
Theorem 3 Assume that a polygenic trait is determined by the genotypes whose relative fitnesses are (1, 1 − hs, 1 − s), where the selective advantage coefficient s is fixed and the degree of the dominance h is uniformly distributed over [ −1, 1]. Then inbreeding mating reduces the mean fitness of the population for all initial frequency x satisfying Proof By the assumption, for each 0 ≤ h ≤ 1, there is an over-dominant genotype (1, 1 + hs, 1 − s). It suffices to show Note that the left hand side of the last inequality is since it is a decreasing function of s. Letȳ be the smaller root of 6 − 3f y 2 − 8 − 3f y + 2. Then we haveȳ = 2 6−3f and 6 − 3f y 2 − 8 − 3fh 2 y + 2 is a decreasing function for y ≤ȳ.
In fact, we have that