Knowing What We Don ’ t Know : A Meta-Analysis of Children Raised by Gay or Lesbian Parents

Objective. This meta-analysis integrates the available empirical evidence of the past 35 years and critically discusses the current state of knowledge on children raised by gay or lesbian parents. Design. Data from 81 studies on children living with gay or lesbian parents and opposite sex couples was included in the analyses. Results. There were negative associations between living with gay or lesbian parents and several outcomes. Most outcomes showed significant variation across studies and were treated as random effects. There appeared to be evidence consistent with publication bias. Significant moderators included child age, environmental stability, adequacy of heterosexual comparison sample, sampling technique, sample location, researcher allegiance, citation rate, and publication status. Conclusions. Future research on this topic is needed, consisting of adequately-powered probability-based samples, with detailed measurement and safeguards against researcher bias.

The study of families headed by gay or lesbian parents is relatively recent when compared to the study of other family formations.For decades, large foundations would not permit their funds to be used for studies considering homosexuality (West, 1967), and grant support to study children of gay and lesbian parents came decades later.Initial assumptions were that children raised by gay or lesbian parents could be at risk of socio-emotional deficits (Osman, 1972;Krueger, 1978;Weeks, Derdeyn, & Langman, 1975).Early researchers in this area called those assumptions into question by reporting no significant differences between children of gay or lesbian parents and children of heterosexual parents (Mandel & Hotvedt, 1980).Since that time, researchers have frequently reported finding no differences between children of gay or lesbian parents and heterosexual parents (Patterson, 2005;Tasker 2005).
More recently, researchers have drawn attention to the fact that parenthood is much harder to achieve for same-sex couples (Rosenfeld, 2010), which implies a selection effect.
Consistent with this selection effect, some authors report gay or lesbian parents as significantly better than their heterosexual counterparts (Bigner & Jacobsen, 1989).In fact, a growing number of researchers suggest that when compared to heterosexual parents, gay or lesbian parents may in fact provide a better childrearing environment (APA amicus curiae brief in Bottoms v. Bottoms, Nov 15 1993;Biblartz & Stacey, 2010;Bozett, 1989;Crowl, Ahn, & Baker, 2008;Patterson, 2005;Patterson, 2006;Strohschein, 2010;Tasker, 2010).These suggestions have made their way into non-academic media, as illustrated by a recent article in the Washington Post titled "Children of same-sex couples are happier and healthier than peers, research shows" (Bever, 2014).In sum, it is unclear whether compared to children of heterosexual parents, children raised by gay or lesbian parents are advantaged, or no different.A quantitative summary of the available research on parent sexual orientation and child development will provide an evidentiary base upon which these competing claims may be evaluated.
The research on gay and lesbian parenting spans from 1979 to 2015, and more recent research might show different associations due to: 1) changes in cultural views toward gay or lesbian parents, 2) a greater number of gay or lesbian parents, and 3) a greater willingness among gay or lesbian parents to participate in parenting research.Forty years ago homosexuality was widely considered pathological (Freund et al., 1974;Siegelman, 1974), and gay or lesbian parents understandably expressed concerns about being judged as poor parents simply because of their sexual orientation (Ostrow, 1978).That concern may be less prevalent now that gay or lesbian couples have been granted legal and social recognition equal to heterosexual couples.

IF CHILDREN OF GAY OR LESBIAN PARENTS ARE DIFFERENT, CAN
RESEARCHERS INTERPRET THOSE DIFFERENCES?Bronfenbrenner (1979)  theory and intervention, of greater interest are identifying mechanisms through which social address variables may influence child development (Parke, 2013).For instance, the presence of a gay or lesbian parent may be nothing more than a marker of previous family transitions (e.g., divorce), different ideas of normative gender roles, differential access to models of normative gender roles, flexible belief systems that allow children to be more accepting and tolerant, or an increased exposure to discrimination.Simply identifying a family as headed by a gay or lesbian parent provides little insight into how the parent's sexual orientation might have an effect on children.
One particular strength of meta-analysis is the ability to test for moderation across studies in the magnitude of effect.Ideally, this moderation can provide clues as to potential mechanisms.For instance, the effect of growing up in a family headed by a gay or lesbian parent -if there is any -may vary between gay father families versus lesbian mother families (Patterson, 1992).If such a difference were found between children of gay fathers and children of lesbian mothers, it could support explanations involving parent gender.
Were a difference found between children raised from birth (or adopted early) into families headed by gay or lesbian parents versus children originally raised in heterosexual families who later in life transitioned into living with a gay or lesbian parent, it could support explanations involving environmental stability (Tasker, 2005).Were a difference found between children raised by gay or lesbian parents in U.S. versus children raised by gay or lesbian parents in other countries, it could support explanations involving the cultural acceptance of gay parenting (Anderson & Fetner, 2008;van den Akker, van der Ploeg, & Scheepers, 2013).A second purpose of this meta-analysis is to test for moderating effects of these and other variables identified in prior narrative reviews of this literature that could explain any differences between children living with gay or lesbian parents and children living with heterosexual parents.

IF CHILDREN OF GAY OR LESBIAN PARENTS ARE DIFFERENT, SHOULD RESEARCHERS INTERPRET THOSE DIFFERENCES?
Studies of children raised by gay or lesbian parents have been criticized on methodological grounds (Cameron, Cameron, & Landess, 2001;Marks, 2012;Schumm, 2008;Sullins, 2015).A third purpose of this meta-analysis is to test for moderators that would address each of these concerns.One concern regarding findings in this area is that if researchers find differences between children of gay or lesbian parents and children of heterosexual parents, they may be comparing children from fundamentally nonequivalent groups: single, lowsocioeconomic status gay or lesbian parents and married, high-socioeconomic status heterosexual couples.Such biased comparisons conflate the presence of a gay or lesbian parent with differences in environmental instability, divorce, lower socioeconomic status, or number of caregivers (Tasker, 2005).
A second concern is that differences may be reporter-specific.This is particularly relevant in situations when parenting is being assessed, or the parenting of a group with whom the parent identifies (i.e., gay or lesbian parents) as it would motivate self-enhancement.In situations where self-enhancement motives are salient (Crocker, 2002;Sedikides, Gaertner, & Toguchi, 2003), or where group identity may feel threatened, individuals are motivated to positively rate ingroups or people with whom they identify (Roese & Olson, 2007).As acknowledged by Bos and colleagues ( 2007) "Society's less favorable attitudes toward lesbian-parent families mean that lesbian mothers are likely to feel more pressured than heterosexual parents to justify or defend the quality of their parenthood" (p.39).
A third concern is the regular use of non-probability and snowball sampling techniques, which, although understandable, could nevertheless reduce the degree to which the published findings generalize to the larger unmeasured population of children raised by gay or lesbian parents.
A final issue raised in some qualitative reviews of this literature concerns the allegiance of researchers in this area to the hypothesis of no differences between children raised by gay or lesbian parents and children raised by heterosexual parents (Marks, 2012;Schumm, 2008).
As early as Glass's (1976) summary of the effects of psychotherapy, meta-analysts have considered as a possible moderator the expectations and motivations of the scientist conducting the research.Even the most responsible researchers can inadvertently affect the results of their studies and thereby find results consistent with their hypothesis (Harris, 1991;Martin, 1977;Rosenthal, 2002).developmental outcomes (e.g., internalizing problems) of children living with at least one gay or lesbian parent to children living with at least one heterosexual parent.The vast majority of these studies were excluded for not reporting any parenting behavior or child outcomes.However, there was a smaller group that did report either parenting behaviors or child outcomes, but were still excluded because they lacked a heterosexual comparison group (Bailey, Bobrow, Wolfe, & Mikach, 1995;Bos, Goldberg, van Gelderen, & Gartrell, 2012;Bos, van Balen, van den Boom, & Sandfort, 2004;Brewaeys et al., 1995;Cameron, 2006;Cameron & Cameron, 1996;Erich, Leung, Kindle, & Carter, 2005;Gershon, Tschann, & Jemerin, 1999;Paul, 1986;Ryan & Cash, 2004;Tacher, 2009).

METHOD
Some manuscripts included in previous reviews were also excluded from the current metaanalysis for having no relevant data (Kweskin & Cook, 1982;Miller, Mucklow, Jacobsen, & Bigner, 1980;Rand, Graham, & Rawlings, 1982), for having no quantitative data at all (Lyons, 1983;Turner, Scadden, & Harris, 1990), or for collecting quantitative data but not reporting any in their results section (Bigner & Jacobsen, 1992).However, when data were provided, it was included in this analysis, even when it was of less than ideal quality.For example, perhaps the most fundamental assumption of the general linear model is that data come from independent sources.The independence assumption was violated in several of these studies that treated information from siblings in the same home as independent (evidenced by the reported degrees of freedom in these studies; Erich, Kanenberg, Case, Allen, & Bogdanos, 2009;Green, Mandel, Hotved, Gray, & Smith, 1986;Huggins, 1989;Javaid, 1993;Tasker & Golombok, 1995;Vanfraussen, Ponjaert-Kristoffersen, & Brewaeys, 2002;Vanfraussen, Ponjaert-Kristoffersen, & Brewaeys, 2003), yet in the interest of thoroughness (and consistency with the prior meta-analyses which chose to include them), they are also included here.Other examples include Regnerus (2012) which appeared to conflate parent sexual orientation with familial instability, and Gartrell and Bos (2010) who reported an effect of family structure on academic achievement (r = .81)which vastly exceeds any documented parenting effect.Of the over 1200 documents, 84 met the inclusion criteria.Three of those studies were ultimately excluded for having insufficient data to calculate an effect size (Leung, Erich, & Kanenberg, 2005;Sullins, 2015;Tasker & Golombok, 1998).Effect sizes for the New Family Structures Study come from a commentary on the original article by Regnerus (Amato, 2012).Amato reported an aggregate effect size across five outcomes (i.e., educational attainment, physical health, overall happiness, (low) depression, and current relationship quality), and that estimate was used in the current study.
The second step of the search procedure was a detailed inspection of the references cited by those 81 documents.This produced no additional studies.Finally, the four existing metaanalyses on this topic were also reviewed for additional studies.This resulted in one additional study by Ostrow (1978).However, Ostrow's dissertation consisted of qualitative interviews and was consequently excluded.This resulted in a final total of 81 documents from 57 independent samples (Table 1), designated with asterisks in the references.This is larger than Allen and Burrell's (1996;k = 18), Crowl, Ahn, and Baker's (2008;k = 19), and Fedewa, Black, and Ahn's (2014;k = 33).A total of 2,713 children raised by homosexual parents and 22,781 children raised by heterosexual parents were represented in these studies, as well as additional studies based on the U.S. National Health Interview Survey, the U.S. census, and the Canadian census.

COMPARISON FAMILIES
Many studies included data which allowed for comparisons of gay or lesbian parents and heterosexual parents in similar family structures.In cases where data were available on multiple heterosexual comparison groups, children from gay or lesbian single-parent families were compared to children from heterosexual single parent families, and children from divorced families now living with a gay or lesbian parent were compared to children from divorced families now living with a heterosexual parent.For instance, in a study like the New Family Structures Study (Regnerus, 2012) in which many children raised by a gay or lesbian parent(s) had experienced parental divorce, the comparison group for these analyses was children of heterosexual parents who had either experienced a divorce or been continuously single.Our intention was to create a group of baseline studies in which the homosexual-parent and heterosexual-parent groups were very closely matched, so it could be used as a reference group in the moderator analyses.

OUTCOME VARIABLES
Outcome variables were combined into nine categories based on face validity and how they were interpreted in the original manuscripts.The nine categories included six child outcomes (i.e., internalizing problems, externalizing problems, social competence, positive relationships, academic competence, gender development) and three parenting outcomes (i.e., gender socialization, control, warmth).Lists of all outcome variables represented in each category are available in Appendix A. Effect sizes were coded positive if living with gay or lesbian parents was associated with better child outcomes (or better parenting behavior).MODERATOR VARIABLES I examined 14 potential moderators of these nine outcomes, based on concerns expressed in prior reviews on this topic.Each of the studies was coded by the author, and independently coded by a research assistant.Percent agreement was above 90% for all study information; disagreements were resolved by discussion.Moderators are presented in Table 2. Gender of same-sex couples.Where data for gay fathers and lesbian mothers were reported separately, they were recorded separately.Some studies contained information on only gay fathers or lesbian mothers, this was recorded as well.
Data reporter/source.We recorded effect sizes for child outcomes and parenting behavior by reporter (i.e., parent report, child report, teacher report, other report) so reporters could be compared with each other.
Age of child.Mean sample age was coded for most studies, and for studies in which only the lowest and highest ages were reported (e.g., "children ranged from 9 to 18"), we used the average of those two points.
Stability in home environment.We coded whether children of gay or lesbian parents had been living with the homosexual parent(s) since infancy (i.e., 24 months; 1 = yes, 0 = no).We also coded whether the homosexual couple's relationship predated the child's entry into the home (1 = yes, 0 = no).
Biological relatedness.Studies in which the children were biologically related to one of their parents were coded as "1", and studies in which the children were adopted were coded as "0".
Comparable family structure.We coded whether the families in the heterosexual comparison group were of the same family structure (i.e., single parent) as the families in the gay or lesbian group (1 = yes, 0 = no).Researcher allegiance.Following the procedure of Glass, McGaw, and Smith (1981), we coded whether the authors appeared biased in favor of the "no differences" hypothesis (coded as 1) biased against the "no differences" hypothesis (coded as -1), or did not appear biased (coded as 0).The author independently rated each study on researcher allegiance and a group of research assistants (n = 8 per study) independently rated all of the published studies, as well as the first 10 pages, hypotheses, method, and last ten pages of each unpublished study.Only one study was rated as biased against the "no differences" hypothesis, so it was combined with the "not biased" group.This produced an observed range from 0 to 1.The intraclass correlation was r = .75suggesting that it was possible to reliably determine experimenter allegiance from the tone and substance of the research report.The modal rating was used in moderation analyses.A second measure was to code whether or not any authors of a particular document had presented data (i.e., via conference or publication) more than once on homosexual parents (1 = yes, 0 = no).
Impact of manuscript.The first assessment of manuscript impact was based on the journal impact factor (coded using 5-year averages when available), with unpublished manuscripts receiving a score of zero.Impact factors averaged 1.73 (range: 0 -5.44).The second assessment of manuscript impact was based on citation rate, or the number of times it had been cited per year since publication, according to Google scholar and Publish or Perish (Harzing, 2007).Citation rates ranged from 0 to 28.
Impact of dataset.An additional moderator is whether the dataset has been used in more than one publication (1 = yes, 0 = no).
Manuscript year.Studies were coded by year they were produced.Although not tested as moderators, to help assess the adequacy of these comparisons across family type we recorded the differences between gay or lesbian couples and heterosexual couples in income and educational attainment, as well as differences in pre-adoption characteristics in adoption-based samples.
Effect sizes were all transformed to Fisher Zr values for analyses, then back-transformed for interpretation.In cases where authors provided multiple effect size estimates from the same reporter on the same sample, or where multiple assessments were available over time for the same samples, we averaged the effect sizes into a single estimate (Matt & Cook, 1994;Rosenthal, 1994).Effect sizes more than 1.5 the interquartile range were windsorized for moderation analyses (Hastings, Mosteller, Tukey, & Winsor, 1947;Lipsey & Wilson, 1999), and we used a random effects approach using the software Comprehensive Meta-analysis (Borenstein, Hedges, Higgins, & Rothstein, 2007).Additional checks for publication bias were conducted in R (R core team, 2013).

DESCRIPTIVE DATA
We conducted several preliminary analyses to determine the degree to which there were baseline differences between households headed by gay or lesbian couples and those headed by heterosexual couples.This was important because socioeconomic differences could potentially account for any observed difference between children living with gay or lesbian couples and children living with heterosexual couples (Gates, 2013).Gay or lesbian couples in these studies made on average 10.9 thousand dollars per year more than heterosexual couples (based on 13 datasets that included the data needed for direct comparison).Gay or lesbian parents were on average 1.5 years more educated than heterosexual parents.
It was also possible that preadoption differences could potentially account for any observed differences between adopted children living with gay or lesbian couples and adopted children living with heterosexual couples.Many of the studies based on adopted samples controlled for age of adoption by matching heterosexual couples and gay or lesbian couples based on age of adoption (e.g., Farr, Forsell, & Patterson, 2010;Farr & Patterson, 2013;Goldberg & Smith, 2009;Goldberg, Kashy, & Smith, 2012;Tan & Baggerly, 2009).When we combined results from the three studies that allowed for a direct comparison of age at adoption (Averett, Nalavant, & Ryan, 2009;Golombok et al., 2014;Erich et al., 2005) children adopted by gay or lesbian couples were slightly younger than children adopted by heterosexual couples (r = -.10).Among the other studies, there was no evidence of gay or lesbian couples adopting more special needs children (Farr & Patterson, 2013), no difference in preadoption history between children adopted by gay or lesbian and heterosexual couples (Golombok et al., 2014), and no difference in age of adoption (Goldberg & Smith, 2009).

WITHIN-STUDY MODERATORS
The next set of analyses involves three within-study moderators: child sex, parent sex, and reporter of the data.There was no difference in the magnitude of association between parent sexual orientation and child/parenting outcomes between boys (ṝ = .00,range = -.36 -.40) and girls (ṝ = -.04,range = -.47 -.83), based on the 16 studies that provided data for girls and the 13 studies that provided data for boys, Z = 0.63, p = .26.Consequently, I combine results from boys and girls in the remaining analyses.
There was no difference in the magnitude of association between parent sexual orientation and child/parenting outcomes between lesbian mothers (ṝ = -.02,range = -.45 -.40) and gay fathers (ṝ = -.05,range = -.40 -.23), based on the 37 studies that provided information for families headed by a mother who was lesbian and the 13 studies that provided information for families headed by a father who was gay, Z = 0.30, p = .38.Consequently, results from children of gay fathers and children of lesbian mothers are combined in the remaining analyses.
The average effect size across outcomes was compiled for each reporter.I then compared those four mean rs via pairwise comparisons (e.g., parent report compared to child report, parent report compared to teacher report, etc.).There was no difference across reporters

MEAN EFFECT SIZE ANALYSIS
In cases where multiple manuscripts reported effect sizes using the same data, they were averaged into a single estimate.For example, child grade retention from the 2000 U.S.
census as reported in Rosenfeld ( 2011) and child grade retention from the 2000 U.S. census as reported in Allen, Pakaluk, and Price (2013) were combined into a single estimate.Effect size estimates and 95% confidence intervals for all studies per outcome are available in Appendix B.
Table 3 contains aggregate effect sizes for all nine outcomes.Positive effect sizes reflect a benefit from living with a gay or lesbian parent, and negative effect sizes reflect a benefit from living with a heterosexual parent.For example, the 25 studies that contained measures of internalizing problems produce an aggregate effect size of r = .012.The test for heterogeneity is also significant for internalizing problems, meaning that these estimates of internalizing problems across studies come from more than one population.This supports tests for moderation of this aggregate effect.Tests of heterogeneity are significant for all outcomes except gender socialization and parent control.Aggregate probabilities for significant effects correspond to Z = 9.40 for gender development, and Z = 7.29 for gender socialization.To assess publication bias, PET-PEESE (Stanley & Doucouliagos, 2013) was run on all nine outcomes.The PET-PEESE results offered a different picture of the data.The associations that were significant (gender development and gender socialization) were no longer significant, whereas three associations that were not significant became significant (internalizing problems, externalizing problems, and social competence).Note.k = number of independent samples; r = average effect size; CI = confidence interval; Q = test of heterogeneity; p = probability that heterogeneity across studies is due to chance; rʹ = average effect size based on PET-PEESE; CIʹ = confidence interval based on PET-PEESE.

BETWEEN STUDY MODERATORS
Between-study moderators are presented in Table 4.The intercept represents the effect of parent sexual orientation conditional on the moderator.For example, the intercept for the This estimate has a standard error of .03 and an associated probability of .23.The coefficient for the moderation effect is .12,suggesting that among children who lived with gay or lesbian parents since infancy, the average effect size was .06.This reflects a small, negative association between living with gay or lesbian parents and internalizing.Children of gay or lesbian parents showed better or worse outcomes depending on whether they lived with the homosexual parent since infancy (significant in 2 out of 6 tests), whether the parent's samesex relationship predated the child's entry into the home (significant in 2 out of 6 tests), whether the heterosexual comparison group was matched on family structure (significant in 3 out of 6 tests), whether the sample was probability-based (significant in 1 out of 6 tests), whether the same was based in the U.S. (significant in 3 out of 6 tests), whether the researcher was rated as high on allegiance to the "no differences" hypothesis (significant in 5 out of 6 tests), whether any of the authors had published more than once in this area (significant in 3 out of 6 tests), whether the manuscript was published (significant in 2 out of tests), whether the sample was used for multiple publications in this area (significant in 1 out of 6 tests), and whether the manuscript was cited comparatively often (significant in 1 out of 6 tests).Gay or lesbian parents showed comparatively higher or lower levels of parenting behaviors depending on child age (2 out of 3 tests), and whether the heterosexual comparison group was matched on family structure (1 out of 3 tests).Based on the coefficients from Table 4, under the most favorable circumstances the benefit of living with gay or lesbian parents is indexed by an average effect size of .048(range: -17 to .15).Under less favorable circumstances, the benefit of living with heterosexual parents is indexed by an average effect size of -.15 (range: -.07 to -.34).Results from continuous moderators (i.e., child age, citation rate) were not included in these averages.
Impact factor was significantly correlated with average effect size (r = .38,p = .0010).
Studies published in more widely-cited journals tend to report more positive effects of being raised by gay or lesbian parents.Neither the year a manuscript was produced, nor whether the child was biologically related to the parents significantly moderated any of the outcomes.
Compared to unpublished studies, published studies were not more likely to use probability samples (p = .35),larger samples (p = .31),to include families where the child lived with the parents from infancy (p = .19),to have the heterosexual group matched on family structure (p = .92),or to include families where the parents' relationship predates the child's entry into the home (p = .54).Published studies were more likely than unpublished studies to be rated as high on allegiance t(54) = 3.78, p < .

DISCUSSION
The first purpose of this investigation was to assess whether children of gay or lesbian parents are different or not different compared to children from heterosexual parents.The answer is, it depends.Without correcting for publication bias, there are differences in both gender development and gender socialization.After correcting for publication bias, there are differences in internalizing problems, externalizing problems, and social competence.In these studies, children of gay or lesbian parents were similar to children of heterosexual parents across most outcomes, but these aggregate estimates contained significant variability, suggesting that interpretation of this aggregate effect is less informative than identifying the conditions that modify the aggregate effect.That is, children living with a gay or lesbian parent appears to be systematically associated with positive outcomes for some families, and negative outcomes for other families.
The second purpose of this investigation was to identify potential explanations of differences between children of gay or lesbian parents and children of heterosexual parents.Moderation tests identified conditions under which the aggregate null effects became significant.For example, living with a gay or lesbian parent was associated with positive outcomes for younger children and negative outcomes for older children.Perhaps children are unaffected by their parents' sexual orientation until they reach adolescence and have the cognitive capacity to actively explore their own identity, or become increasingly aware of others' negative assessments of nontraditional families.
Both measures of stability (i.e., relationship of same-sex couple predates child's entry into the home, and child lived with the same-sex couple since infancy) together moderated three of the six outcomes.This is the first study to document such moderation and suggests that one reason children raised by gay or lesbian parents may show differences in internalizing problems, externalizing problems, and positive relationships in some studies is because they experience more instability than children of heterosexual parents.For instance, children raised by an openly gay or lesbian parent from infancy never deal with the process of thinking their parent is heterosexual and learning otherwise.According to Harris and Turner (1986), this is not a pleasant experience for most children; they write, "Not one subject in the present study reported that a spouse or child showed a positive reaction to the discovery that the subject was gay (p.112)" Although additional research is needed to definitively identify the reason for this moderation, there was modest support for the role of instability in accounting for some of the differences between children raised by gay or lesbian parents and children raised by heterosexual parents.
Whatever causes the differences observed in these samples between children raised by gay or lesbian parents and children raised by heterosexual parents, it is not parenting behavior.
Neither control nor warmth varied between gay or lesbian parents and heterosexual parents, even after accounting for publication bias.Although this finding conflicts with previous conclusions that "Sexual preference of fathers produces qualitative differences in selfreported parenting behavior (Bigner & Jacobsen, 1989, p. 185)" it was replicated in the current analyses across child reports of parenting, observer ratings of parenting, spouse reports of parenting, and self-reports of parenting.The only evidence for an effect of parent sexual orientation on parenting behavior was less warmth expressed by homosexual parents toward older children, and less warmth by homosexual parents when compared to heterosexual-parent families not matched on family structure.The former finding merits further study.The latter could be due to unobserved confounds, and is potentially more a reflection of the quality of literature on gay or lesbian parents than of the parenting behavior of gay or lesbian parents.
The one consistent exception to this overall lack of evidence for parenting as a potential explanatory mechanism for differences between children of gay or lesbian and heterosexual parents involves gender development.Gay or lesbian parents were more likely than heterosexual parents to show support for an open attitude about gender and sexuality, which may explain the lower level of normative gender development among children living with gay or lesbian parents.Fulcher et al. (2008) interpreted this difference to mean "…that sexual orientation can predict parental attitudes which in turn may predict flexibility in children's own attitudes" (p.339) whereas Gartrell et al. (2011) wrote "The offspring of lesbian and gay parents might be more open to homoerotic exploration and same-sex orientation" (p.1205).One alternative explanation is that differences in normative gender development are to be expected if one's parent is gay or lesbian, as homosexuality has a heritable component (Långström, Rahman, Carlström, & Lichtenstein, 2010), but this biological explanation would imply a stronger link between parent and child gender development in families in which children and parents are biologically related, which was not the case.Considering the strong empirical support for behavioral mimicry (Chartrand & Lakin, 2013), is it logical that children of gay or lesbian parents would be more likely to show less adherence to gendered behaviors and stereotypes.An important caveat is that associations were no longer statistically significant after accounting for publication bias.
One potential explanation for these differences between children with gay or lesbian parents and children with heterosexual parents was disparities in socioeconomic status, as same-sex couples (in the U.S.) are generally socioeconomically disadvantaged (Gates, 2013).However, the small socioeconomic differences among the families represented in these studies favored the gay or lesbian parents.Parenthetically, this shows that the gay or lesbian parents upon which this literature is based (mostly drawn from snowball samples) do not represent the general population of gay or lesbian parents.A second potential explanation for these observed differences in child outcomes by family type is pre-adoption differences between children adopted by gay or lesbian couples and those adopted by heterosexual couples.
However, there was no evidence for such differences among the adopted children represented in these studies.A third potential explanation, based on theories of gender development (Leaper & Friedman, 2007;McHale, Crouter, & Whiteman, 2003), has to do with the presence of a parent of the same or opposite gender as the child.There was, however, no support for this explanation.Children raised by gay fathers were no different than children raised by lesbian mothers, and the differences associated with parent sexual orientation did not vary between boys and girls.A fourth explanation for potential differences is that children of gay or lesbian parents likely experience more teasing or bullying as a result of discrimination (Patterson & Redding, 1996).There was no evidence that this was the case in most of these studies (see MacCallum & Golombok, 2004;Tasker & Golombok, 1995;Vanfraussen, Ponjaert-Kristoffersen, & Brewaeys, 2002).The lack of reported discrimination, as well as its lack of consistent effect among these families (compare van Gelderen, Gartrell, Bos, & Hermanns, 2012with van Gelderen, Bos, Gartrell, Hermanns, & Perrin, 2012), does not offer strong support for its role as a mechanism to explain any differences.Nevertheless, future research may show that cultural attitudes and social context account for some observed differences, as studies conducted in the U.S. showed differences from non U.S. studies on three of the six child outcomes.At the very least, this suggests that studies on this topic do not necessarily generalize across cultures.
A third purpose of this investigation was to address concerns previously expressed regarding literature in this research domain.One concern involves the potential influence of methodological differences on findings in this area.For example, there was no evidence that the magnitude or direction of effect varied across reporters (e.g., parent, child, observer, teacher).Although some of these parents were highly invested in advocating for recognition of gay or lesbian parents (see Harris & Turner, 1986;Miller, Jacobsen, & Bigner, 1982), this investment did not translate into discrepancies with other reporters.This finding contradicts the findings of Crowl et al., who noted how gay or lesbian parents report having better relationships with their children than heterosexual parents, but the children themselves report no difference in the parent-child relationship.It may be that the inclusion criteria of Crowl and colleagues resulted in a larger proportion of the highly invested parents referenced by previous researchers (Harris & Turner; Miller et al.).Alternatively, it may be that the discrepancy in findings can be explained by which effects were counted in the category "positive relationships."The current study includes all relationships, whereas Crowl et al. focused specifically on parent-child relationships.
Another concern involves the degree of equivalence between gay or lesbian and heterosexual comparison groups.There was some evidence (significant for two out of six child outcomes) that studies in which the samples are matched on structure (e.g., 1-parent vs. 2-parent) produce smaller differences in externalizing problems and positive relationships.This could be interpreted as evidence that poorly matched comparison groups inflate (or create) differences between children raised by gay or lesbian parents and those raised by heterosexual parents.There was also limited evidence (significant for one out of six outcomes) that when compared to data from probability samples, data from convenience samples produces smaller differences in internalizing problems between children living with gay or lesbian parents and children living with heterosexual parents.Samples of convenience (which are the vast majority in this research domain) may produce underestimates of the difference in child internalizing associated with parent sexual orientation.
The next concern addressed in this meta-analysis involves researcher expectancy effects (Rosenthal & Rubin, 1978).The allegiance of the authors to the no difference hypothesis was consistently associated with effect size.One plausible interpretation is that researchers who are more invested in a given topic may be more experienced in that topic and conduct their research with higher methodological rigor.That is, what we coded as allegiance could be https://thewinnower.com/papers/3532-knowing-what-we-don-t-know-a-meta-analysis-of-children-raised-by-gay-or-lesbian-parents 19/37 indirectly tapping study quality.However, there was no evidence of methodological superiority in studies coded high on allegiance, either in terms of sampling techniques, measurement strategies, analytic decisions, or accuracy of statistical interpretations.
Another plausible interpretation is that authors who repeatedly find empirical support for the no differences hypothesis will eventually show high allegiance to this hypothesis.
However, there was no evidence that authors had one expectancy in early research and changed their expectancy over time.It is almost as though researchers are studying two distinct, relatively homogenous groups of gay or lesbian parents: one group (studied by researchers high on allegiance) parents exceptionally well and has well-adjusted children, the other group (studied by researchers low on allegiance) shows deficits in both parenting as well as child adjustment.Future research based on adequately-powered probability-based samples, with appropriate safeguards against researcher bias, is needed to determine which of these two groups is more representative of the larger population of gay or lesbian parents.
The fourth concern involved publication bias.Almost forty years ago, immediately after the American Psychological Association adopted a resolution to support the American Psychiatric Association's decision to remove homosexuality from the list of mental disorders, one psychologist reflected that the APA's active gesture of support for people who are gay or lesbian could influence the perception of what is considered acceptable scholarship until eventually "We might begin to see research reflecting a homosexual bias-for example, that gays often function better than straights" (Morin, 1977, p. 631).Year of publication was not a significant moderator so narrowly speaking, the concern voiced by Morin that changes in institutional views towards homosexuality itself could have influenced scholarship over time was unfounded.However, there does appear to be bias.The overall estimates of effect size change markedly after accounting for publication bias.Studies on children of gay or lesbian parents which are published are more likely to report smaller differences between children of gay or lesbian parents and children of heterosexual parents (significant for two out of six outcomes).Studies published in high-impact journals are more likely to report smaller differences between children of gay or lesbian parents and children of heterosexual parents.
It is unusual to have publication bias operate in favor of small nonsignificant effects.These multiple indicators of bias do not necessarily suggest any deliberate attempt to manipulate the literature.They are equally likely to be a reflection of the institutional assumptions described by Morin.Such expectancy effects and researcher artifacts, although unintentional, can dramatically and consistently influence scientific findings (Rosenthal & Rosnow, 2009).
Unfortunately, this bias is further compounded in that the samples from which most of the published work is based show smaller differences between children of gay or lesbian parents and children of heterosexual parents (significant for one out of six tests).This is important because this research domain has been strongly influenced by several high-profile studies.
For example, the combined impact factors of published articles from the National Longitudinal Lesbian Family Study (less than 100 lesbian-headed families) is greater than the combined impact factors of all the studies that have resulted in a single manuscript (over 1,000 families headed by lesbian or gay parents).Finally, studies which offer support for the "no differences" hypothesis are more likely to be cited, which is the canonical measure of scientific interest.You may use this space for a freeform review.We request that you at some point address the following two questions: (1) Does the data support the authors' conclusions?(2) Do you have any competing interests to declare?
I am a researcher at the same institution as the author of this manuscript (although in a different department) and was asked to comment on this meta-analysis because of my experience with conducting meta-analyses in another field.I know very little about the subject matter at hand and will therefore restrict my comments and suggestions to the methodology and analyses.
However, it does seem that the need for such a meta-analysis is well articulated and the topic appears to have clear scientific and public policy implications.
Overall I find the meta-analyses to be well done.The literature search was thorough, coding decisions were well justified, the moderator analyses was well described and presented, and most of the analytic to see some justification of this approach over alternative methods such as Egger's test.

4.
The author uses Q-statistics to test for heterogeneity of effect sizes.This is largely similar to a significance test and has all the problems associated with null hypothesis testing.That is, it has low power for small number of studies.Further, when the number of studies is large then even trivial departures from homogeneity are indicated as significant.That is, it is difficult to know how to interpret Q statistics as an indicator of moderator effects.Instead I would recommend reporting an effect size measure of heterogeneity such as SDrho value or credibility intervals (rather than confidence intervals).This should provide readers more information about the size of any undetected moderators.

5.
The author appears to have not made corrections for unreliability in the dependent variables.I assume that these outcomes were not assessed with perfect measurement precision and I would encourage the author to take unreliability into account.Unreliability not only attenuates effect sizes downward but also increases the observed variability in effect sizes. 6.
I imagine that there is relatively good likelihood that the examined moderators are somewhat confounded with each other.In such circumstances it might be a good idea to try to disentangle the relative effects of each moderator by regressing the observed effect size onto more than one moderator variable using a weighted-least-squares regression approach in which individual studies are differentially weighted according to the inverse-variance of each effect size.

7.
This is not my area of research but I wonder if the year of publication (or year of data collection) might be another reasonable moderator variable to examine.My personal sense is that parenting by gay or lesbian parents has become more widely accepted over time such that the obstacles presented by society in the forms of the attitudes from schools, teachers, other children, other parents etc. might have be reduced over time.This might in turn, suggest, that any observed differences are reduced over time.

8.
Similarly, I wonder if it might be possible to examine if the country (or state) in which data was collected might also be examined as a moderator by perhaps importing data on the acceptance of gay and lesbian parents in that country or state.Right now the researcher has only coded US versus non-US but there are probably a lot of relevant variability within the US (across states) or between other countries that could be captured in some way.

9.
I think the understanding of readers of this manuscript would be further enhanced if some easy-to-understand explanation of the effect sizes were to be included.Something like a bivariate effect size illustration may be helpful.RT @Neuro_Skeptic: Can the "R-factor" from @verumanalytics (not to be confused with @R__INDEX) measure the reproducibility of science?http… 0 8 . 2 1 .2 0 1 7 RT @petergrabitz: e other day we had an idea on how to make reserach more reproducible.e idea is to use claims citing papers make http… 0 8 . 2 0 . 2 0 1 7 RT @petergrabitz: You can score #citations and help to make research more reproducible here: https://t.co/qTIA9KrIPmBackground: https://t… Knowing What We Don't Know: A Meta-Analysis of Children Raised by Gay or Lesbian Parents https://thewinnower.com/papers/3532-knowing-what-we-don-t-know-a-meta-analysis-of-children-raised-by-gay-or-lesbian-parents 3/37 ARE CHILDREN OF GAY OR LESBIAN PARENTS DIFFERENT OR NOT?
emphasized the importance of examining the context in which children develop in order to understand child development.Parent sexual orientation potentially creates such a context.However, although social address variables like parent sexual orientation are often correlated with measures of child development, they are not necessarily causal agents themselves.Such variables (e.g., parent age, parent race, parent sexual orientation) are commonly found in data used to study children and families, in part because they are comparatively easy to collect.Nevertheless, from the standpoint of both 8/22/2017 The Winnower | Knowing What We Don't Know: A Meta-Analysis of Children Raised by Gay or Lesbian Parents https://thewinnower.com/papers/3532-knowing-what-we-don-t-know-a-meta-analysis-of-children-raised-by-gay-or-lesbian-parents 4/37 LITERATURE SEARCH PROCEDURE/ INCLUSION CRITERIAThe first step of the search procedure involved entering related keywords into the PsycINFO, Web of Science, Academic Search Premier, Behavioral Sciences Collection, Sociological Abstracts, and Social Services Abstracts databases.The first broad search was based on the keyword "homosexual parents".The second search was based on the conjoined predictor keywords (i.e., lesbian, gay, homosexual, and same-sex parent), and a list of outcome keywords (parenting, parental characteristics, or parenting style).Together these searches (conducted on December 5, 2013) produced over 1200 unique documents.Additional searches conducted on December 4, 2014 and February 7, 2016 based on the same keywords produced seven additional studies published after the initial literature search.To be included in the meta-analysis, the authors must have either a) compared parenting behaviors (e.g., warmth) of gay or lesbian parents to heterosexual parents or b) compared What We Don't Know: A Meta-Analysis of Children Raised by Gay or Lesbian Parents https://thewinnower.com/papers/3532-knowing-what-we-don-t-know-a-meta-analysis-of-children-raised-by-gay-or-lesbian-parents 9/37
What We Don't Know: A Meta-Analysis of Children Raised by Gay or Lesbian Parents https://thewinnower.com/papers/3532-knowing-what-we-don-t-know-a-meta-analysis-of-children-raised-by-gay-or-lesbian-parents 17/37 What We Don't Know: A Meta-Analysis of Children Raised by Gay or Lesbian Parents https://thewinnower.com/papers/3532-knowing-what-we-don-t-know-a-meta-analysis-of-children-raised-by-gay-or-lesbian-parents 18/37

37 Figure 1 . 37 Figure 2 . 37 Figure 3 .Figure 7 .Figure 8 .
Figure 1.Pearson r-based Effect Sizes and 95% Confidence Intervals for Internalizing Problems decisions were in line with what I would consider to be meta-analytic best practices.I also find the inclusion of all coded data and the display of effect sizes in the Appendix to be particularly valuable.This greatly increases the transparency and replicability of this manuscript.I do have a number of suggestions that might further improve what is already a very strong manuscript.I outline these below.1.I would encourage the author to make additional literature searches to ensure that sources from what is sometimes referred to as the "grey" literature are not omitted.Searches of dissertations databases and general internet searches may yield additional sources.In my own field these searches often double the number of data sources that are obtained.2. I would like to see a more detailed description of what is meant by "within-study" and "between-study" moderators.It was not clear to me what the difference is.3.I would suggest a more detailed description of the PET-PEESE publication bias analysis.I am not familiar with it and would like What We Don't Know: A Meta-Analysis of Children Raised by Gay or Lesbian Parents https://thewinnower.com/papers/3532-knowing-what-we-don-t-know-a-meta-analysis-of-children-raised-by-gay-or-lesbian-parents 36/37 Thank you for the opportunity to review such a well-executed study on such an important and interesting topic.I have no conflict of interest to declare although I do work at the same institution as the What We Don't Know: A Meta-Analysis of Children Raised by Gay or Lesbian Parents https://thewinnower.com/papers/3532-knowing-what-we-don-t-know-a-meta-analysis-of-children-raised-by-gay-or-lesbian-parents 37/37 LICENSE This article and its reviews are distributed under the terms of the Creative Commons Attribution 4.0International License, which permits unrestricted use, distribution, and redistribution in any medium, provided that the original author and source are credited.©2017 T H E WI N N OWE R -I S S N 2 3 7 3 -1 4 6 X -All rights reser ved.
Sampling quality.Studies in which the sample could be considered a probability-based sample were coded as "1", and studies based on non-probability samples were coded as "0".U.S. based sample.Studies conducted in the United States were coded as "1", and studies conducted outside the United States (typically in Western Europe) were coded as "0".

Table 3
Effect sizes across outcome, and heterogeneity tests The meta-analysis by Fedwa et al. included more studies which supported the "no differences" hypothesis with regard to social competence, t(25) = 3.29, p < .001,andgenderdevelopment, t(19) = 2.82, p = .005.The meta-analysis byCrowl et al.was not significantly more or less likely to include studies which supported the "no differences" hypothesis.
001, andwritten by authors who had produced additional research on children of gay or lesbian parents t(54) = 11.42,p<.001.A final series of tests compared studies included in the previous meta-analyses with studies excluded from those meta-analysis.The meta-analysis by Allen et al. included more studies which supported the "no differences" hypothesis with regard to social competence, t(6) = 8/22/2017 The Winnower | Knowing What We Don't Know: A Meta-Analysis of Children Raised by Gay or Lesbian Parents https://thewinnower.com/papers/3532-knowing-what-we-don-t-know-a-meta-analysis-of-children-raised-by-gay-or-lesbian-parents 16/37 3.57, p < .001.
(Huggins, 1989; Don't Know: A Meta-Analysis of Children Raised by Gay or Lesbian Parents The literature on children of gay fathers and lesbian mothers represents the hard work and dedication of dozens of researchers, who faced the complicated task of studying a new family structure as it emerged.With the growing cultural support for gay and lesbian parents, there is an opportunity for researchers to take a more critical look at a possible disconnect between what this literature shows and what we may assume it shows.Regardless of social and political allegiances, ideally scientists strive to support parents, including gay and lesbian parents, by providing them accurate information about what the data show.It isimportant that scientists supplement this information with equally accurate conclusions.As one early researcher of differences between the children of gay or lesbian and heterosexual parents advised, "It is imperative that decisions that affect the lives of children be made on the basis of empirical data rather than assumptions or personal emotions"(Huggins, 1989;   p. 123).It appears that some children of gay and lesbian parents are in need of help, yet without acknowledging that need, interest in and funding for such help is unlikely.The time has come for scientists to move this field of study away from overstated conclusions based on poorly measured constructs administered to non-representative samples, away from endless narrative reviews and amicus briefs based on a literature that cannot support their weight.This field desperately needs high quality data: interdisciplinary teams collaborating to maintain the integrity of the sampling approach, construct measurement, and data analysis.Social scientists will best serve the children of gay and lesbian parents the same way we serve all other children, by working together, and not letting our assumptions outpace science.