Introduction

Reducing HIV incidence among adolescents represents an urgent global priority, given the high individual and societal costs of the epidemic on youth populations (Bekker et al. 2015). In the United States, adolescents make up a disproportionate share of new HIV infections, accounting for approximately 17 % of the population and 26 % of new infections in 2010 (Centers for Disease Control and Prevention 2014). Among adolescents aged 13–14 and those aged 20–24, new HIV infections steadily increased from 2009 to 2013. Young people aged 20–24 suffer the second highest rate of new infections (35.3 per 100,000) of any age group; people aged 25–29 have the highest rate (Centers for Disease Control and Prevention 2015). Young men who have sex with men (YMSM), especially young men of color, are at highest risk among youth populations. Although the excess burden of the epidemic among youth rests squarely on YMSM and incidence has declined among other youth populations, some youth remain at risk as a consequence of heterosexual sexual activity and injectable drug use. Among these youth, HIV incidence remains disproportionately high among young women of color and those who are impoverished (Prado et al. 2013). The present study examines the outcomes of a structural-change community mobilization intervention to reduce at-risk adolescents’ exposure to HIV.

Creating structural changes to reduce HIV infection has gained popularity as a necessary and complementary approach to conventional behavioral and biomedical interventions. The approach is widely viewed as essential for adolescents, who face unique and significant structural barriers to prevention and HIV testing, such as policies restricting them from independently consenting to prevention, testing, and treatment services (Delany-Moretlwe et al. 2015; Prado et al. 2013.) At the broadest level, structural change interventions aim to alter the features of social, economic, and policy environments that are implicated in driving HIV risk and to create settings that better facilitate and promote HIV preventive actions (Blankenship et al. 2000, 2006; Gupta et al. 2008). Because they alter the context in which risk and protection develops, structural changes are theorized to be more enduring and conferring of widespread benefits as compared with interventions focused solely on individuals (Frieden 2010; Sumartojo 2000). Moreover, structural changes may reinforce and enhance the effects of individual-level interventions by creating environments that better enable individuals to maximize the benefits of these approaches. As noted by Parkhurst (2014), because risk behaviors rarely have a single cause and occur in complex socio-cultural contexts, comprehensive efforts at HIV prevention require modifying the broad structures that shape risk and vulnerability in locally appropriate ways (Parkhurst 2014, p. 2), in addition to direct efforts to help individuals at risk to modify their behaviors.

Although structural interventions are evaluated less frequently than interventions focused at the individual level, available evidence supports the value of intervening on “upstream” determinants of risk. In a review of the research on adolescents at risk, Prado et al. (2013) observed that structural interventions such as providing condom access and comprehensive sexual education reduce diverse HIV-related risk behaviors among adolescents in domestic and international studies. Hardee et al., in a comprehensive review of rigorously evaluated structural change interventions to prevent HIV infection among women, identified multiple interventions targeting structural dynamics affecting gender norms supporting violence against women, gender-based educational and economic inequities, and gendered stigma for which the available evidence of effectiveness was positive (Hardee et al. 2014). For instance, Hardee et al. identified 15 rigorous evaluation studies in which researchers observed advancing educational attainment among girls through strategies such as abolishing school fees lowered HIV risk.

One approach to effecting structural change is through mobilizing community partnerships (Roussos and Fawcett 2000; Watson-Thompson et al. 2008). Community mobilization comprises a set of approaches for establishing community-based collectives to promote community betterment. These approaches reflect a commitment to engaging communities in developing and implementing structural solutions to local and population-specific concerns (Roussos and Fawcett 2000; Zakocs and Edwards 2006). By mobilizing collectives to engage in critically analyzing the root causes of local problems, identifying an array of potential solutions, developing multi-sector partnerships, and implementing strategies for creating local change, community mobilizations are demonstrated to strengthen communities and make them more health-promoting environments (Allen et al. 2013; Cheadle et al. 2008; Darnell et al. 2013; Emshoff et al. 2007; Javdani and Allen 2011; Salazar et al. 2007).

Evaluating the outcomes of structural change through community mobilization has proved equally challenging to implementing these complex interventions (Bonnell et al. 2006; Francisco et al. 1993). These approaches engage multiple and complex actions on the part of numerous actors and presume indirect and complicated causal sequences leading to public health outcomes (Bonnell et al. 2006). Structural changes may not be easily traced to the mobilization effort itself and outcomes may not occur in close temporal proximity to mobilization and its consequent activities (Berkowitz 2001). The farther upstream interventions become and the greater their potential to have multiple effects, the longer it may take for change to be detected at the level of individual and interpersonal behavior (Parkhurst 2014). There is no strong empirical basis for estimating how long it might take for structural changes to become evident in behaviors and it is reasonable to assume that it may take many years before a consistent pattern of lower risk activity at the individual level can be discerned.

Attributing change back to a mobilization effort may become harder to do as the work becomes more successful in part because the immediate targets of change are the community actors, institutions, and settings that have indirect effects on or direct interaction with populations at risk. The more these kinds of settings change internally and in relationship to one another and the greater number of actors there are contributing in new ways to an improved community environment, the harder it may be to identify the contribution of the initial mobilization effort to outcomes (Earl et al. 2001). Moreover, the chain linking individual behaviors to structural causes of risk is so long and complex that even if changes in individuals do manifest, causal attributions are complicated to make given competing alternative explanations for change and the lengthy passage of time this kind of intervention requires to succeed (Sumartojo 2000; Yin and Kaftarian 1997).

Another evaluation challenge lies is the fact that community mobilization for structural change by its very nature leads to locally specific pursuits affecting multiple and emergent sets of outcomes. Identifying unifying cross-site structural determinants for monitoring may be difficult even when mobilization efforts are part of the same multi-site initiative. Finally, evaluating such efforts across multiple communities may often be infeasible given the high costs associated with conducting this kind of an evaluation and the power needed to detect behavioral and biomedical effects (Cheadle et al. 2013; Prado et al. 2013), though using multi-site designs that incorporate a means to establish the counterfactual condition would strengthen our ability to draw inferences confidently about the merits of these kinds of interventions (Hollister and Hill 1995).

One solution to these challenges has been to use a theory-driven evaluation approach (i.e., Chen 1990; Funnell and Rogers 2011; Weiss 1995) in which action strategies are documented and compared against implementation plans and the theoretical linkages along the causal chain between structural change and behavioral outcomes examined. For instance, in Clark and colleagues’ evaluation of seven Allies Against Asthma coalitions, coalitions achieved a combined 89 structural changes over roughly 5 years, each of which was logically linked to the risk context for children. Theoretically logical outcomes were observed in reduced feelings of helplessness among parents with asthmatic children and reduced asthma symptoms among children living in coalition communities (Clark et al. 2010). Although other studies using a theory-driven evaluation approach find similar support for the promise of community mobilization efforts to create structural change, within the domestic United States public health literature, the bulk of the evaluations identifying successful structural change coalitions examine health issues that are not heavily stigmatized, such as childhood asthma, cancers, diabetes, and heart disease. Whether community mobilization approaches to pursue structural changes will achieve similar success within the context of highly stigmatized diseases such as HIV remains a key area for research, especially given evidence that HIV-related structural change efforts at the local level may be politicized, resisted, and undermined (Campbell 2010; Miller et al. 2012; Reed et al. 2013, 2014).

The Current Study

The objective of the current study is to examine the intermediate community-level outcomes of a coalition-based community mobilization effort to create structural changes to prevent new HIV infections among at-risk youth who reside in HIV-prevalent communities. The Connect-to-Protect Partnerships for Youth Prevention (C2P) rests on the belief that creating structural changes in local communities will ultimately result in reduced exposure to HIV among at-risk youth. With time, youth are expected to report lower frequency of risk behaviors and, in turn, HIV incidence among youth is expected to decline. These behavioral and biomedical changes are expected to occur because the aggregate effects of structural changes will combine to reduce internalized HIV-related stigma, which is a known barrier to HIV prevention, testing, and care (Barker et al. 2012; Mahajan et al. 2008; Prado et al. 2013; Radcliffe et al. 2010), and increase the quantity and quality of HIV-related community resources for youth, creating a youth-supportive environment (see Fig. 1). All coalitions’ locally formed objectives link to these overarching macro-structural factors known to shape HIV risk among adolescents.

Fig. 1
figure 1

C2P logic model

Full size image

In the current study, we examine perceived changes in these intermediate outcomes across the initial 4 years of C2P’s implementation. Specifically, we explore changes in youth’s perceptions of their community as a supportive, well-resourced context for HIV prevention and in youth’s degree of internalized HIV stigma. We examine whether these perceptions improve over time (e.g., at each study wave) and whether changes in perceptions are moderated by whether the coalition’s work centered on YMSM or other at risk-youth. We hypothesize that youth will perceive their community as more HIV-supportive and well-resourced at the end of the initial 4 years of C2P than they did at the start of C2P’s implementation and that internalized HIV stigma will decline. We expect these perceptions are moderated by coalition focus on YMSM versus other high-risk youth, as evidence suggests YMSM may have less favorable community experiences than other youth and face greater levels of stigma (Arnold et al. 2014; Grollman 2012). Mobilization efforts on their behalf may also be met with greater resistance than coalitions focused on other populations of at-risk youth given entrenched homophobia, which would make it harder to enact structural changes that are fundamentally socially transformative and do so quickly (Miller et al. 2012). We also compared perceived change among youth who lived in the high-prevalence community areas targeted by each coalition and youth living elsewhere in each city. Although many of the structural changes implemented by coalitions should benefit youth throughout each city, youth living in areas targeted for intervention were the primary intended beneficiaries when the intervention was initially conceived. Failing to benefit these youth or increasing the gap between these youth and youth living elsewhere in the city might reasonably be considered a failure.

Method

Connect-to-Protect (C2P)

The C2P intervention is a 10-year community mobilization effort led by the Adolescent Medicine Trials Network for HIV/AIDS Interventions. Mobilization began in 2006 and is ongoing. The Adolescent Medicine Trials Network for HIV/AIDS Interventions comprises an interdisciplinary team of HIV research scientists and a set of adolescent medicine trials units located throughout the United States. Each of the adolescent medicine trials units is located in a community with high rates of HIV infection among adolescents. Adolescent medicine trials units are typically hospital-based clinical research units specializing in the treatment of adolescents living with HIV.

Adolescent medicine trials units participating in C2P established coalitions, each with the mission of selecting and implementing structural change objectives to improve the local risk environment for an identified target population of youth living in community areas with high background adolescent HIV prevalence (see Ziff et al. 2006 for a detailed description of C2P mobilization). The C2P coalitions selected their target populations and community areas through analysis of local epidemiologic and other data (see Chutuape et al. 2010 and Ziff et al. 2006 for details), ultimately choosing community health areas or sets of contiguous ZIP code zones in which HIV among particular sub-populations of youth was heavily concentrated (e.g., YMSM, injecting drug users, high-risk young women). About half of the coalitions focused on YMSM, one focused on injecting drug users, and the remainder focused on high-risk young women of color.

Once identified, adolescent medicine trials units recruited members to the coalition who worked with the target population, preferably though not exclusively in the community areas of chief concern. Adolescent medicine trials units primarily recruited adult members into their C2P coalitions. The national protocol focused on recruiting adult leadership from the sectors and institutions working in the targeted community areas. Few of the adolescent medicine trials units are steeped in a culture of or experienced at engaging youth in the clinical work and medical research which constitutes their staff’s greatest expertise. Youth involvement varied substantially as a consequence (Reed and Miller 2014). Though most tried to engage youth, all but a few coalitions struggled to engage youth in meaningful roles over their initial 4 years (Reed and Miller 2014).

All coalitions observed a common set of operating procedures to guide planning, but otherwise pursued changes that made sense locally. The C2P coalitions followed a structured process of root cause analysis and strategic planning to determine local structural change objectives, develop a local logic model for change, and create an action strategy for achieving each objective (Willard et al. 2012). The coalitions were instructed to target the local policies, institutional practices and relationships, environmental conditions, and issues related to resource availability and access contributing to youth’s risk in the selected community area. No restrictions were placed or expectations set on the sectors coalitions elected to target, their number, or their variety. Thus, coalitions could focus all of their energy on a single sector or multiple sectors and every coalitions’ overall portfolio remained unique.

The nature of the intervention evolved rapidly because the prescribed strategic planning process led each coalition in unexpected directions and because of the steep learning curve C2P members faced in operationalizing structural change. Root cause analyses led coalitions to establish objectives that were communitywide, as coalitions quickly came to recognize that problems such as homelessness among YMSM and stigma associated with HIV were not limited to one community area, resulting instead from citywide policies and practices and widespread cultural narratives pertaining to women, sexual and racial/ethnic minorities, drug users, and people who are economically disadvantaged. Over time, addressing the risks facing the chosen target population became of higher priority than focusing solely on a single geographic area within the community, though these areas remained of priority concern because of their high background HIV prevalence among youth. For example, the coalition in San Juan rapidly focused on the fact that injection drug-using youth in their community area of focus were unable to obtain an HIV test without parental consent, preventing many of these youth from learning their HIV status. To increase access to testing for these youth required the coalition purse a legal change that could ease access to HIV testing for these youth and youth in all risk groups island-wide. By contrast, the coalition in Tampa wanted to reduce the likelihood that young girls engaged in survival sex with older males. The coalition nurtured a partnership between the police and public housing authority which led to a new program in the targeted community area in which police, housing authority personnel, and citizens established a neighborhood watch in which members were trained to identify and intervene in cases of sexual exploitation and abuse of minor females (Chutuape et al. 2010).

Structural thinking did not come quickly or easily to coalition members. Some early structural change objectives were of a conventional and non-structural nature; others proved unrealistic to achieve and were abandoned (Reed et al. 2012). The coalitions focused on YMSM abandoned many of their early objectives because of entrenched resistance in sectors such as schools (Miller et al. 2012). Coalitions were permitted to pursue the non-structural change objectives they developed early on in order to gain traction and achieve small wins that might bolster member commitment (Weick 1984). With time, coalition members improved their ability to frame structural change objectives.

Throughout C2P’s mobilization, structural change objectives were routinely reviewed, revised, and new objectives were established. Coalition composition was also routinely reviewed and new members recruited. Over the course of the first 4 years of implementation, the 13 coalitions achieved 245 of their objectives. Coalition organizers engaged roughly 4000 actors in their work over the initial 4-year period. Examples of the objectives coalitions completed during the initial years are listed in Table 1.

Table 1 Examples of completed structural change objectives
Full size table

Design and Data Collection

A controlled randomized trial was deemed inappropriate by Adolescent Medicine Trials Network for HIV/AIDS Interventions representatives in part because C2P was conceived as providing a community infrastructure to support multiple studies that are part of the network’s ongoing research agenda. Coalition outcomes were therefore evaluated using an observational study design which relies on examining intermediate outcomes of community change that have a logical substantive relationship to structural change and to individual behavior. In the current design, detailed implementation and outcome monitoring were combined in order to evaluate how coalitions’ action plans were carried out and whether desired changes in the communities’ contexts occurred. Data sources included tracking systems to document planning for and achievement of structural change objectives, coalition member surveys and interviews, annual key informant interviews, and a variety of reports to the staff at the project’s national coordinating center. In addition, of the 13 coalitions, eight participated in an outcome study that was conducted in years 1 through 4 of the project. Of the eight adolescent medicine trials units participating in the outcome study, five targeted YMSM (Baltimore, Los Angeles, New York City, San Francisco, Washington DC), two targeted young at-risk heterosexual women (Tampa, Chicago), and one targeted youth who use injectable drugs (San Juan). Each year for 4 subsequent years, a cross-sectional convenience sample of target youth living in these C2P communities was obtained. Recruitment and data collection for the outcome study were conducted in neighborhood venues (e.g., clubs, parks, neighborhood hangouts) that were identified each year through a mixed-method process of mapping and qualitative research (see Chutuape et al. 2010 for a detailed description of the venue identification procedures). Study staff visited identified venues at diverse times of day and on different days of the week. Using a standardized protocol, staff recruited, screened, and obtained youth’s informed consent to participate. A waiver of parental consent was obtained for minor participants at all sites. Minors provided their informed assent. Staff at each adolescent medicine trials unit continued their recruitment of youth until at a minimum 20 interviews had been conducted in at least two venues. Over the 4-year period, 2461 youth completed the interview, about 8 % of whom did so in more than a single year. In Table 2 we display the number of youth who completed interviews at each adolescent medicine trials unit site at each wave of data collection.

Table 2 Number of participants by C2P site and study wave (N = 2461)
Full size table

Participants

Study eligibility criteria were that youth report: being between 12 and 24 years of age, having engaged in consensual sexual activity in the prior 12 months, and identifying as a member of the target population that the particular adolescent medicine trials unit selected as its intervention focus. Thus, only YMSM were interviewed at YMSM-focused sites (n = 1350), only at-risk heterosexual females were interviewed at female-focused sites (n = 477), and only youth using injectable drugs were interviewed at the site focused on injectable drug users (n = 777). As we display in Table 3, this ethnically and racially diverse sample of youth averaged 20.2 years of age (SD = 2.6). The majority was male (66.2 %), in school (60.6 %), and lived with a biological or adoptive parent (58 %). Although all interviews were conducted in the community area that the adolescent medicine trials unit originally selected as its geographic focus for structural change, the majority of youth interviewed lived outside of that targeted community area (62.1 %).

Table 3 Demographic characteristics of study respondents (N = 2461)
Full size table

Measures

Youth who consented to participate in the outcome study completed an anonymous audio computer-assisted interview. Youth completed the interview in English or Spanish. The interview assessed respondents’ demographic characteristics, perceptions of their community, mental health, and risk behaviors. Only those measures pertaining to the current analyses are described.

  • Satisfaction with community support Four items, based on measures developed by Schriner and Fawcett (1988) and Suarez-Balcazar et al. (1998), assessed whether youth were satisfied with the support their community provided for youth. Items included “How satisfied are you that people are looking for new ways to support youth like you in your community?” Youth provided their response on a 5-point Likert-type scale that ranged from 0 (Very unsatisfied) to 4 (Very satisfied). We created a composite score by taking the average of these items (Cronbach’s α = 0.81). The scale mean was 2.71 (SD = .85).

  • Internalized HIV stigma HIV stigma was measured with 13 items adapted from Smith (1998). Representative items include “If you had HIV, people would think badly of you” and “If you had HIV, people would be disgusted by you.” Youth responded on a 5-point Likert-type scale that ranged from 1 (Strongly agree) to 5 (Strongly disagree). We created a composite score by taking the average of these items (Cronbach’s α = 0.94), after reverse-coding items so that high scores represent high levels of perceived HIV stigma. The scale has a mean of 3.51 (SD = .96).

  • Resources needs A final single-item measure assessed if respondents’ needs were met by the community. Respondents indicated their responses on a 5-point Likert scale that ranges from 1 (Strongly agree) to 5 (Strongly disagree). The mean on this item was 2.68 (SD = 1.17).

  • Time Because of the observational nature of the study design, time is used to measure the effect of the intervention in all analyses. As mobilization effects should accumulate over time, time is measured by treating the initial wave of data collection (1) as the reference category against which each subsequent wave of data (2–4) are compared.

  • Coalition focus We coded whether or not the coalition focused on YMSM or other at-risk youth (YMSM = 0, other = 1).

  • Residence Youth were asked whether they lived in the community area where the interview was being conducted (0 = no; 1 = yes).

  • Age Youth reported their age in years.

  • School status Youth reported whether they were currently enrolled in school (0 = no; 1 = yes).

  • Importance of community support Youth reported how important it was to them to have community support by answering four questions adapted from Schriner and Fawcett (1988) and Suarez-Balcazar et al. (1988). Items included “How important is it to you that people are looking for new ways to support youth like you in your community?” Youth provided their response on a 5-point Likert-type scale that ranged from 0 (Not at all important) to 4 (Very important). We created a composite score by taking the average of these items (Cronbach’s α = 0.80). The scale mean was 3.34 (SD = .70).

Analytic Approach

Given that respondents were nested within C2P communities and we had too few coalitions to use multilevel modeling techniques, generalized estimating equations were chosen as the statistical approach to account for the dependencies inherent in these data. Generalized estimating equations are a population-averaged model that provides accurate estimates of correlated data by treating the dependency between participants as a nuisance parameter and providing appropriate estimates by adjusting for this dependency (Burton et al. 1998). Generalized estimating equations were chosen over alternative methods such as mixed models because of their robustness in the presence of model specification error (Hubbard et al. 2010). One model was examined for each of the three outcomes (i.e., satisfaction with community support, internalized HIV stigma, and having resource needs met). The primary predictors of interest were the wave of the study, neighborhood of residence, and the interaction between wave of the study and the type of coalition (i.e., YMSM-focused or other-focused). We control for respondents’ age, school status, and the importance respondents place on having community support in all analyses. Wave 1 is treated as the reference category in these models. Interaction terms between wave of the study and coalition type were created and included in the models. All models used an unstructured correlation matrix and robust estimates for standard errors. Because the main effect estimates for wave reflect how youth in the other at-risk communities have changed, we calculated simple slopes for the communities with YMSM-focused coalitions to determine whether change over time in these communities was significant. For categorical variables, the statistical significance of model effects can differ from the parameter estimates at specific values of the categorical variables when interaction terms are included in the model. We therefore treated parameter effects as of highest importance in interpreting findings. We performed all analyses first with and then without the data from the San Juan coalition to be certain that its larger sample size and unique target population were not driving observed differences between youth from the YMSM-focused and other at-risk-focused communities. Because the results are substantively unaffected by the inclusion of the data from San Juan, we report here on the analyses using data from all coalition cities.

Results

Correlations between all model variables can be found in Table 4. We provide means and standard deviations for all outcomes in each city at each time point in Table 5. Model results for each outcome are displayed in Table 6.

Table 4 Correlations among study variables (N = 2461)
Full size table
Table 5 Mean and standard deviation of outcome variables by site and wave
Full size table
Table 6 Estimates and confidence intervals for models of the predictors of satisfaction with community support, resource needs, and HIV stigma
Full size table

Satisfaction with Community Support

Satisfaction with community support was significantly higher at wave 3 (b = 0.201, p < 0.001) and wave 4 (b = 0.755, p < 0.001) than at wave 1. Satisfaction was also higher among youth living in communities where the coalition focused on other at-risk youth than in cities where the coalition focused on YMSM and among youth in the community areas targeted for intervention than among youth living in other parts of the city. The interaction between YMSM-focused coalition and wave indicated that from wave 1 to wave 3 (b = −0.164, p = 0.006) and wave 1 to wave 4 (b = −0.680, p < 0.001) youth from cities with a YMSM-focused coalition reported less change in their satisfaction levels than did youth from cities with an other-at-risk youth-focused coalition (see Fig. 2a). Analysis of the simple slopes suggested that after accounting for the interaction, youth from communities with YMSM-focused coalitions reported significantly higher satisfaction with community support at wave 2 compared to wave 1 (b = 0.126, p = 0.015) but no significant change in satisfaction with community support between wave 1 to wave 3 (b = 0.037, p = ns) or wave 1 to wave 4 (b = 0.075, p = ns). After accounting for other parameters, the degree of change over time at the YMSM-focused coalitions was not significant (b = −0.095, p = ns).

Fig. 2
figure 2

Results of interactions between time and coalition type controlling for other model parameters. a Satisfaction with community resources, b resource needs met, c internalized HIV stigma

Full size image

Resource Needs

Youth were more likely to perceive that their resource needs were met by their community at wave 2 (b = 0.437, p < 0.001) and wave 3 (b = 0.156, p < 0.001) than at wave 1. However, in the final wave, perceptions of having needs met by their community were significantly lower (b = −0.715, p < 0.001) than they were at wave 1. Youth from the communities with other-at-risk-youth-focused coalitions reported their needs were better met than did youth from communities with YMSM-focused coalitions. Youth in target community areas reported their needs were better met than youth residing elsewhere in their cities. Furthermore, the interaction between YMSM-focused coalition and study wave indicated that youth from YMSM-focused coalition communities reported more modest differences in scores between wave 1 and wave 2 (b = −0.484, p < 0.001) and wave 1 and wave 3 (b = −0.172, p = 0.002) than did youth from communities where coalitions focused on other at-risk youth (see Fig. 2b). Youth from the YMSM-focused coalition communities perceived much less change in how well their needs were met than youth in the other at-risk-youth-focused coalition communities; simple slopes suggested there was no significant change in perceiving their resource needs were met between wave 1 and wave 2 (b = −0.047, p = ns), 3 (b = −0.016, p = ns), or 4 (b = −0.088, p = ns) among youth in the YMSM-focused communities. After accounting for other parameters, change over time in the YMSM-focused coalition communities was not significant (b = 0.127, p = ns).

Internalized HIV Stigma

Internalized HIV stigma was higher in wave 3 (b = 0.177, p = 0.022) and wave 4 (b = 0.528, p < 0.001) than in wave 1. Youth from the communities with YMSM-focused coalitions reported significantly lower levels of internalized HIV stigma overall when compared to respondents from communities with coalitions focused on other at-risk youth (b = −0.360, p = 0.001). Youth living in targeted community areas perceived higher levels of internalized HIV stigma than did youth living outside these areas. The interaction indicated that among youth in communities with YMSM-focused coalitions, levels of internalized HIV stigma changed more modestly from wave 1 to wave 3 (b = −0.214, p = 0.037) and wave 1 to wave 4 (b = −0.532, p < 0.001) than it did among youth from communities with coalitions focused on other at-risk youth (see Fig. 2c). Finally, simple slopes for youth in communities with YMSM-focused coalitions suggested there was no significant change from wave 1 to wave 2 (b = −0.004, p = ns), 3 (b = −0.037, p = ns), or 4 (b = 0.049, p = ns) in levels of internalized HIV stigma.

Discussion

Meeting the challenges posed by the HIV epidemic requires strong collaboration across diverse community sectors banded together to create structural change. Despite longstanding calls for structural change approaches to HIV prevention, few multi-year demonstrations of collaborative structural change approaches exist because their study is expensive, lengthy, and difficult (Gupta et al. 2008). We sought to add to the small but growing body of longitudinal evidence on structural change interventions by assessing whether cross-sector community coalitions established in eight United States cities showed early signs of progress toward improving the risk-environment for adolescents at risk of exposure to HIV. Although an 8-city study is limited in what it can demonstrate about the value of mobilizing communities to pursue structural change, we believe examining adolescents’ perceptions of how their communities change over time early in the life of this collaborative effort adds useful information to the growing base of evidence on the effects of these initiatives within the context of HIV. Our results aid in understanding whether structural changes can shift adolescents’ perceptions of their communities as AIDS-competent settings (Campbell et al. 2007; Reed et al. 2013).

Multiple pathways exist between root structural causes of risk and protection for adolescents, some via formal and informal laws and policies, some via normative social and cultural messages, and others via availability of and access to resources (Cohen et al. 2000; Parkhurst 2014). Our findings illustrate that it is possible, despite these lengthy and complex pathways, to see significant and consistent changes in how young people view their communities manifest within a brief time span. Our findings suggest that structural changes manifested in adolescents’ perceptions of their communities within 2–4 years following the inception of community mobilization efforts to stimulate structural changes. The results of our assessment indicate that youth reported improvements in the extent to which they thought their resource needs were met by their communities and in their reports of satisfaction with their communities as supportive places. Perceived improvements in these outcomes were evident at nearly all time intervals over the 4-year period. Moreover, youth living in the community areas targeted as high priorities reported greater satisfaction and met needs than youth from surrounding community areas. Youth from outside the targeted areas may have benefited from community mobilization efforts because many of the structural changes achieved over the 4 years we studied occurred county or citywide. Policy and legal changes at the levels of school districts and county institutions, for instance, should have this kind of broad scale reach (Chutuape et al. 2010). Localized changes meant to increase availability of and the access to community support and resources in target community areas may also serve to attract youth from neighboring communities as these environments become better able to meet youth’s needs. Continuing to improve resources in targeted communities through structural change may be especially important for YMSM who must often migrate outside of their neighborhoods of residence to find supportive environments (Stall et al. 2008) and for whom these data suggest our efforts to create structural change have not yet realized observed benefit.

We observed community mobilization had no impact on lowering internalized HIV stigma among youth from communities with coalitions focused on other at-risk youth. Indeed, for these youth, internalized HIV stigma appeared to become more pronounced over time. For youth from communities where YMSM were the focus, levels of HIV stigma remained stable. These findings may simply reflect the fact that although stigma was an overarching concern for every coalition, the bulk of the structural changes each achieved over the first 4 years of C2P’s implementation focused directly on changing policies and practices that would call attention to HIV and stimulate discussion about its impact on local youth in community sectors ranging from juvenile justice to foster care to housing. By engaging new sectors in HIV issues and setting off the alarm, the community mobilization effort may have produced the counter-intuitive effect of exacerbating internalized HIV-related stigma. The finding underscores the need for more study of the pace and time scale over which de-stigmatization of HIV occurs in communities. 4 years may be too brief a time period over which to detect desirable effects of structural change and community mobilization on perceived HIV stigma.

Our findings also indicate the drivers of reduced HIV stigma at the community level warrant further examination. Wide-scale de-stigmatization of HIV may follow rather than precede community mobilization efforts and improvements in community resources to address it. The early years of the epidemic among gay male communities in epicenter cities suggest this explanation may be plausible. Sense of community and a strong infrastructure reduced stigma and shame associated with HIV infection among gay and bisexual men over decades. Like their older counterparts, YMSM often form a community based on their shared social identity and one which overlaps substantially with the longstanding HIV-resource infrastructures developed from within these communities. By contrast, other youth at-risk may not share a community of identity linked to longstanding support structures for HIV, so may be more prone to view living with HIV as a source of shame and cause for social isolation (Harper et al. 2014). Each of the populations of youth on whom C2P focused inherited dramatically different HIV legacies. The very different levels of internalized HIV stigma between youth from communities targeting YMSM and those from communities targeting other at-risk youth and the different trajectories of change over time between these two groups may suggest that it is unreasonable to assume identical chains of structural causality across youth populations who have such different historical relationships to HIV and who encounter very different kinds of entrenched discrimination within their communities. Close examination of the patterns in our data over C2P’s next 4 years may shed needed light on the link between developing community infrastructure and levels of internalized HIV stigma among youth.

The power of structural change in promoting adolescent and young adult’s public health is best established in disease areas less strongly associated with social stigma. Given entrenched homophobia, it is perhaps unsurprising we found YMSM coalition status moderated the effect of community mobilization on all study outcomes, such that desired change in community perceptions were generally limited to coalitions addressing other at-risk youth. Adolescents in communities where C2P coalitions focused on YMSM did not perceive their communities to improve as more supportive places or as places that met their resource needs over the 4-year period. And, although they had lower levels of internalized HIV stigma when compared with those from communities where the coalitions focused on other at-risk youth, internalized HIV stigma did not change appreciably over time for these youth. This finding may be attributable to the fact that structural changes may be easier to achieve when serving groups who are not highly stigmatized; previous analyses of coalition process data suggested that during their early years of mobilization, the C2P coalitions that focused on YMSM struggled to implement structural change objectives because of strong community resistance, especially in educational and faith-based sectors (Miller et al. 2012). Due in part to this community resistance, YMSM-focused coalitions took longer to accomplish objectives in their communities than did coalitions focused on other at-risk youth during the early years of implementation (Miller et al. 2012). YMSM and other at-risk youth coalitions also differed in the quality of objectives that they achieved. For instance, 49 % of achieved objectives during this time period targeted schools and churches, sectors in which YMSM coalitions were more prone to abandon their efforts due to strong resistance against structural changes focused on YMSM concerns (Miller et al. 2012). The coalitions focused on other at-risk youth succeeded in impacting schools, faith-based organizations, legal and criminal justice systems, as well as parents during these early years. YMSM-focused coalitions’ achievements were largely confined to the social service sectors that were already engaged in issues concerning HIV and YMSM (Miller et al. 2012; Reed et al. 2013). For these reasons, achievements by the coalitions focused on other at-risk youth may have been more likely to have an impact on perceived resources and community support. YMSM coalitions’ achievements may have been more likely to impact on variables we did not assess.

Several methodological limitations of the C2P evaluation merit consideration when interpreting study findings. Real world evaluations of the kind reported here are often done on a shoestring due to a combination of political, monetary, time, and data constraints (Bamburger et al. 2012). C2P’s evaluation is no exception, as these types of constraints led to key tradeoffs in design. The evaluation lacked a counterfactual condition because it was deemed unacceptable to withhold critical infrastructure resources from any C2P community. A randomized design or strong quasi-experimental alternative were not possible as a consequence. Although the design took advantage of collecting extensive process and implementation data and the study will proceed over 10 years, these are insufficient to rule out all plausible alternative explanations for the study findings. Historical shifts at the national level to better address HIV across the continuum of care may better account for the changes in these communities than C2P’s change efforts. However, the findings are generally consistent with other studies of structural change which show evidence of effects on community perceptions (Allen et al. 2013; Javdani and Allen 2011). Moreover, our findings correspond with the prevailing view in the extant literature that it is especially difficult to make changes that will benefit sexual minority youth and are also consistent with the process data suggesting that the YMSM coalitions faced significant political obstacles during their start-up period (Miller et al. 2012; Reed et al. 2013), providing us some confidence in our findings, despite the limitations of a descriptive study design for drawing causal inferences. Additionally, our ongoing evaluation data including key informant interviews conducted annually in each city and data on youth access to key services affirm C2P’s impacts are multiple and salient to local youth in ways that might plausibly effect their community perceptions related to HIV.

A second limitation concerns the way in which youth were sampled. The sampling plan followed some of the procedures associated with well-studied time–space-sampling techniques (i.e., MacKellar et al. 1996; Muhib et al. 2001) which were developed to obtain community-based probability samples of YMSM in an efficient manner. However, the aspects of these sampling procedures that are designed to ensure that the characteristics of a probability sample of youth are obtained were not used. The resulting sample, though venue based, is a sample of convenience and its representativeness of the youth in the community unknown. The lack of a probability sample means that the youth samples are biased in unknown ways and that the samples of youth obtained at each time point may not be comparable from one to the next. We therefore cannot rule out the possibility that the results are exclusively a product of sampling error resulting from non-sampling and sampling bias.

The small sample size is also of concern. Resource limitations led to a sample that is small at the individual and site level, so the evaluation lacks statistical power. The small sample size at the community level is especially problematic given that statistical methods to analyze nested data such as generalized estimating equations assume that the number of nested clusters is sufficiently large. Fortunately, simulation studies suggest that generalized estimating equations may not be substantially biased even with small samples (Hubbard et al. 2010).

Finally, it is worth noting that the intermediate outcomes that we measured here were few. For instance, we only measured internalized HIV stigma and not experienced stigma and discrimination related to HIV, gender, sexual minority, and socio-economic status, all of which are documented ecological risk factors for HIV acquisition among youth (Prado et al. 2013). In addition, coalitions may have affected change in other intermediate outcomes that have a logical substantive relationship to structural change objectives and individual behavior in their local environment. Future research on structural change interventions of this kind would benefit from measuring a greater array of community-level and individual-level variables that are hypothesized to be intermediate determinants of individual risk.

Conclusion

Although the particular contribution of any one structural change to shifts in youth’s perceptions cannot be easily made, these data suggest community mobilization for structural change affects the ways in which some adolescents perceive their communities. Using a theory-driven approach to evaluation, we observed positive shifts in some at-risk youth’s perception of their community as a place that was supportive and met their needs. We simultaneously saw negative shifts in internalized HIV stigma among these youth, which may be a precursor to viewing HIV as a routine health risk facing adolescents. Youth from communities targeting YMSM perceived no change in their environments as more supportive and resource-rich settings over the 4 years we examined. Strengthening healthy communities for YMSM may be especially difficult to achieve because of entrenched discrimination and stigma. Structural change on behalf of these youth may fundamentally undermine the status quo and therefore be harder to accomplish over short timeframes. More research is needed to guide community mobilizations aimed at fostering health-promoting community environments for YMSM.