Resilience-Informed Community Violence Prevention and Community Organizing Strategies for Implementation: Protocol for a Hybrid Type 1 Implementation-Effectiveness Trial

Background Community violence is a persistent and challenging public health problem. Community violence not only physically affects individuals, but also its effects reverberate to the well-being of families and entire communities. Being exposed to and experiencing violence are adverse community experiences that affect the well-being and health trajectories of both children and adults. In the United States, community violence has historically been addressed through a lens of law enforcement and policing; the impact of this approach on communities has been detrimental and often ignores the strengths and experiences of community members. As such, community-centered approaches to address violence are needed, yet the process to design, implement, and evaluate these approaches is complex. Alternatives to policing responses are increasingly being implemented. However, evidence and implementation guidance for community-level public health approaches remain limited. This study protocol seeks to address community violence through a resilience framework—Adverse Community Experiences and Resilience (ACE|R)—being implemented in a major US city and leveraging a strategy of community organizing to advance community violence prevention. Objective The objective of this research is to understand the impact of community-level violence prevention interventions. Furthermore, we aim to describe the strategies of implementation and identify barriers to and facilitators of the approach. Methods This study uses a hybrid type 1 effectiveness-implementation design. Part 1 of the study will assess the effectiveness of the ACE|R framework plus community organizing by measuring impacts on violence- and health-related outcomes. To do so, we plan to collect quantitative data on homicides, fatal and nonfatal shootings, hospital visits due to nonaccidental injuries, calls for service, and other violence-related data. In Part 2 of the study, to assess the implementation of ACE|R plus community organizing, we will collect process data on community engagement events, deliver community trainings on community leadership and organizing, and conduct focus groups with key partners about violence and violence prevention programs in Milwaukee. Results This project received funding on September 1, 2020. Prospective study data collection began in the fall of 2021 and will continue through the end of 2023. Data analysis is currently underway, and the first results are expected to be submitted for publication in 2024. Conclusions Community violence is a public health problem in need of community-centered solutions. Interventions that center community and leverage community organizing show promise in decreasing violence and increasing the well-being of community members. Methods to identify the impact of community-level interventions continue to evolve. Analysis of outcomes beyond violence-specific outcomes, including norms and community beliefs, may help better inform the short-term and proximal impacts of these community-driven approaches. Furthermore, hybrid implementation-effectiveness trials allow for the inevitable contextualization required to disseminate community interventions where communities drive the adaptations and decision-making. International Registered Report Identifier (IRRID) DERR1-10.2196/50444

1 R01 CE003191-01 3 ZCE1 KFJ (01) GRAHAM, P study.The ACE|R approach itself is well-grounded in theory, and the proposed project builds upon four years of work in Milwaukee, WI.The investigators, along with the Prevention Institute, have worked with the community group in the development of the Milwaukee Blueprint for Peace.The applicant is now proposing to retrospectively evaluate the effectiveness of the framework's implementation in Milwaukee, WI on CAN and YV, which will then be taken forward to integrate a community organizing and mobilizing approach.The community partner in Milwaukee, WI will continue to provide and expand technical training to the community members to determine whether the strategies developed can be implemented in the four priority communities.The novel Hybrid Type 1 effectiveness implementation design to be employed in the comparison communities to identify barriers and facilitators to implementation is also well-described.Additionally, incentives are planned for the qualitative component.
Weaknesses also were identified.Regarding significance, there are some limitations with regards to the potential generalizability of the findings to communities outside of Milwaukee, WI, which is not addressed in the application.With respect to the investigators, over 600 qualitative interviews with community members are proposed.The project team, however, appears to have limited qualitative research experience.The individual identified that is responsible for this task is a public health analyst for surveillance data who does not have any publications in qualitative research.Additionally, it is not clear whether any of the three team members from the Prevention Institute are physically located in Milwaukee, WI.A letter of support is provided from the City of Milwaukee Office of Violence Prevention with which the investigators are partnering, however, the particular staff from that office is not included in the proposal as a team member.This, however, is tempered by the proposed study having a strong focus on being community-driven, the emphasis on building capacity with community leadership, and the fact that the project truly is being placed in the hands of the community.
In terms of the approach, it is unclear whether the qualitative expertise and resources identified will be sufficient for the considerable amount of qualitative data that are likely to be generated.Clear plans are not delineated for how these data will be analyzed and managed or who will be responsible for this effort.This is a potential shortcoming, given that analysis of this large amount of data is going to require a major amount of effort.Seemingly, there is a lot of dependence on retrospective administrative data; however, there is insufficient discussion about the inherent limitations of this type of data.In terms of fidelity, it is not clear that it will be possible to adequately document how well some of the programs have been implemented over the last three years to ensure that the retrospective data are clean.There was a lack of control in the previously completed community-level program in terms of fidelity and dosage, and it is not clear how this would relate to the proposed project's outcomes.Additionally, there was a comment about the limitations on how quickly the available administrative data are ready to use.However, a statement is made that the investigators are working more closely with Milwaukee, WI to focus on real-time available data on a monthly basis.Beyond this, there is little discussion about the potential limitations of the administrative data.It is unclear whether the administrative data will lend itself to clear measurement with all of the outcome variables mentioned, particularly with regards to the demonstration of impact and change at the community-level.Nevertheless, the investigators have been receiving these data for a long period of time and they have an existing relationship with the city of Milwaukee, WI.The power analysis does not clearly center upon a primary endpoint, and a power analysis is not provided for the proposed idiographic clinical trials (ICTs) analysis.The translation plan is somewhat limited in terms of broader dissemination to policymakers and to the public.Also, regarding the environment, it is difficult to assess the Prevention Institute's facilities and resources that will be dedicated to the proposed study.
Protection of human subjects is acceptable.Inclusion is acceptable in terms of gender, minority groups, and age.Animal welfare, biohazards, and dual use research are not applicable.Budget modifications are recommended, as 60% of the funds are allocated to a sub-awardee.An acceptable resource 1 R01 CE003191-01 4 ZCE1 KFJ (01) GRAHAM, P sharing plan/data management plan (DMP) is provided.Overall, this is an excellent application that is very strong with only some minor weaknesses.

DESCRIPTION (provided by applicant):
Violence is one of several Adverse Childhood Experiences (ACEs) that can significantly impact children's future physical and behavioral health.Different types of violence against youth, such as child abuse and neglect and community youth violence, often share the same risk and protective factors.Community disadvantage and disorganization are risk factors associated with community trauma and violence, which can be addressed by community-level primary prevention strategies that emphasize community resilience through community organizing.This study will evaluate Prevention Institute's implementation of the Adverse Community Experiences and Resilience (ACE|R) community-level violence prevention framework, responding to Objective 1: Effectiveness research to determine which community-or societal-level strategies effectively prevent multiple forms of violence and other ACEs that impact children and youth.More specifically, it has three aims: (1) retroactively measure the effectiveness of the ACE|R framework implemented in Milwaukee on child abuse and neglect and youth violence using an idiographic clinical trial approach; (2) prospectively test the comparative effectiveness of the ACE|R framework/Milwaukee Blueprint for Peace with community organizing in four Priority Communities; and (3) use a Hybrid Type 1 effectiveness-implementation design to explore the barriers and facilitators to implementing community organizing within the ACE|R framework in a large metropolitan area (Milwaukee) to inform future implementation efforts by determining strategies to address implementation barriers arising from (a) the intervention characteristics, (b) outer and inner setting factors that shape implementation barriers and facilitators, (c) individual characteristics among stakeholders that influence implementation barriers and facilitators, and (d) strategies that facilitate implementation of community organizing within the ACE|R framework.By addressing several factors associated with community trauma and violence, this framework has the potential to prevent child abuse, child neglect, and youth violence.Findings will expand research evidence for the facilitation of community resilience efforts and contribute valuable insight regarding efficacious, practice-based violence prevention programming.

PUBLIC HEALTH RELEVANCE:
This study will evaluate the Adverse Community Experiences and Resilience (ACE|R) framework for community-level violence prevention through economic, social, and place-based strategies to heal trauma in communities.We will test (a) whether this process has reduced rates of child abuse and neglect and youth violence over the past 4 years of its implementation in the Milwaukee Blueprint for Peace and (b) the benefits of adding community organizing focused on engaging and mobilizing community members to support strategies to prevent violence.Our findings will provide insight about the best ways to help neighborhoods and cities fight child abuse/neglect and youth violence.

CRITIQUE 1
Significance: 1 Investigator(s): 2 Innovation: 1 Approach: 2 Environment: 1 Overall Impact: Strengths • If the aims are achieved, the proposed project will identify an effective community-level approach to reducing rates of child maltreatment and youth violence, as well as an effective strategy for sustaining community change.
• The project will also provide valuable information on potential barriers and facilitators to inform implementation of this approach across communities.Weaknesses 1 R01 CE003191-01 5ZCE1 KFJ (01) GRAHAM, P • The limitations on the generalizability of findings to communities outside Milwaukee, WI are not addressed.

Significance: Strengths
• The proposed project aims to identify an effective community-level, community-driven approach to preventing child maltreatment and violence, as well as other adverse childhood experiences, and to examine its comparative effectiveness across communities.
• The project further aims to evaluate the effectiveness of community organizing and mobilizing as an effective strategy for sustaining positive change. Weaknesses: • The limitations on the generalizability of findings to communities outside Milwaukee, WI are not addressed.

Investigator(s): Strengths
• The project team, including researchers at RTI International; the technical training experts at the Prevention Institute, whose role includes providing technical assistance to participating communities; and the City of Milwaukee Office of Violence Prevention, which will facilitate implementation and retrieval of community-level data, is well-suited to conduct the work proposed.
• The principal investigator has more than 20 years of experience conducting community-based research and evaluation on youth violence, including multiple federal-funded projects, and will be supported by a strong research team, including individuals with extensive experience managing and coordinating projects of this scale.
• Both the principal investigator and the project director also have expertise related to the role of culture and diversity in prevention research.
• The project leverages collaborative relationships built throughout the four-year implementation of the Milwaukee Blueprint for Peace, which is based on the Adverse Community Experiences and Resilience (ACE|R) framework, which is the subject of evaluation in the proposed study.Weaknesses • The specific personnel at the Prevention Institute and the City of Milwaukee Office of Violence Prevention who will be involved with the project were not identified.

Innovation: Strengths
• The project proposes to evaluate the effectiveness of community organizing and mobilization as a novel approach to building and sustaining community resilience by addressing community disadvantage and disorganization, which are risk factors associated with community trauma and violence.
• The project employs a novel hybrid-effectiveness implementation design with the goal of maximizing communities' ability to sustain intervention strategies shown to be effective by moving beyond describing standard implementation elements, such as dose and fidelity, to closing the implementation gap.Weaknesses • No weaknesses are noted.

Approach: Strengths
• The aims of the proposed project are directly aligned with Objective 1 of the NOFO.The project has the potential to identify an effective community-level strategy for preventing child maltreatment and youth violence, as well as other adverse childhood experiences.• The theoretical and evidentiary base for the Adverse Community Experiences and Resilience (ACE|R) framework, as well as the addition of community organizing to enhance sustainability by building resilience and addressing risk factors associated with community risk for violence, are well-developed.
• The proposal anticipates and provides a clear, detailed plan for activities required to complete the project successfully, from implementation through data analysis and dissemination.Also, there is a clear logic model.
• The project methodology and plan for data analysis is rigorous and empirically sound.Also, mixed methods are employed.Weaknesses • The plan for management and analysis of qualitative data was not well-developed.

Environment: Strengths
• RTI International is a well-established independent, nonprofit research organization with an expansive research infrastructure to support complex research projects in partnership with a wide range of entities, including government, business, and universities.
• The project leverages existing collaborations among the project partners, including a five-year partnership between the Prevention Institute and the City of Milwaukee to implement the Blueprint for Peace.
• There is ample evidence of a clear delineation of the specific roles and responsibilities of the researchers and other project partners, as well as their understanding of these roles and responsibilities.Weaknesses

Overall Impact Strengths
• The project directly addresses the NOFO as well as the lack of evidence on the effectiveness of community-level strategies for the prevention of ACEs (specifically child abuse and neglect and youth violence).The investigators propose to examine the effectiveness of community organizing and mobilization in changing community-level risk factors to reduce child maltreatment and youth violence within urban communities.
• There is a strong potential for the proposed project to have a positive impact on multiple forms of youth violence at a community level.
• The proposed project builds on a current initiative and current partnerships.

Weaknesses
• A minor weakness is the lack of a community partner (City of Milwaukee Office of Violence Prevention) on the investigative team.However, the investigators have worked in these communities for the past four years and the Prevention Institute has experience working with communities.

Significance: Strengths
• The project directly addresses the lack of evidence on the effectiveness of community-level strategies for the prevention of ACEs (specifically child abuse and neglect and youth violence).
• The proposed intervention activities, using the Adverse Community Experiences and Resilience (ACE|R) framework, have the potential to reduce the level of child abuse and neglect and youth violence at the community level.The ACE|R framework clearly focuses on risk and protective factors at the community and societal level.
• If successful, the proposed research will enhance our understanding of the effectiveness of community organizing and mobilization in changing community-level risk factors to reduce child maltreatment and youth violence within urban communities.Weaknesses • No weaknesses are noted.

Investigator(s): Strengths
• The PI Dr. Graham has a long history of conducting rigorous, community-based research and evaluating the effectiveness of youth violence prevention interventions in real-world and community-based settings and brings expertise in program evaluation, intervention adaptation, effectiveness trials, social determinants, implementation research, community-based and community-level research among diverse populations and settings, and implementation and prevention science.• The PI Dr. Yaros, an earlier career investigator, also brings expertise in mental health services in schools, school safety and discipline, parental aggression (e.g., child maltreatment), ACEs research, and evaluation of mental health services, and is well suited to lead the project with PI Dr. Graham.
• The Co-PIs both have extensive experience leading multi-site, multi-investigator evaluation and research teams.
• The Prevention Institute TTA team members have a history of providing technical assistance and support for ACE|R framework activities in Milwaukee, WI.The Prevention Institute has been working with the City of Milwaukee Office of Violence Prevention since 2016 to implement the ACE|R framework through the citywide Blueprint for Peace.
• The investigators have a history of collaboration and are building on an ongoing community initiative.Weaknesses • A minor weakness is that the City of Milwaukee Office of Violence Prevention is described as a partner but is not included as part of the investigative team.

Innovation: Strengths
• The ACE|R framework is innovative.
• The incorporation of community organizing to the current ACE|R framework and Blueprint for Peace initiative is innovative.
• The hybrid effectiveness-implementation design is innovative.

Weaknesses
• No weaknesses are noted.

Approach: Strengths
• The investigators clearly address primary outcomes 1 and 2 of the NOFO, including: 1) to determine which community-level or societal-level strategies effectively prevent multiple forms of violence and other ACEs that impact children and youth; and 2) evaluate strategies that enhance protective factors.
• The investigators propose a retrospective and prospective design that builds on current and previous work (work that begun in 2016) and enables the ability to examine long-term effectiveness of a community-level prevention strategy.
• The proposed project leverages other activities and funding in the community, including BUILD funding in one of the community sites.
• The proposed hybrid type 1 effectiveness-implementation design is appropriate for the proposed study.The evaluation includes an extensive evaluation of the implementation process guided by the Consolidated Framework for Implementation Research.Weaknesses • It is unclear what outcome was used to determine power, as the power analysis is not specific to an outcome.
• A minor weakness is the application includes a limited translation plan.The application includes discussion of creating an implementation manual and predominantly academic publications and presentations.It lacks detail on broader dissemination to the public and policy makers.

Environment: Strengths
• Letters of support are included.

Weaknesses
• No weaknesses are noted.• The approach includes longitudinal qualitative methods of archival materials and stakeholder interviews to explore barriers and facilitators which may likely provide relevant information to improve implementation of community-level interventions.

Weaknesses
• With a focus on surveillance data, unknown and unreported incidents of violence, abuse, and neglect will not be captured, which may be important for seeing change occurring at a smaller level.
• There are many proposed qualitative interviews (600) that may result in a large amount of data that may be unwieldy to analyze, particularly without a qualitative expert.
• The power analysis includes appropriate information but is not explicitly specific to the proposed idiographic clinical trial analysis.

Environment: Strengths
• The proposed application is to be completed at Research Triangle Institute (RTI) International which has a strong record of research accomplishment, substantial organization support, and a quality research operations center.• A large portion of the budget (60%) is intended for a sub-awardee.

Resource Sharing Plans Acceptable
Data Management Plan (DMP) Acceptable • The data management plan includes provisions for maintaining confidentiality of qualitative data, as well as for sharing quantitative data in appropriate formats.

THE FOLLOWING SECTIONS WERE PREPARED BY THE SCIENTIFIC REVIEW OFFICER TO SUMMARIZE THE OUTCOME OF DISCUSSIONS OF THE REVIEW COMMITTEE, OR REVIEWERS'
No weaknesses are noted.

•
Several large organizations (RTI, Prevention Institute, City of Milwaukee Office of Violence Prevention) can combine efforts to evaluate previous community-level interventions, while introducing and evaluating new components, building on the capital of current efforts.Weaknesses • A sub-awardee is the Prevention Institute and is budgeted at 60% of the award.The facilities and resource of Prevention Institute are only minimally addressed in the application.Protections for Human Subjects Acceptable Risks and/or Adequate Protection • The application clearly outlines the procedures in place to minimize risk, maintain confidentiality, and appropriate planned research procedures for human subjects research.Data and Safety Monitoring Plan (Applicable for Clinical Trials Only) Not Applicable Inclusion of Women, Minorites, and Age (Applicable Only for Human Subjects Research) G1A=Includes both genders, acceptable M1A=Includes minorities and non-minorities, acceptable C1A=Includes children, adults, and older adults, acceptable • The proposed inclusion is unknown because they are not selecting, recruiting, or limiting by gender/sex, race/ethnicity, and age, but are using archival data from Child Protective Services and surveillance data.Qualitative interviews with adults are based on demographics of community stakeholders and interventionists.