Predicting Community Engagement? The Carnegie Foundation’s Elective Classification

The Carnegie Foundation’s elective Community Engagement classification is valuable for colleges and universities seeking to demonstrate a mutually beneficial relationship with the community. This study uses logistic regression analysis to identify predictive institutional characteristics for applying for and receiving the Community Engagement classification. The findings suggest that publicly available institutional financial variables are not predictive of an institution applying for and receiving the classification.


Introduction
In 2006, the first set of institutions was granted the elective Community Engagement classification from the Carnegie Foundation for the Advancement of Teaching.The Community Engagement classification represents the first of a proposed line of elective classifications that are intended to serve a descriptive purpose for colleges and universities.The traditional Carnegie classification system was originally developed to aid researchers in higher education, but has since been used for a variety of other purposes not directly related to objective research, including rankings of institutions (McCormick & Zhao, 2005).The Carnegie Foundation has continuously evolved the classification system to keep with the original purpose of objectively providing information for research, but the drive for maintaining and moving up the rankings has made this difficult.To move the classification system forward, the Carnegie Foundation sought to "fill some of the gaps in the national data" by developing elective classifications to capture more of the work done by colleges and universities (McCormick & Zhao, 2005, p. 56).
The community engagement classification was the first of these elective classifications and was intended to "respect the diversity of institutions and their approaches to community engagement; engage institutions in a process of inquiry, reflection, and self-assessment; and honor institutions' achievements while promoting the ongoing development of their programs" (Driscoll, 2008, p. 39).In other words, this elective classification process was designed to bring a greater degree of nuance to how colleges and universities report their activities.The institutions that choose to participate in this elective classification process would have a Carnegie-sponsored endorsement of their engagement with the community.The institutional applications are evaluated by a National Advisory Panel made up of leading experts in the field of community engagement.
The first group of institutions to receive the elective classification was named in 2006, with a second round coming in 2008.In these first two cohorts, institutions could be classified as community engaged through curricular engagement, outreach and partnerships, or both (Carnegie Foundation, 2012).Curricular engagement refers to the integration of community engagement with the institution's teaching and learning.Outreach and partnerships required documentation of established and productive partnerships with the community.Applicants could choose to apply for either or both subcategories.The next time the classification was awarded, 2010, there were no longer subcategories; institutions had to demonstrate community engagement at all levels of the institution.In 2015, the institutions that were awarded the community engagement classification in either 2006 or 2008 will be required to reapply to maintain their classification, and the institutions that received the classification in 2010 will reapply in 2020.In addition to receiving the formal designation, the process of applying has shown to be instructive on many campuses, serving as a self-assessment tool.This has helped institutions to identify opportunities for improvement of their community engagement practices (Zuiches, 2008).
In order for colleges and universities to either continue with their community-engaged classification or to become classified for the first time, it is important for them to understand what is necessary for a successful application.The application form requires institutions to document and report their community-engaged activities.These applications are thoroughly evaluated by a review committee with regard to how well community engagement has been institutionalized.The purpose of this study is to examine elements related to community engagement on campus that are not reported on the Carnegie Foundation's documentation form.Specifically, this study examines publicly available financial data that are theoretically related to community engagement to determine whether those indicators can be predictive of an institution being awarded the classification.

Conceptual Framework
This study is guided using multiple theoretical and conceptual lenses.First, the scholarship of engagement explains how community-engaged work is viewed in higher education.The scholarship of engagement sets the stage for the community-engaged relationship that the Carnegie Foundation is seeking to reward.Second, this study examines how resources are allocated in higher education, relying primarily on the process described by Massey (1996).According to Massey, resources are in part allocated where they will do the most good and where the institution places its priorities.However, universities undertake complex processes in order to judiciously allocate resources.This theory will partially help to explain how institutions decide how much money to allocate to engagement initiatives.Finally, academic capitalism (Slaughter & Leslie, 1997) plays a role in the allocation of resources.Academic capitalism describes how colleges and universities have taken on characteristics of the market in order to remain competitive.These market forces have largely led to an increased emphasis on research and other activities that increase institutional prestige in higher education.

The Scholarship of Engagement
In Scholarship Reconsidered, Boyer (1990) outlined four areas of scholarship in higher education.The scholarship of discovery is associated with advancing new knowledge through research.The scholarship of teaching is associated with the transmission of knowledge from an expert (the professor or instructor) to a novice (the student).The scholarship of integration refers to scholars reaching across disciplinary lines in order to bring new perspectives to solving academic problems.Finally, the scholarship of application means answering the question, "How can knowledge be responsibly applied to consequential problems?How can it be useful to the individual as well as institutions" (p.21).In other words, how can colleges and universities use their collective resources beyond the ivory tower and move beyond basic research that advances knowledge for the pure sake of advancing knowledge?
Boyer continued to develop his ideas regarding the scholarship of application, and the concept is now more commonly referred to as the scholarship of engagement (Boyer, 1996).
Engagement extended what Boyer discussed with application.Whereas application refers to institutional resources being applied to consequential community problems, engagement takes this idea further and involves a greater role for the community in identifying and solving the problem.The scholarship of engagement means, "Connecting the rich resources of the university to our most pressing social, civic, and ethical problems…creating a special climate in which the academic and civic cultures communicate more continuously and more creatively with each other" (pp. 19-20).The key transition was the movement from the idea of the university doing work for the community, and instead doing work with the community.An engaged relationship between the institution and the community is now also seen as mutually beneficial (American Association of State Colleges and Universities, 2002).It is this type of engaged relationship that the Carnegie Foundation is interested in recognizing through the elective Community Engagement classification.For an official definition, the Carnegie Foundation states that "community engagement describes collaboration between institutions of higher education and their larger communities (local, regional/state, national, global) for the mutually beneficial exchange of knowledge and resources in a context of partnership and reciprocity" (Carnegie Foundation, 2012).

Resource Allocation in Higher Education
Massey (1996) discussed the fact that the allocation of funds in higher education goes beyond simply investing in the programs the institutions believe are the most important.While this is a simple solution, the practice of allocating resources is a process that involves actors with expertise and knowledge making informed decisions about the priorities of the institution.The actual process that is followed has a significant impact on where resources are allocated, and can provide insight into where the institution places its priorities.The process can help a college or university align its institutional mission and purpose with the demands of the market.Both of these elements are of crucial importance to the university: Proper response to the markets allows for an institution to remain competitive for the best students and the best faculty and staff, leading to greater rewards in terms of more diversified streams and increased prestige.Keeping consistent with its mission allows the university to remain true to its core values and many of the factors that make the institution unique.
According to Massey (1996), budgeting and allocating resources is often used strategically by universities and colleges.This framework applies to this particular study because of the decisionmaking process that is necessary when deciding the extent of the institution's financial commitment to community engagement.While receiving the Community Engagement classification may align with the institution's commitment to its mission, the amount of money may not necessarily reflect that commitment.Institutions can employ multiple strategies of resource allocation, and because of this, there are several paths available for institutions to be able to engage with their respective communities.

Academic Capitalism
The process of resource allocation has been influenced by the growing trend of academic capitalism, which is the theory that guides higher education's growing involvement in markets and market-like behaviors, with diversified streams of revenue (Rhoades & Slaughter, 1997;Slaughter & Leslie, 1997).This idea of academic capitalism stems from resource dependency, which dictates that an organization is beholden to its environment and the resources that environment makes available (Pfeffer & Salancik, 1978).In the current higher education landscape, research has often become the mechanism for increased institutional prestige, and investment in this arena has often come at the expense of other areas seen as less prestigious, or that generate less revenue.
Public service activities have not traditionally been seen as activities that increase the prestige of the institution, and are therefore slighted when institutions prepare their annual budgets as they decide how to allocate financial resources as the tendency toward academic capitalism increases (Jaeger & Thornton, 2005).As public service work (of which community-engaged work is a part) continues to take on an increasingly diminished role in higher education, the funding continues to decrease.This is a troubling trend, as stable funding is seen as crucial to maintaining a community-engaged institution (Kellogg Commission, 1999).If institutions are dedicated to community engagement to the degree that they are applying for the Community Engagement classification, then it should follow that the financial support of public service and community-engaged work is sufficiently supportive.In this study, financial variables associated with public service and communityengaged work are examined in order to see if they can serve as predictors of an institution receiving the Community Engagement classification.This study asks are market-like trends in fact diverting funding away from public service and community engagement, and are institutions still able to perform work the Carnegie Foundation deems worthy of the Community Engagement classification?
Within these conceptual frameworks, this study addresses the following research question: Can financial variables (both in terms of revenue and expenses) serve as significant predictors of whether an institution applies for and receives the Carnegie Foundation's elective Community Engagement classification?

Review of the Relevant Literature
In 2009, a special issue of New Directions for Higher Education was dedicated to lessons learned from the first group of institutions receiving the Carnegie Community Engagement classification (Sandmann, Thornton, & Jaeger, 2009).The articles in this compilation addressed issues related to leadership and engagement (Sandmann & Plater, 2009), rewards associated with community-engaged scholarship (Saltmarsh, Giles, Ward, & Buglione, 2009) (Weerts & Hudson, 2009), institutional understanding of engagement (Thornton & Zuiches, 2009), and the future of community engagement in higher education (Holland, 2009).However, these are not research studies per se, and rely primarily on the authors' experience and expertise with the institutions classified as community engaged to provide a set of best practices and theoretical implications for readers.Those who worked on successful Community Engagement applications have shared lessons that they learned during the application process (Zuiches, 2008), but those reports are largely anecdotal in nature and do not include empirical tests.
There have been multiple qualitative research studies involving the Community Engagement classification.For example, researchers have examined issues such as the adoption of engagement in higher education, how institutional characteristics and control influence approaches to engagement, and external understanding and evaluation of engagement (Weerts & Sandmann, 2008).Researchers have also qualitatively examined promotion and tenure policies as reported in the 2006 Carnegie Community Engagement classification applications (Saltmarsh, Giles, O'Meara, Sandmann, Ward, & Buglione, 2009).The Community Engagement classification has also been used to provide insight and commentary on how health collaborations have been encouraged in higher education (Sandmann & Driscoll, 2011).
While these qualitative studies and essays have been useful resources for those seeking to understand more about the elective Community Engagement classification, there is a paucity of quantitative research studies on the subject.This is understandable from the perspective that the application and documentation form largely requires narrative descriptions of programs that demonstrate connections and partnerships with the community as well as institutional commitment.However, in order to gain a different perspective of the Community Engagement classification, quantitative analysis should also be employed in order to begin to identify national and generalizable trends.This study seeks to begin to fill that gap in the literature by analyzing publicly available institutional data to determine whether these indicators are predictive of institutions receiving the Community Engagement classification.

Methods
This study employs logistic regression to determine whether a set of publicly available institutional financial characteristics can be predictive of the maximum likelihood of an institution applying for and receiving the Carnegie Community Engagement classification.The information provided in this study will lend insight as to whether there are factors other than the reporting application at play in the granting of the Community Engagement classification.

Sample
The institutions selected for this study are all four-year public colleges and universities.This study examines a cross-sectional sample of data from the academic year 2009 and uses that data to examine institutions that received the Community Engagement classification as a part of the 2010 cohort.The final sample includes 366 institutions, 47 of which received the Community Engagement classification and 319 which did not.Institutions that had previously received the designation in either 2006 or 2008 were omitted.

Variable Selection
The data for this study are publicly available from the Delta Cost Project (http://www.deltacostproject.org/),which was initiated to help researchers learn more about the spending and revenue practices of colleges and universities.Specifically for this study, several measures related to public service spending are examined as independent variables to determine whether they are potential predictors of the Carnegie Community Engagement classification.
Dependent variable.The dependent variable for this study is whether an institution was awarded the elective 2010 Carnegie Community Engagement classification.Institutions had to go through a rigorous application process in order to be considered.This study is only able to account for institutions that applied for and received the classification.
Independent variables.All independent variables used for this study are from the academic year 2009, which would have been the current academic year as the institutions were applying for the classification.The first independent variable used for this analysis is a categorical determination of whether the institution is a land-grant college or university.Leaders at landgrant colleges and universities have been working to return their institutions to their roots and give back to the state and communities in which they exist (Kellogg Commission, 1999; National Association of State Universities and Land-Grant Colleges, 2008).Land-grant institutions have a traditional commitment and service to their states and communities, and therefore, landgrant colleges and universities may be more likely to be awarded the Community Engagement classification.
One independent variable is related to revenue; however, there are no national measures of institutional revenue directly related to community engagement.To fill this gap, this study examines state appropriations to institutions.State appropriations are related to community engagement because links have been established between institutional commitment to outreach and engagement and state appropriations (Weerts & Ronca, 2006).To control for differences in total revenue among institutions, state appropriations as a share of total revenue will be examined.Initial analyses also included state appropriations per full-time equivalent student as an independent variable, but this was later dropped due to a high correlation (0.85) with public service spending per full-time equivalent student.
Two independent variables will be examined that are related to spending.While no national data exist that are dedicated solely to community engagement, the Delta Cost Project does have data related to overall public service spending.Community-engaged activities are public service activities, so they will generally be included in these data.To control for variances in enrollment, public service spending will be examined per fulltime equivalent student.Public service spending as a share of total spending was initially examined, but omitted from the final analysis due to high correlations with other variables.The Delta Cost Project also has data that track costs related to public service.The final independent variable related to public service spending is devoted to human resources.This variable is the proportion of total public service expenditures that are spent on salaries and wages.
The descriptive statistics for each variable are shown in Table 1.The Pearson correlation matrix for the independent variables is shown in Table 2. Spearman's rank correlation test was also run, but the results are consistent with the Pearson correlations and are not reported.Table 3 provides mean comparisons between the classified and non-classified institutions for the independent variables.As shown, the differences between the means for these independent variables are all insignificant.
Based on the variables, multiple logistic regression models were used to estimate whether an institution is classified as Community Engaged.Stepwise regression analysis was utilized to build toward the final model.The first model included the public service spending, the second model added the two variables related to state appropriations, the third model added the spending related to human resources, and the final model included land-grant status.The progression of the four models is shown below: Several other independent variables available from the Delta Cost Project were also initially examined, but were not used in the final analysis because very high correlations were found.These variables were eliminated in order to prevent multicollinearity (Menard, 2002).

Methods of Analysis
Logistic regression is the primary, and appropriate, method of analysis for this study because the dependent variable is dichotomous (Hosmer & Lemeshow, 2000).The goal of logistic regression is to determine a maximum likelihood fit for the data.In other words, the independent variables are used to predict the maximum likelihood of the dependent variable in the sample of data being observed being true (joint probability of Y 1 …Y N ).
Pearl: Predicting Community Engagement?The Carnegie Foundation's Electi Published by Nighthawks Open Institutional Repository, 2019 For this study, the following null and alternative hypotheses for the entire logistic model are designated below: The two-tailed hypotheses tests for each of the individual j-th variables determine if the independent variables have a significant effect on the dependent variable holding all other independent variables constant.For each j-th variable, the null and alternative hypotheses are shown below:

Findings
For the purposes of comparison, the findings from both logistic regression models are shown.Table 4 shows the results from the four logistic regression models, built stepwise toward the final model as described above.Results are presented in terms of b coefficients as well as odds ratios.As shown in Table 4, the results are consistent in terms of the significance of the findings.
The most striking result from the logistic regression analysis is that none of the models as designed for this study are good fits for finding the maximum likelihood of whether an institution was awarded the Carnegie Community Engagement classification (p > 0.05 in all four cases).Even in the final model, only about 2% of the maximum likelihood is explained, as with the progressive pseudo R 2 values.In addition, none of the subsequent models represents an improvement over the previous model, as none of the chi-square values were statistically significant.In addition to the entire model being statistically insignificant, each individual variable was also insignificant in each progression of the model.The Delta P method was used to approximate the probabilities of the variables for each model.The Delta P method estimates the overall change in the dependent variable (Cabrera, 1994).The Delta P estimates are shown in Table 5.
Based on the Delta P estimates, several elements stand out.First, in both models, the independent variable with the greatest impact is state appropriations per full-time equivalent student, although it is important to reiterate that this variable is not statistically significant in any of the models.Delta P estimates are useful for understanding the impact of independent variables in a logistic regression model; however, in this study, none of the independent variables are statistically significant in any of the models, so it is imprudent to use the Delta P estimates to say anything definitive about the independent variables' impact on whether an institution is classified as community engaged.

Discussion and Implications of the Findings
The findings of this study indicate that the logistic regression models developed are not predictive of an institution applying for and receiving the Carnegie Foundation's elective Community Engagement classification.These findings can have several important implications for policies at the Carnegie Foundation, institutions of higher education, and the state systems of higher education.
For the Carnegie Foundation, these findings indicate that a great deal of emphasis is placed in the documentation and reporting form.The application form is focused primarily on the actual practice of community-engaged work, not on the amount of money that has been allocated.Financial indicators and other quantitative measures play a role in the all-inclusive Carnegie classification system, but the elective classifications are intended to provide an additional level of insight into what colleges and universities do and "recognize important aspects of institutional mission and action that are not represented in the national data" (Carnegie Foundation, 2012).From this perspective, the initial findings suggest that the Carnegie Foundation has achieved this goal to a degree.By and large, the Pearl: Predicting Community Engagement?The Carnegie Foundation's Electi Published by Nighthawks Open Institutional Repository, 2019 nationally available data do not appear to serve a predictive function for an institution receiving the Community Engagement classification.It is largely the practice of community engagement that leads to being awarded the elective classification.
For the institution, there are also important implications that can be drawn from the findings.First among these is that those involved in community-engaged activities are often asked by the institution to do more good work with fewer financial resources.There is no clear and consistent connection between financial indicators and being rewarded for community-engaged activity by the Carnegie Foundation elective classification.
The Carnegie Foundation Community Engagement classification is just one way for a college or university to demonstrate its commitment to its community.Simply spending more money at the state level or the institution level does not necessarily mean that the institution is taking the necessary steps to fully commit to community engagement.For institutionalization to occur, engagement must become a priority on campus with a fully developed plan for achieving this goal that includes interdisciplinary activity and proper incentives for faculty involvement (Kellogg Commission, 1999).This study indicates that among the sample, more is at work in higher education community engagement than finances, and that institutions may be working toward achieving institutionalization at multiple levels.

Limitations of the Study
This study is limited by several factors.First, the Carnegie Foundation makes it clear that the Community Engagement classification is not intended to be comprehensive, largely because it is a voluntary classification (Carnegie Foundation, 2012).Institutions are not required to document their community-engaged work and apply for the classification.Therefore, there is a strong possibility that many colleges and universities are engaged with their communities, but have not taken the initiative to apply for the classification.This lack of comprehensiveness is further emphasized by the relative newness of the classification.The first cohort of colleges and universities designated as Community Engaged was named in 2006, and there have only been two subsequent classifications since then.As the Community Engagement classification gains traction in higher education, it is possible that more institutions engaged with their communities will apply for the classification.In addition, information is only available for institutions that applied for and received the classification.The Carnegie Foundation does not make available the institutions that began the process but did not submit a completed application, or the institutions that completed the process but did not receive the designation.
This study is also limited by the national data available for analysis.There are no national measures of community engagement, so this study utilized variables that are theoretically linked to public service, but may not paint a complete picture.For example, public service spending includes community-engaged work, but also a variety of other activities related to serving the public.Institutions may also have different reporting structures, and community-engaged work may not always be reported as public service work in the national data.Until there is mandatory national data available, a study like this will be limited by the availability of data.
Finally, this study is limited by the sample.Only investigating public four-year institutions leaves out private institutions and community colleges (and other two-year schools) that participate in community-engaged work.Many of these institutions have been awarded the Community Engagement classification, but because of the inherent structural differences, it is difficult to

Conclusion
The Carnegie Foundation's elective Community Engagement classification is an important first step in recognizing a greater variety of the work done in higher education that is not encompassed in the general Carnegie Foundation classification.It is an encouraging sign that the national data available through the Delta Cost Project is not predictive of an institution receiving the classification.This implies that the elective classification is indeed serving its purpose of telling a more complete picture of colleges and universities.If the national data were predictive, the Community Engagement classification would simply be telling the same story in a different way.Telling the complete story of higher education is important.Without being able to effectively communicate the full range of university activity with all stakeholders, competitiveness can suffer, and mission drift can occur.

Table 3 .
Carnegie and Non-Carnegie Classified Institutions

Table 5 .
Delta P Estimates (Final Model)