Changing organisational routines in doctoral education: an intervention to infuse social justice into a social welfare curriculum

This paper describes one effort to infuse a social justice framework into a social work doctoral education programme in a prominent research university of the United States. The “Social Justice in Doctoral Education” (SJDE) Project identified Social Justice Learning Objectives (SJLOs) in the categories of scholarship, teaching, and service. Doctoral students were surveyed in 2010 to determine the extent to which the SJLOs were being systematically facilitated by their doctoral programme. The forms that guide and shape the milestones of doctoral education at that institution were revised in 2011 in an attempt to create new opportunities for social justice learning. A second survey of doctoral students in 2013 resulted in two findings. First, doctoral students reported using the SJLOs to guide their education. Second, a pre/post comparison of student perceptions indicated an increase in opportunities for social justice learning through doctoral education. This case study provides preliminary support for the modification of organisational routines to expand social justice education in social work.

ABSTRACT: This paper describes one effort to infuse a social justice framework into a social work doctoral education programme in a prominent research university of the United States. The "Social Justice in Doctoral Education" (SJDE) Project identified Social Justice Learning Objectives (SJLOs) in the categories of scholarship, teaching, and service. Doctoral students were surveyed in 2010 to determine the extent to which the SJLOs were being systematically facilitated by their doctoral programme. The forms that guide and shape the milestones of doctoral education at that institution were revised in 2011 in an attempt to create new opportunities for social justice learning. A second survey of doctoral students in 2013 resulted in two findings. First, doctoral students reported using the SJLOs to guide their education. Second, a pre/post comparison of student perceptions indicated an increase in opportunities for social justice learning through doctoral education. This case study provides preliminary support for the modification of organisational routines to expand social justice education in social work. KEYWORDS: social justice; doctoral student; doctoral education; social work education. RESUMEN: En este artículo se describe el esfuerzo para infundir un marco de justicia social en un programa doctoral de trabajo social dentro de una universidad prominente de investigación de los Estados Unidos. El proyecto de investigación "Justicia Social en la Educación Doctoral" (SJDE) identificó los Objetivos de Aprendizaje de la Justicia Social (SJLOs) en una serie de categorías de la investigación científica, como la enseñanza y el servicio. Los estudiantes de doctorado respondieron a una encuesta en 2010 para determinar el grado en el que los SJLOs se facilitaban sistemáticamente en el programa de doctorado. En 2011 se revisaron los formularios que guían y dan forma a los hitos de la educación doctoral en esa institución, en un intento de crear nuevas oportunidades para la justicia social de aprendizaje. En 2013, una encuesta seguimiento a los estudiantes de doctorado dio lugar a dos conclusiones. La primera es que los estudiantes de doctorado informaron del uso de las SJLOs como guía de su educación. La segunda es que una comparación pre / post de las percepciones de los estudiantes indicó el aumento de oportunidades para el aprendizaje de la justicia social por medio de sus estudios de doctorado. En conclusión, este caso de estudio nos proporciona evidencia preliminar para la modificación de las rutinas organizativas, como un medio para ampliar la educación de la justicia social en el trabajo social.

INTRODUCTION
Social work, like many allied service professions, holds an aspiration for inclusive, equitable, and justice-promoting professional practice (Banks, 2001;CSWE, 2002;Reamer, 2006). While conceptual clarity around the term "social justice" is lacking, the profession has embraced the promotion of justice as an ideal (Reisch, 2002). This imprecise ideal requires a wide array of strategies to address the broad aims of preparing a diverse professional workforce to honor individual difference, embrace self-determination, and confront systems of inequity and oppression (Finn and Jacobson, 2003;Gil, 1998;Lieberman and Lester, 2004;Swenson, 1998). As a result, scholars, educators, administrators, practitioners, students, service users, and concerned citizens have called for innovative strategies to recruit and retain a diverse student body, promote critical thinking, create an equitable and inclusive learning environment, and infuse diversity and social justice content into graduate curricula in ways that enable emerging social workers to develop relevant knowledge and skills for advancing human rights (see Lee and Greene, 2004;Nagda and Derr, 2004;Thyer and Myers, 2009;Van Soest, 1995).Although these objectives are interdependent and contextualized by the modern university and professional contexts of social work education (Gewirtz, 2006;McInerney, 2007;Osei-Kofi, Shahjahan and Patton, 2010), this paper specifically focuses on one effort to infuse asocial justice framework into one social work curricula within a major research university of the United States.

APPROACHES TO CURRICULAR REFORM
There are a variety of strategies that have been employed to enhance social work curricula to address professional mandates and contemporary workforce needs. Approaches to curriculum reform can be broadly organized into three categories: specialization, integration, and infusion (Hooyman, 2006). As examples of the specialization approach, some universities have given students the opportunity to choose a multiethnic practice concentration (Ishisaka et al., 2004), take specialized elective courses, or participate in experiential program components (e.g., service learning involvement, intergroup dialogue). Specialization has advantages for students who wish to deeply pursue social justice content or justice promoting practice methods (Rodriguez et al., 2010;Nagda and Derr, 2004). This approach also requires a large investment to develop and sustain, is often challenged to recruit and retain sufficient faculty expertise without disparate burden, can promote isolation of persons or content, and is ill-suited for reaching a large number of students (Hooyman, 2006).
A second approach, the integration of multicultural or social justice content into required trainings or coursework for all students, may be an alternative to the potential marginalization of the specialization approach (Rodriguez et al., 2010). Yet an integration approach also carries a set of challenges (Calley et al., 2011;Pittman, 2009). Integration may rely on external presenters or outside lectures featured as guests in spaces in which they are not fully integrated and that they do not control. Integration may be more successful if improvements could be made to the pedagogical practice and preparation of all instructors. Facilitating the uptake and high-quality implementation of social justice content would likely require deliberate training of existing or upcoming social work educators (Funge, 2011;Gutiérrez, Fredricksen and Soifer, 1999;Garcia and Van Soest, 2000;Hackman, 2005;Hudson et al., 2014). Yet, even if existing and upcoming instructors were universally prepared to deliver social justice content, a number of barriers persist. For example, content to be integrated is often (a) positioned as "in competition" for space in an already crowded course or course sequence, (b) delivered in a way that may seem supplemental and non-essential to the "core" objectives of the course or the program, (c) treated with insufficient depth and/or skipped in cases of insufficient time in non-dedicated courses, and(d) alienated as yet another burden on faculty and student time should a dedicated course be developed (Hooyman, 2006). Social work education faces the challenge of selecting a means for integrating social justice content into curricula despite insufficient empirical exploration of the associations between various training models, sustained practice behavior, and practice outcomes with clients in the area of multicultural or justice-oriented education (Yaffe, 2013;Brach and Fraser, 2000).Recent studies have suggested that very little variance in student attitudes toward social justice is explained by whether a student participated in a required multicultural or social justice oriented course (Osteen, Vanidestine and Sharpe, 2013). Similarly, recent research did not detect a relationship between participation in such requirements and subsequent social justice actions (e.g., challenging derogatory comments; Pittman, 2009).
A third strategy of curriculum reform attempts to avoid "adding one more thing" to a "full bucket" but instead strives to "stir the bucket in a new way" a202 (Hooyman, 2006, p.13). An infusion approach involves identifying cross-cutting themes that present across the entire curriculum, communicating these themes in program materials, updating the objectives of existing courses to reflect these themes, and/or using assignments to assess learning of novel content related to these themes. Developing and embedding such themes does not prohibit specialization or integration strategies for curricular reform, but may actually complement or enhance those other efforts. An infusion approach attempts to work in collaboration with key stakeholders and governance bodies, leverage social influence (e.g., change social norms), and initially target the modification of organizational routines rather than individual attitudes or behavior (Hooyman, 2006;Pittman, 2009).
Although not explicitly named as such, an infusion approach to curricular reform has been used to shape the Council for Social Work Education (CSWE) reaccreditation review process that shapes bachelor of social work and master of social work education in the United States. Implementation of the 2008 Education and Policy Accreditation Standards (EPAS) ask programs to designate broad themes that constitute a program framework, which in turn communicate the focus of the program, organize the content of the curriculum, and provide a structure for teaching and assessing learning objectives (Holloway et al., 2009). Ultimately, this process is intended to connect the broad program mission to a matrix of student learning competencies. Doctoral education in social work and social welfare is not accredited by CSWE, or any other entity, and thus may not be subjected to similar selfstudy, review, and revitalization projects as regularly or on a similar scale.

CURRICULAR REFORM IN DOCTORAL EDUCATION
Although it does not receive as much attention, doctoral education in social work and social welfare is also an important place for curricular innovation and renewal. Doctoral education functions as a pipeline for generating the content knowledge and pedagogical approaches for the rest of social work education, as doctoral education is the profession's primary means for training future social work scholars and educators. Doctoral education has specifically been referenced as an essential mechanism for the profession achieving its ideals (Anastas and Congress, 1999;Funge, 2011;Hudson et al., 2014;Lubben and Harootyan, 2003). Doctoral education in social work and social welfare, however, is often described as being in a state of crisis (CSWE, 2012;Khinduka, 2002;Lindsey and Kirk, 1992;Orme, 2003;Valentine et al., 1998). Regardless of the nature of the crisis within doctoral education being described, approaches to resolving the crisis and thus improving doctoral education are opaque. The Group for the Advancement of Doctoral Education in Social Work (GADE), a member-run organization whose primary purpose is the promotion of excellence in doctoral education in social work, does not endorse accrediting or otherwise regulating doctoral programs as a means to advance doctoral education, strongly affirming "university control of the quality of doctoral programs and, concomitantly, that doctoral education should develop within the philosophy of the host institution" (GADE, 2003, p. 2). Thus, the advancement of doctoral education is largely dependent upon innovations within individual university settings that are ultimately shared, scrutinized, refined, tested, and applied (Thyer, 2002). GADE first produced Quality Guidelines for PhD Programs in Social Work in 1992 that were "not meant to be proscriptive, but instead provide guidelines to which programs may aspire." (2013, p. 1). The 2003 revision of the Quality Guidelines suggested that doctoral programs have diversity content in the curriculum. The 2013 revision suggests that graduates "understand how knowledge in social work is relevant to public issues, including promoting social justice and increasing equity" (2013, p. 2). A larger discussion regarding how to support doctoral students in preparing for careers of social justice infused scholarship, teaching, and service have been relatively absent (Schiele and Wilson, 2001). A report commissioned by CSWE summarizing the results of a national survey intended to capture the doctoral student perspective recently concluded, "It would seem urgent that diversity issues be discussed with respect to doctoral education in social work." (Anastas, 2012, p. 113).
The Social Justice in Doctoral Education (SJDE) Project is a multi-phased action-research project designed to explore social justice in doctoral education and strategies for curricular innovation. The SJDE Project began at one institution, where work focused on using an infusion model to curriculum change and targeted the organizational structures and routines that guide the highly individualized experience of doctoral education. The SJDE Project used incremental, institutionalized change approaches adapted from strategies to address disparities by changing routines in service organizations (Lee, 2010). Such models articulate a change process that progresses from building aware-a202 ness of a strategic issue, systematically evaluating the issue, exploring pathways for improvement, making changes to the organization, to renewing and expanding commitments to organizational change (Dreachslin, 1996). This paper describes an effort to infuse a social justice framework into doctoral education in social work by changing the organizational routines that shape doctoral education at one institution.

STAGE 1: BUILDING AWARENESS
The SJDE Project's first initiative represented an attempt at curricular reform during one moment in a school's institutional history, rich with change-seeking efforts to promote social justice (Hudson et al., 2014). The project was initiated when students perceived a misalignment between the emphasis on social justice in the mission and recruitment priorities of the doctoral program and the curricular emphasis and institutional support for social justice learning and action as embodied in course objectives, student learning plans, dissertation work, community and institutional service, and pedagogical training.
A Social Justice Committee (SJC) was commissioned by the governing body of this doctoral program, comprised of self-nominated student and faculty representatives who contributed diverse perspectives based on their identities, positionalites, and educational experiences. The SJC initiated the SJDE Project initially to raise awareness of the breadth and depth of social justice ideals held by students and faculty to make more explicit the expectations for student competence and for programmatic opportunities and to more explicitly recognize the ways in which students and faculty were embodying and expanding their commitments to social justice. The SJDE Project did not attempt to reach a singular consensus definition of social justice, but rather, identify exemplars of ways to promote social justice through scholarship, teaching, and service that was inclusive of many conceptualizations of the term (Bell et al., 1997;Deal and Hyde, 2004;Fleck-Henderson and Melendez, 2009;Funge, 2011;Garcia and Van Soest, 2000;Granruth, 2009;Tummala-Narra, 2009). Thus, the SJDE Project's first effort was to develop and build internal consensus around exemplars of social justice in doctoral education, a participatory process described in detail elsewhere (Hudson et al., 2014). Reflective of an infusion approach to curricular reform, these ideals emerged as learning objectives that reflected crosscutting themes of the doctoral education: preparing students for the three domains of the professorate (scholarship, teaching, and service), developing a broad understanding of the major policy and practice trends and issues in the field of social welfare and the profession of social work, acquiring substantive knowledge of a field of social welfare, and developing competence to use rigorous methods. Although some voices expressed disappointment in the emerging learning objectives, in part because competency-based education did not resonate with their ontological/epistemological views, the Social Justice Learning Objectives (SJLOs; Hudson et al., 2014) resulted in a surprising degree of consensus and were formally adopted by the governing body of the doctoral program as a framework for social justice education in the doctoral program (see Figure 1).

STAGE 2: SYSTEMATIC EVALUATION
The SJDE Project, through actions of the SJC, next undertook a systematic inquiry into the present status of social justice education at the institution that had endorsed the SJLOs as a framework. Data were collected in the autumn of 2010 through a web-based survey of doctoral students and recent alumni (graduated since 2007) to determine the extent to which the SJLOs were already being accomplished and which objectives were their most urgent priorities for change. The survey was designed and administered through WebQ, a Catalyst Tool 2 available through the University Learning and Scholarly Technologies suite, an internet-based survey platform developed for institutional use by the University Department of Information Technology. The anonymous survey, accessed through an Internet link, consisted of 22 items. The first three items helped to define the respondent's relationship to the doctoral program (e.g., affirmed student status, assessed stage of program). The fourth and fifth questions asked, on four point scales, "how satisfactory do you believe doctoral training is at the School of Social Work" and "to what extent is your 'opinion of how satisfactory the program is' explained by your perceptions of social justice training opportunities and the sense of a socially just institution?". The sixth question inquired, on a five point scale, about the respondent's prior knowledge of the SJLOs and/or participation in the development of them.
Questions 7-18 were delivered across three survey pages, with each page addressing the SJLO's contained in one of the matrix columns (scholarship, teaching, or service). Participants were asked to what extent the training program provided opportunities for engaging with the SJLOs on a 5 point Likert Scale (from 2 to -2) with the anchors of (2) Facilitates; (1) Supports, but does Facilitate; (0) Does not actively Support or (1) Cultivate a working knowledge of major theories of social justice (across disciplines, historical contexts, and communities) and their implications for social welfare scholarship.
(2) Develop capacity to assess and communicate how social welfare research, policies, and practices can both empower and oppress communities they are purported to serve.
(3) Develop reflective practices to understand self as a scholar given positionality in the context of power dynamics.
(1) Demonstrate a commitment to integrating diverse teaching/mentoring methods.
(2) Understand how historical and contemporary education policies have shaped social work education in ways that oppress, liberate, and transform the classroom and the profession.
(3) Articulate teaching philosophy that reflects social justice values.
(1) Articulate approaches to building and engaging in just partnerships.
(2) Reflect upon the impact of identity, power, and the privilege of the academy in service work.
(3) Advocate for an institutional definition of service that values work both within and outside the academy.
Substantive Area (1) In chosen area of interest, understand dominant paradigms and critiques that center social justice across multiple levels of investigation, translation, and dissemination.
(2) Identify and articulate social justice goals and implications of individual research program and applications for the profession.
(1) Incorporate social justice content into instruction within teaching specialty.
(2) Gain and develop a working knowledge of positionality, biases, and beliefs that may influence teaching, mentoring, and/or supervising to improve capacity to work effectively across difference.
(1) Know systems/structures in area of interest and confront associated disparities and injustices that perpetuate oppression/marginalization.
(2) Build and maintain constructive relationships with communities in area of interest to bridge gap between research and practice.
(3) Honor community priorities and wisdom in the academy and use appropriate academy resources to catalyze community goals.

Methods
(1) Demonstrate and apply critical inquiry into uses/misuses of research methods and articulation of just methodology.
(2) Seek out, identify, and work to enhance transformative potential of chosen research tools.
(3) Understand social justice implications and issues present throughout each stage of the research process.
(1) Design learning objectives and implement instructional strategies that promote critical thinking.
(3) Solicit student feedback and strive to continuously improve instruction from a social justice perspective.
(4)Effectively facilitate group dynamics around issues of power and oppression in the classroom.
(1) Learn strategies for collegial and responsible engagement.
(2) Assume leadership roles with humility and thoughtfulness.
(4) Approach and engage people with awareness of your own positionality and cultural lens. Discourage; (-1) Discourages, but does not Obstruct; and (-2) Obstructs. Question 19 asked respondents to select up to five of the SJLOs that represented the most urgent areas to focus on improving over the next year. Questions 20-21 inquired whether the respondent intended to come to the town hall meeting and/ or participate in focus groups to help interpret the data and develop an action plan, and provided space to solicit initial thoughts in that regard. Question 22 asked respondents, "To what extent do you identify as a person belonging to identity groups structurally marginalized in academia?" with four response choices ranging from "not at all" to "very much so." This question was intended to help describe our sample and the relationship between marginalization and other survey responses. The phrasing of this question was intentionally vague as to not reveal the identities of the respondents by asking about different dimensions of intersecting identities. The web-survey was administered over a 10 day period during the first month of the academic year. Participation was solicited through electronic communications to the School of Social Work doctoral student listservs as well as through personal communications.
Survey respondents (n=32) included 64% of the active doctoral student body enrolled during the autumn of 2010 and some recent graduates. Most doctoral programs in the United States have sequential phases of doctoral training characterized by common coursework or coursework distribution requirements, active work towards a milestone that renders the student eligible for dissertation work, and active work on the doctoral dissertation. Survey respondents identified as being in diverse phases of doctoral training, with about 42% in coursework, 26% working on the milestone qualifying the student for candidacy, 16% working on their dissertation, and 16% recent graduates (see Table 1). Thirty percent of the respondents had been involved in SJLO development in some capacity. Another 60% of respondents were aware of the SJDE project, but had not participated, while 10% of respondents were learning about the SJDE project for the first time while taking the survey. Fifty-seven percent of students considered themselves to "very much" belong to identity groups structurally marginalized in academia and the vast majority of students indicated overall satisfaction with doctoral training (34%=very satisfied; 38%=satisfied; 25%=unsatisfied; 3% very unsatisfied).

Description of milestone
Changes made

End of Year Advising Form
Filled out annually by each student and their primary mentor, this form documents the student's progress toward degree requirements and sets goals for the upcoming year.
Added a prompt for student/mentor to reflect on any activities that furthered the SJLOs.
Added a second prompt for student/mentor to identify resources that the student needs from the program to help meet SJLOs.

General Exam / Dissertation
The general exam (which advances students to candidacy) involves writing a proposal, writing of a paper, an oral presentation of the paper, and an oral defense of the paper. The dissertation involves similar steps (proposing, writing, presenting, and defending).
The program manual contains guidelines describing both milestones. Students' advisory committees complete evaluation forms to assess whether the student has successfully demonstrated mastery.
Added language to the guidelines for the general exam and dissertation proposals that asks students to articulate how the paper/dissertation is "relevant to social work's mission to enhance social justice".
Added and revised items to the evaluation forms that assess the extent to which students thoughtfully engaged with social justice throughout the process of conceptualizing, proposing, carrying out, writing, and defending the general exam and the dissertation.
Plan for Dissemination and Community Engagement Dissertation guidelines suggested that students write an Op-Ed for publication in a newspaper or other venue following completion of the dissertation.
Removed Op-Ed suggestion. Added a requirement that students describe their plan for community engagement and/or dissemination of research findings in the dissertation prospectus. This transformed the original Op-Ed into a broader goal that encourages students to think about dissemination and community engagement, and the accompanying social justice implications, early in the planning stage of the dissertation.

a202
In an attempt to understand whether satisfaction with the training program was explained by issues related to social justice, we dichotomized the distribution of satisfaction data into the broad categories of satisfied and unsatisfied. When respondents were satisfied, the modal response was that their satisfaction was "partially explained" by issues related to social justice (58%), followed by "significantly explained" (29%), "unexplained" (8%), and "entirely explained" (4%). When respondents were unsatisfied, the bimodal response was that their satisfaction was "partially" or "significantly" explained by issues related to social justice (45% each), followed by "entirely" explained (10%), with no one reporting that their dissatisfaction was unexplained by issues of social justice. An independent sample t-test revealed marginally significant differences between these distributions (p=.08). The disparity in satisfaction was also examined by identification with a marginalized identity group. While respondents satisfaction with doctoral training varied in their degree of affiliation with communities structurally marginalized in academia (4%=not at all; 13%=not too much; 33%=somewhat; 38%=very much so), all of the unsatisfied respondents who responded to the question about their identity (89%), indicated that they "very much" identified with marginalized communities. The mean differences in the degree of marginalization by groups of satisfied and unsatisfied doctoral students were statistically significant (p < .001).
The respondents reported that the most actively supported objectives were in the scholarship domain (average across items; M=.52; SD=.97). This finding is consistent with what may be expected from a researchoriented doctoral program. Items in the teaching domain were less actively supported (average across items; M=.42; SD=.91). The least supported items were in the service domain (average across items; M=.14; SD=.78). The specific learning objectives with the lowest means (< .10) were all in the teaching and service domains. In the teaching domain these items included: effectively facilitate group dynamics around issues of power and oppression in the classroom; understand how historical and contemporary education policies have shaped social work education in ways that oppress, liberate, and transform the classroom and the profession. In the service domain these items were: honor community priorities and wisdom in the academy and use appropriate academy resources to catalyze community goals; build and maintain constructive relationships with communities in area of interest to bridge gap between research and practice; participate in public discourse; and advocate for an institutional definition of service that values work both within and outside the academy.

STAGE 3: EXPLORING PATHWAYS FOR IMPROVEMENT
The data acquired from Questions 7-18 were presented to participants at the Doctoral Community Town Hall Meeting 3 at the end of October 2010. Participants in the Town Hall Meeting discussed the items that were most frequently endorsed as being a high priority for change (endorsed by more than 25% of respondents). Based upon this criterion, no items were prioritized for immediate change efforts from the teaching domain. Items that were highly endorsed as a priority included items in the scholarship(Develop reflective practices to understand self as a scholar given positionality in the context of power dynamics; Demonstrate and apply critical inquiry into uses/ misuses of research methods and articulation of a just methodology; Cultivate a working knowledge of major themes of social justice across disciplines, historical contexts, and communities and their implications for social welfare scholarship) and service domains (Know systems and structures in area of interest and confront associated disparities and injustices that perpetuate oppression/marginalization; Build and maintain constructive relationships with communities in area of interest to bridge the gap between research and practice; Approach and engage people with awareness of your own positionality and cultural lens). Discussion at the Town Hall Meeting considered the degree to which doctoral students have very different experiences within the same program. Students acknowledged that some of these opportunities are related to whether their mentors strive to center social justice in their research, teaching, and service activities, and their willingness to involve the mentee in such work. Students also acknowledged a different opportunity structure in doctoral education based on the student's source of funding, whether the student has a community network in the region where the training institution is located or whether the student is a temporary resident during his or her education, and whether the social problem under study is well represented in the immediate geographic area of the training institution.
Choosing an infusion approach to curricular reform, the SJDE Project, acting through the SJC and with key stakeholders, proactively explored pragmatic changes that could be implemented within an academic year, make a meaningful difference, and would be sustainable. The SJC pursued incremental change, focusing on small changes that could build upon each other, that would be consistent with the program's mission, fit within existing institutional values, be holistic, and provide a vision for change that empowered the ben-a202 eficiaries to create its meaning (Proehl, 2001). The SJDE Project's commitment to an infusion approach that targeted organizational change by "stirring" existing buckets led to a decision to target the "routines" that organized the progression of doctoral education. A routine, or a "repetitive, recognizable pattern of interdependent actions, involving multiple actors" is often documented in organizations as formal procedures or rules (Feldman and Pentland, 2003, p. 96) that reduce complexity and conflict by creating efficiency, accountability, standardization, and stability. Routines can create legitimacy for the organization, in this case, a doctoral program, by helping the student and mentor behavior conform to agreed upon norms in ways that reinforce and reproduce an underlying structure. These routines can serve to maintain the status quo, but may also be a mechanism for flexibility and change (Feldman and Pentland, 2003).
The routines of a typical doctoral program in Social Work and Social Welfare are communicated through the guidelines that structure program milestones, the artifacts of these routines being the paperwork (instructions, forms, and evaluations) that give subjective meaning to the routine by shaping the performative aspect of the routine. The paperwork associated with program milestones ensure routines that are broad enough to provide flexibility and agency to students, but constrain the range of acceptable performances by delegitimizing performances that are viewed as inconsistent with the function of the routine.
Within the doctoral program adopting the SJLOs, the major organizational routines that shape the progression through doctoral educational include admissions, creating individual learning plans during the first year of study, participating in research and teaching practica, advancing to candidacy through a qualifying examination, proposing a dissertation, and defending a dissertation. These milestones were opportunities to consider how to infuse social justice training within the students' individualized programs of study. The SJDE Project, acting through the SJC and with key stakeholders, decided to evolve the guidelines that describe these milestones, the forms used to track progress through each of these milestones, and the evaluations of these milestones, to infuse a social justice framework into the structure of doctoral education at this institution. These guidelines, forms, and evaluations ensure mutual responsibility; students demonstrate progress through each of the milestones and the doctoral program provides sufficient resources and support to help students progress. Be-cause of the mutual responsibility embedded in these program milestones, changing the routine was conceived to impact both the individual and programmatic level. In addition, most of the guidelines, forms, and evaluations shape multiple routines, including how students and faculty plan for each student's learning experience and retrospectively reflect on the learning and progress that students have made, providing opportunities to create learning experiences, assess competence, and receive recognition.

STAGE 4: MAKING CHANGES TO THE ORGANIZATION
The SJDE Project, acting through the SJC, undertook a careful review of the doctoral program manual and the guidelines, forms, and evaluations pertaining to each of the program milestones to identify how these materials could be revised to create and expand opportunities for engagement with the SJLOs. The result of this review process was the preparation of detailed recommendations for changes that would integrate the SJLOs into the organizational routines that structure the doctoral experience. Feedback on the suggested revisions was gathered until a broad consensus was reached. All revisions were approved for adoption by the governing body of the doctoral program and instituted for use in the summer of 2011. Table 1 includes a few examples of the changes that were made to program forms. The aim of these revisions was to provide students and mentors an opportunity to reflect, appraise, and incorporate the SJLOs into students' overall training program. They allow for assessment of individual student progress, while also providing the opportunity to assess programmatically how students and their mentors interpret and operationalize the SJLOs. This opportunity to assess progress at the program level may help with ongoing implementation, including assisting the program in identifying what additional resources are needed to support students in meeting the SJLOs. For example, in response to feedback requesting more facilitation of the SJLOs in the service domain, the program is piloting a service-oriented practicum experience, in the spirit of existing research and teaching practicums, to have mentored experiences in completing peer reviews of manuscripts and engaging communities to determine research priorities.
In the autumn of 2013, data were again collected through a web-based survey of doctoral students and recent alumni (graduated since 2012) to determine the extent to which the SJLOs were actively being used in doctoral student routines, which we would a202 expect to see given the changes made to the forms that structure the routines. Participation was solicited through electronic communications to doctoral student listservs as well as through personal communications. Survey respondents (n=13) included 26% of the active doctoral student body enrolled during the autumn of 2013 and some recent graduates. Table 2 reports responses to the prompt, "In what way(s) has this matrix of objectives been used or referenced in your doctoral education thus far? (Check all that apply)." Over half of the respondents reported using the SJLOs to "inform personal goals for doctoral education" and creating their Individualized Learning Plan. Since many students enrolled in the doctoral program in 2013 encountered program milestones before organizational routines were modified, we also report the percentage of students reporting uses of the SJLOs who matriculated in the autumn of 2010 or later. Among these students, over 50% report SJLO use in classroom conversations, in conversations with their advisors, and in creating their Individualized Education Plan. In fact, there was a statistically significant increase in the average number of ways in which the SJLOs were used between students who matriculated before and after 2010 (p=.05) for milestones prior to candidacy (milestones that both groups of students are likely to have completed). The year of matriculation explained 37% of the variance in the number of uses of the SJLOs for early program milestones (p=.02), while the respondent's identification with marginalized identity groups, satisfaction with doctoral training, or the degree of participation in the creation of the SJLOs were not significant predictors of number of uses. This data provides initial evidence that modifying the program forms that structure doctoral milestones may impact the routines of the organization in ways that further the infusion of a social justice framework into doctoral education. I have used it to support a complaint / grievance 00% 00%

Use of the Social Justice Learning Objectives
The matrix has NOT actively been used in my education 17% 00%

Average number of uses endorsed
Average number of total uses 3.67 4.60 Average number of early uses* 3.08 4.40 *Statistically significant difference between 2013 respondents who matriculated before (n=7) and after 2010 (n=6) at p=.05.
Number of early uses is predicted by year of matriculation (p=.02), explaining 37% of the variance, but not predicted by identification as a person belonging to marginalized identity groups, how satisfied the person purports to be with doctoral training generally, or the degree of participation the person had in creating the matrix.

a202
Respondents were further given the opportunity to explain how the SJLOs have been referenced or used for the purpose(s) they endorsed. Student responses convey uses of the SJLOs that are consistent with the ways in which teaching tools have been conceptualized to facilitate different types of social justice learning, including for critical thinking, action and social change, personal reflection, and awareness of multicultural group dynamics (Hackman, 2005). One student stated, "I have used the social justice matrix to remind myself at every step of my doctoral training that all of my work needs to incorporate social justice values." Another student reported, "Social Justice is consistently brought up and mentioned in meetings with staff and other students." A third student directly attributed his or her engagement with the SJLOs to the revised routines, reporting use of the matrix "with my advisor (mostly because [the] program learning plan specifically states that it should be used), in thinking about my courses and scholarship." Students have also reported using the SJLOs in ways that were not necessarily scripted or anticipated. For example, one student reported: I used it to advocate for opportunities that were not available. I used it to show first year students that they could get 'credit' for doing the work they thought was important. I used it to make sure I was adhering to it while developing my dissertation prospectus. I used it to inform … [input] in doctoral admissions. I used it when talking with a visiting scholar about how our program thinks about issues of social justice.
It appears that the revised forms have served to modify some programmatic routines, creating more room for students to conduct and celebrate their diverse visions for social justice work.

STAGE 5: RENEWING AND EXPANDING COMMITMENTS
As part of the process of renewing and expanding the work of the SJDE project, doctoral students responding to the 2013 survey were also asked the same questions as students in 2010: the extent to which the SJLOs were being facilitated and their current priorities for change. Surveying the student body a second time raises the possibility for inferential tests of mean differences over time as a way of monitoring progress. This must be done with caution, since so many threats to validity exist in pretest/ posttest designs, and the research design does not readily support the creation of generalizable knowledge. Although we could not authentically rule out alternative explanations (e.g., history, maturation) for any changes we observe, we did use independent t-tests to assess for significant differences between our two samples of responding students in regard to phase of the doctoral program, overall satisfaction with the doctoral program, extent to which they explain their satisfaction as related to issues of social justice, participation in the development of the SJLOs, and the extent to which the respondents reported belonging to marginalized identity groups. No significant differences between groups were detected.
The respondents in 2013 reported a similar pattern to that of students in 2010; the most actively supported objectives were in the scholarship domain (average across items; M=1.07; SD=.66). Items in the teaching domain were less actively supported (average across items; M=.73; SD=.85), and items in the service domain were least actively supported (average across items; M=.48; SD=.88). Although students in 2013 were not significantly more satisfied with doctoral education overall, the 2013 sample did report the SJLOs to be facilitated to a significantly greater extent in the research domain relative to the 2010 sample (p=.04). Since most of the priorities for change were in the research domain, and most of the modified routines concerned research milestones, this outcome makes logical sense. Statistically significant change was not detected in the teaching or service domain.
Next, we created a subsample of students who were advanced students or alumni when they participated in the survey in 2010, indicating that none of their doctoral education was likely to have occurred once programmatic routines had been modified (N=10). We created a second subsample of students who matriculated to the program after programmatic routines had been altered, and their entire experience of doctoral education was with the new routines (N=6). Although the samples are understandably small and may be biased for the sake of generalization, we found no significant differences in the samples along the aforementioned characteristics (e.g., satisfaction, belonging to marginalized identity groups) aside from their phase of doctoral training at the time of survey completion (p=.05; see Table 3). When these samples of students were compared, significant differences were detected in the average rating of the program's facilitation of the SJLOs in the domains of scholarship, teaching, and service over time (p=.01). Figure  2 displays statistically significant changes detected in SJLOs at the item level when comparing these non-overlapping sub-samples of students. SJLOs were reported to be more actively facilitated in 17 areas in 2013; have the same level of facilitation in 10 areas in 2010 and 2013; and no SJLOs were reported to be less actively facilitated (or more strongly obstructed) in 2013.In other words, those students who enrolled in the program after the adoption of the SJLOs and the revision of organizational routines reported more programmatic support for meeting the SJLOs than those who completed the program prior to the changes.   In chosen area of interest, understand dominant paradigms and critiques that center social justice across multiple levels of investigation, translation, and dissemination

1.22
Identify / articulate social justice goals & implications of individual research program and applications for the profession 1 1.47 * Demonstrate and apply critical inquiry into uses/misuses of research methods and articulation of just methodology 1.38 Seek out, identify, and work to enhance transformative potential of chosen research tools 1.33 Understand social justice implications and issues present throughout each stage of the research process 1.67

Teaching d =
Demonstrate a commitment to integrating diverse teaching/mentoring methods 1.60 Incorporate social justice content into instruction within teaching specialty 1.30 Gain and develop a working knowledge of positionality, biases, and beliefs which may influence teaching, mentoring, and/ or supervising

1.49
Create instructional spaces that are engaging, inclusive, responsive, liberatory, and non-oppressive 1.67

Service d =
Learn strategies for collegial and responsible engagement 1.14 "Supported" or "Facilitated" in 2010 -No Improvement Detected by 2013

Scholarship
Develop capacity to assess and communicate how social welfare research, policies, and practices can both empower and oppress communities they are purported to serve

CONCLUSION
Progress toward the social justice ideals of the social work profession may benefit from the infusion of a social justice framework into doctoral education. This paper described a process used to infuse a social justice framework into doctoral education at one School of Social Work in the United States. Although this organizational self-study and incremental change effort at one institution was not designed for the purpose of creating generalizable knowledge, what was learned may serve as a model for how an infusion approach to curriculum reform may be completed at the doctoral level through the shifting of programmatic routines. This paper illustrates ways in which doctoral program forms can be revised, doctoral students routines can evolve, and doctoral student perceptions of opportunities for social justice learning in doctoral education can shift between sequential doctoral student cohorts. It also calls attention to the perceptions of surveyed doctoral students that there are many ways in which they envision promoting justice as social welfare scholars, teachers, and public servants that are not actively facilitated through their doctoral education, but that these perceptions are dynamic and appear modifiable. It also suggests a need for further inquiry into the ways in which social justice and marginalized identities are related to student satisfaction with doctoral education.
Although the ordinal measures, pretest/posttest design, and uneven response rate to the survey has clear limitations, this exploratory study provides preliminary evidence of an overall increase in student perception of the program's facilitation of their social justice related learning since the programmatic adoption of the SJLOs and the subsequent infusion of these objectives into existing organizational rou-tines. Students who entered the doctoral program after the infusion effort report utilizing the SJLOs and perceive more programmatic facilitation of training goals related to social justice in the areas of scholarship, teaching, and service. Because the SJLOs were infused into existing program routines, it is plausible that these improvements can be sustained over time. Efforts will continue to modify routines at this institution and monitor student perceptions of opportunities for a social justice oriented education. To further understand how student engagement with the SJLOs has shifted the routines of doctoral education, textual analyses are planned to more directly analyze the ways in which the SJLOs are represented and used on program forms and through programmatic milestone completion.
The incremental change of organizational routines to infuse a social justice framework into doctoral education at one School of Social Work represents only one effort of the larger SJDE Project. Other ongoing efforts include a national survey of students in GADE-affiliated doctoral programs in the United States to determine the extent to which a) the SJLOs created at one institution resonate with a broader understanding of justice promoting scholarship, teaching, and service among doctoral students; b) students feel prepared and intend to promote justice through scholarship, teaching and service; and c) the SJLOs are facilitated by diverse institutions providing doctoral education in social work and social welfare. Finally, the SJDE Project is also conducting focus groups to collect innovative strategies for infusing social justice into doctoral programs as a means to spread successful ideas widely for the sake of advancing doctoral education and the social justice ideals of social work.

The doctoral program used quarterly
Town Hall Meetings to create spaces for doctoral program constituents to gather, build community, and exchange ideas. The SJDE project requested time at these Town Hall Meetings to further the project at key moments in the process.