Measuring the Return on Investment for AFRICOM’s African Enlisted Soldier Development Efforts III

,


Background
Africa is a land of tremendous challenge and promise.It holds some of the nations with the highest density of people living below the poverty line (Katayama & Divyanshi, 2019); it is home to thousands of different ethnic groups and languages comprising a diverse array of cultures; and it contains unlimited economic potential, if supported correctly.Recognizing these facts, the United States Africa Command (AFRICOM) asserts American influence in Africa by developing the human capital contained within its partner nations' non-commissioned officer (NCO) corps.Specifically, through the application of its Africa Enlisted Development Strategy (AEDS), AFRICOM aims to increase the individual and collective competence of these NCO corps, thereby solidifying the organizational layer between partner nations' officers and enlisted soldiers (United States Africa Command, 2018).If successful, greater stability can be developed within a nation's military via job pathways for unemployed or disgruntled youths.This newfound job security helps to limit the influence of Violent Extremist Organizations within the continent, further developing African stability and advancing American interests and influence (United States Africa Command, 2018).
Proceedings of the Annual General Donald R. Keith Memorial Conference West Point, New York, USA April 28, 2022 ISBN: 97819384962-2-6 309 A Regional Conference of the Society for Industrial and Systems Engineering

The EDML, EDR, AIT, and Customer Need
This project is the cumulative effort of three years of capstone work in support of AFRICOM by the United States Military Academy's Department of Systems Engineering (DSE).The 2019 effort concluded with the development of the Enlisted Development Maturity Level (EDML) scale, as shown in the left panel of Figure 1.Enlisted Development Review (center), and AEDS Investment Tracker (right) This scale seeks to quantify the development of an African country's NCOs over time by assessing how the county develops, empowers, leverages, and trusts them (Bochonok et al., 2020).Despite the gains made towards the assessment of NCO competency based on AEDS-related training during Academic Year (AY) 2020, there remained a lack of clarity on how AFRICOM would be able to assess investments over time.AY 2021's capstone group designed a system to measure the return on investment of AFRICOM's efforts through the development of the AEDS Investment Tracker (AIT) and the Enlisted Development Review (EDR).As shown in Figure 1's center and right panels, the AIT measures monetary, time, and manpower investments, whereas the EDR assists in capturing the development of a country's NCOs across six core competencies (Bender, et al., 2021).
These measures combine to give AFRICOM a comprehensive understanding of the development of a partner nation's NCO corps.The EDML acts at the organizational level, assessing whether a nation's NCO corps is employed to its full potential.The EDR works at an individual level, determining the competence of a nation's NCOs.The AIT measures the total investment AFRICOM applied towards specific training events.Collectively, these measures can be used to assess the return on investment (ROI) of AFRICOM's AEDS-related efforts.
Despite the progress made in previous years, there still exists a gap in the overall implementation and synthesis of these measures.Notably, due their novelty, the EDML, AIT, and EDR are not currently in use by AFRICOM; therefore, realworld data does not currently exist.This year, a three-pronged approach was used to facilitate the implementation of these measures within AFRICOM: (1) ensuring the quality of the data captured by the EDML, AIT, and EDR, (2) improving the statistical analysis and synthesis of EDML, AIT, and EDR data, and (3) encouraging the organizational adoption of AEDS assessment.Ultimately, the integration and implementation of these measures into AFRICOM's current evaluation and Coaching -During this event's real-world, competitive settings where the focus was on organizational winning, the NCOs I observed were able to develop skills, tasks, and specific behaviors in subordinates and oversee improvements in these areas.
3. What African countries were involved in this event?
4. In what country did this event take place? 2 Country understands the value of developing NCOs and sends them to developmental programs, but it does not leverage them effectively.
Mentoring -During this event's relaxed, one-on-one settings where the focus was on the subordinate's selfdevelopment, the NCOs I observed were able to establish voluntary developmental relationships and provide holistic guidance on personal and professional growth.

How would you characterize this event?
3 Country empowers and leverages NCOs at the tactical level, but they are not trusted enough to provide input on tactical decisions.

Not Applicable / Not Observed
6c.Approximately how many AFRICOM personnel were involved in this event's planning and execution?4 Country empowers and leverages NCOs at the operational level, but they are not trusted enough to provide input on operational decisions.
6b.Approximately how many AFRICOM person-hours were invested in this event's planning and execution, including set-up, tear-down, etc.? Empowerment -During this event, the NCOs I observed were able to encourage subordinates to think, behave, decide, and act on their own.5 Country empowers and leverages NCOs at the strategic level, but they are not trusted enough to provide input on strategic decisions.
Force Development -During this event, the NCOs I observed in managerial roles were able to plan, resource, and execute training effectively and efficiently.
7. Approximately how many African servicemembers were involved in this event?6 Country's NCOs are empowered, leveraged and trusted in a manner similar to the U.S.; NCOs are the backbone of their country's military, and they are trusted to provide input on strategic decisions.

Comments (Optional):
Provide any additional comments regarding this event or your interactions with the NCOs.

Strongly Disagree
Proceedings of the Annual General Donald R. Keith Memorial Conference West Point, New York, USA April 28, 2022 ISBN: 97819384962-2-6 310 A Regional Conference of the Society for Industrial and Systems Engineering assessment processes should allow decision makers to judiciously allocate limited resources to the development of partner nations' NCOs.

Methodology
To create a solution that provides value for AFRICOM, the Systems Decision Process (SDP) was utilized.The SDP is a four-phase process that applies to major systems decisions at any stage in their life cycle.It is collaborative, iterative, and value-based (Parnell et al., 2011).The SDP starts with Problem Definition, which results in a redefined problem statement.Following this, work begins within Solution Design, where the key processes and products revolve around creating a slate of feasible alternatives.Next, the project enters Decision Making, and the alternative that best meets the decision maker's needs, wants, and desires is selected as the solution.Upon making this decision, the project enters Solution Implementation, and the solution is put into practice, monitored, and controlled.Feedback received from the solution then drives future improvement, which emphasizes a central tenet of the SDP -solutions live within, and therefore must evolve with, their environments.

Problem Definition
It is essential to correctly identify the problem prior to initiating the process of solving it.Accordingly, to begin the Problem Definition phase of the SDP, it was imperative to conduct literature reviews on the following topics: making sense of the EDR, populating the AIT, transforming AEDS-related metrics into ROI, collecting quality data, and integrating AEDSrelated insights into AFRICOM's assessment and planning framework.These literature reviews generated several key insights.
Regarding the EDR and EDML, it is essential to note that these measures utilize a Likert scale, which is an ordinallike scale that measures the degree to which respondents agree with a particular statement (Artino & Sullivan, 2013).This type of data can be aggregated to draw conclusions using descriptive statistics such as the sample mean from individuals' ratings towards a subject (Harpe, 2015).Regarding the AIT, using a data source containing the three required components of the AIT -time, personnel, and money -allows for sufficient analysis to be completed.Some data sources with this information currently exist, such as the DoD Foreign Military Training Report, which reports "all military training provided to foreign military personnel" (Department of State & Department of Defense, 2020).Once data exists for these metrics, ROI can be calculated using a linear regression model in conjunction with a preliminary time series analysis, thereby allowing for future predictions (Parzen, 1961).To minimize the subjectivity of EDML, EDR, and AIT data, it is crucial that the surveys' design balances objectivity and flexibility.Additionally, it is also essential to emphasize buy-in at the lowest level for those filling out the surveys.Finally, each nation will have a unique approach for implementation; thus, it is beneficial to focus initial implementation efforts on one nation.After communicating this insight to AFRICOM, Botswana was identified as the best choice by the Command Senior Enlisted Leader (CSEL) -Sergeant Major Richard Thresher.
Following the literature reviews, functional and value analyses were conducted to identify the key objectives that the system must accomplish.This begins by identifying the fundamental objective, which is to quantitatively measure the effectiveness of AFRICOM's AEDS.One key tool used to craft the fundamental objective was the findings, conclusions, and recommendations (FCR) matrix, which provides insights grounded in stakeholder feedback.Based on the FCR matrix, three key functions were developed to accomplish the fundamental objective.First is Function 1.0 -Enhancing the data quality of the EDML, EDR, and the AIT -as shown on the far left of Figure 2.This function ensures that AEDS-related events are quantitatively assessed in a way that generates valid input data.Function 2.0 involves improving ROI analysis, which is only valuable to AFRICOM if it is easily interpreted and useful for making decisions regarding the AEDS.Finally, there is Function 3.0 -Facilitating the adoption of AEDS assessment within AFRICOM.To accomplish this, it is crucial for the AFRICOM staff to both understand and appreciate the proposed tools of the system.Following the determination of the system's functions, appropriate objectives were crafted and nested within their respective functions.For instance, the key objectives of Function 1.0 were identified as maximizing the reliability of the input data and minimizing the number of input errors.
After determining objectives for each function, value measures were created to objectively assess how well potential solutions satisfied the objectives.These value measures are color-coded.Light blue indicates an ideal value measure.However, there are some resource limitations associated with certain value measures.For instance, attempting to measure the inter-and intra-reliability statistics of the input data requires that the input data exists; it currently does not.Therefore, alternative scales, such as the perceived effectiveness of the training mechanism, were developed for some of the functions.The realistic value measures to be used to assess each alternative's effectiveness are colored dark blue.Light grey value measures follow a single course of action -the time taken within the selected pilot country, Botswana.Finally, the dark grey value measure for system usability will follow a spiral development methodology, where the usability of a future applet will be continuously assessed and improved.
Proceedings of the Annual General Donald R. Keith Memorial Conference West Point, New York, USA April 28, 2022 ISBN: 97819384962-2-6 311 A Regional Conference of the Society for Industrial and Systems Engineering

Figure 2. Functional and value hierarchy
To produce an effective value model for the stakeholder, the realistic value measures crafted within Figure 2 needed to be ranked in order of importance.Based on recent feedback from AFRICOM, their priority is as follows (from most to least important): perceived effectiveness of the training mechanism, interpretability of ROI analysis, AFRICOM staff impression, perceived effectiveness of QAQC mechanism, and time taken to train an average user.
At the conclusion of the key tasks within the Problem Definition phase, the following revised problem statement was crafted and subsequently approved by AFRICOM: Develop a user-friendly system that improves the data quality and analysis of the EDML, EDR, and AIT to maximize the ROI of AFRICOM/s AEDS-related efforts.

Solution Design
The goal of the Solution Design phase of the SDP is to design alternatives that accomplish the system's functions and meet its objectives.To create a slate of potential alternatives, Zwicky's morphological box was utilized.This methodology allows for the generation of alternatives by nesting design parameters within a value hierarchy's objectives.For this effort, design parameters were created for the following objectives in Figure 2: 1.1 -Maximize the reliability of input data, 1.2 -Minimize the number of input errors, 2.1 -Maximize the utility of the statistical measures used, and 2.2 -Maximize the synthesis of the EDML, EDR, and AIT.
There are multiple design parameters associated with Objective 1.1.These design parameters relate to how AFRICOM should conduct EDML, EDR, and AIT user training.The parameters are the training cohort, modality of training, frequency of training, method of training, and training validation method.Specifically, the training cohort can be a group (collective) or an individual, and the modality of training can be in-person or virtual (remote).The frequency of training ranges from a single occurrence to the more ambitious "prior to every AEDS-related training event," while the method of this training can be interactive or lecture style delivered by a human instructor or a computer program.Lastly, the training validation method can be through a test or by completion, where completion can be formally tracked or on one's honor.The design parameters for Objective 1.2 relate to conducting quality assurance and quality control (QAQC) on the input data.This is important as it should eliminate obvious input errors, thereby allowing AFRICOM to accurately assess the efficacy of AEDS-related training events in a certain country.The first design parameter is the method of error identification and confirmation, which will be conducted by a human or an algorithm.The QAQC approach analyzes the data via screening (i.e., before entry into a database) or auditing (i.e., after entry into a database) (Cross & Newman-Gonchar, 2008), and the percentage of the data analyzed can range from as low as 5% to as high as 100%.Additionally, the design incorporates a QAQC method of verification, where bad data points can be compared against the current submissions from a particular event, against all submissions from a certain country, or against a set criterion.Finally, if erroneous data points are found, solutions must incorporate a QAQC method of adjudication, where a decision-maker would either investigate, accept, or discard that data.The final two design parameters -analysis type and synthesis of metrics -relate to the objectives of maximizing the utility of statistical measures and maximizing the synthesis of the EDML, EDR, and AIT.Collectively, these design parameters set the foundation for building alternatives that improve the data quality and analysis of the EDML, EDR, and AIT.
As seen in Figure 3, four alternatives were developed.The alternatives are based on the number of resources required to implement them, ranging from "Resource Intensive" on the high side to "Completion" on the low end.For instance, consider the "Resource Intensive" option.Here, training would be conducted in-person as a collective group with a human instructor.This training would occur prior to every AEDS-related event and be validated using a test.Minimizing the number of input errors in this alternative would be accomplished through human identification and confirmation, where the input data of each survey would be compared to the other surveys completed for the same event.If potentially erroneous data was discovered, AFRICOM personnel would subsequently investigate the situation to determine whether the errors were legitimate, and they would correct them as necessary.Finally, the statistical analysis for this alternative would be descriptive, inferential, and communicated using a storyboard.The results would then be integrated into an ROI equation.

Decision Making
The next phase of the SDP will analyze the slate of alternatives to identify the best option for AFRICOM.First, raw data pertaining to the value measures seen in Figure 2 must be collected.To do so, communication with the system's key stakeholders is crucial.This will be done by presenting the stakeholders with prototypes of each alternative and gathering feedback on their performance.To this end, a digital survey was created in Microsoft Forms, and the raw data collected from Proceedings of the Annual General Donald R. Keith Memorial Conference West Point, New York, USA April 28, 2022 ISBN: 97819384962-2-6 313 A Regional Conference of the Society for Industrial and Systems Engineering it will feed the value functions, be weighted according to AFRICOM's priorities, and ultimately combined into a final value score.Using this methodology, the alternative with the highest value score will become the recommended solution.

Conclusion
As AFRICOM continues to foster relationships with its partners, it is essential to improve the data quality and analysis of the EDML, EDR, and AIT to inform decision making and maximize the ROI of AEDS-related efforts.Building on the work of previous capstone teams, a value model was built, alternatives were developed, and Botswana was identified as a pilot country.Although real-world data is currently lacking, the fundamental objective still stands.Accordingly, future effort will be focused on delivering a final recommendation to AFRICOM and facilitating its implementation in Botswana, thereby setting the foundation for a process that can be applied in other African states.

Figure 1 .
Figure 1.The Enlisted Development Maturity Level (left), Enlisted Development Review (center), and AEDS Investment Tracker (right) Figure 3. Design parameters and resulting alternatives