The Assessment of Behavioural Attributes and Mind-set of Participants in Government Workforce Innovation and Entrepreneurship Programs: A Feasibility Study

Copyright: © 2018 Aragon R, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. The Assessment of Behavioural Attributes and Mind-set of Participants in Government Workforce Innovation and Entrepreneurship Programs: A Feasibility Study


Introduction
In the recent history of institutional innovation and entrepreneurship programs, stakeholders from business, academia, and government have shown interest in developing a means of assessing the impact and outcomes of such programs on productivity and return on investment [1,2]. The foundations of entrepreneurship in promoting economic vitality have rested in competition and personal attitudes toward self-initiative and the creation of new ventures [3][4][5][6]. Today, there is widespread interest among large institutions to enhance their innovation capacity through structured programs that engage their employees in ideation, testing of new concepts, and scaling of successful pilot implementations [7]. Government and not-for-profit civic organizations have joined the ranks of institutions that seek ways to improve performance in service to their missions by enhancing the entrepreneurship capabilities of their respective workforces [8][9][10]. In recent years, the US federal government has increased its focus on creating innovation programs to internally enhance the ability of government programs to achieve their mission [11,12]. These innovation-based programs, however, do not yet demonstrate the ability to systematically review program achievement with intermediary and primary endpoints that illustrate clear returns or impacts such as greater problem-solving skills among employees [13,14]. Thus, subjective surveys of participant experience are often used to achieve this goal, and less commonly, the longterm success of projects or the career trajectories of participants are studied [15,16]. Additionally, Kuratko et al. [17] established a conceptual model of entrepreneurship behavior among corporate middle-level managers and the association with corporate entrepreneurship that has proven valuable for research applications.
Structured mentoring programs enable employees to achieve their goals more effectively by proposing, testing, scaling, and disseminating ideas and solutions that support mission-related activities across the organization (i.e., cultivating and propagating a culture of innovation, especially in instances where highly innovative ideas might carry a concurrent degree of risk). As such, the Lab was formally established in 2012 within the administrative offices of the HHS Secretary, where its activities were aligned with strategic initiatives such as establishing a programmatic focus that could enhance employee skills in ideation and problem solving. The premise of such an initiative was an underlying recognition by executive leadership that the HHS workforce possesses an untapped potential of ideas and capabilities, which, if harnessed and cultivated appropriately, could be channelled toward more effective support of HHS' mission. Thus, an overt outreach and communications effort was designed and implemented to heighten interest in programs and opportunities that involve testing new ideas and creating a culture that encourages innovation as a core value. For the purposes of this document, the culture of innovation is defined as "An accepted attitude or mind-set of an organization that recognizes innovation as a novel and discontinuously different product, service, process, organizational structure, or business model that adds substantive value and its origin is based in a different way of seeing, understanding and thinking about something in the world" [19]. The study described herein was conducted as one component of a broader series of internal evaluations aimed at determining the effectiveness of entrepreneurship training and cultural adaptation toward balancing risk and innovation. A novelty of this particular study is the application and use of behavioural rather than traditional programmatic or organizational metrics and parameters (e.g., specific outputs or deliverables).
The rationale for this study was based on several factors. First, over the course of the IDEA Lab's existence, managers had noted the differential expression of attitudes toward entrepreneurship and innovation among HHS employees (i.e., some employees embraced it, some did not). Knowledge of staff behaviour and interests in innovation creates an opportunity for organizational leaders and managers to explore how better to develop a culture of entrepreneurship within the agency [20]. Second, an annual survey of HHS employees revealed an upward trend in these attitudes over time, suggesting a potential heightened interest in innovation methods among those employees. Third, there is now an extensive body of literature on the role of personality and psych behavioural attitudes and assessment methods in shaping entrepreneurship impacts. For example, the Adjective Check List of psychosocial characteristics has been used to assess goal-setting, job satisfaction, and problem solving among scientists and engineers [21]. The Environmental Behaviour Inventory, another approach to assessing psychosocial perspectives of employees, was created based on corporate entrepreneurial scenarios and incidents [22]. In addition, Hornsby et al. [23] developed the Corporate Entrepreneurship Assessment Instrument to help managers and leaders measure the internal environmental factors of management support, work discretion and autonomy, rewards and reinforcement, time availability, and organizational boundaries. This employeedirected survey on environmental factors has been validated for use in government innovation settings [24].
Among the first systematic frameworks to study the work on innovation and entrepreneurship in government was that established by Schneider et al. [25]. Their foundational work examined the basis for social, political, economic, and technical influences on publicsector entrepreneurship. Additional studies have shown the beneficial effects of entrepreneurial training and coaching on entrepreneurial career interests [26][27][28][29], entrepreneurial cognition and opportunity recognition [30,31], and new venture success [32]. Further, assessments of employee attitudes toward creativity and innovation, using multifactorial surveys of organizational climate, have been used to test and propose new employee interventions [33]. Historically, the research and literature on innovation and entrepreneurship have been focused on the area of economics, but have increasingly diversified to communications, anthropology, psychology, sociology, and (more recently) administrative and management science [34,35]. Other research has demonstrated that personality traits can distinguish entrepreneurs in several areas, such as entrepreneurs scoring higher on conscientiousness and openness to experience and lower on neuroticism and agreeableness. What can be perceived as predisposition toward entrepreneurial behaviours has been found to relate to individuals' prior knowledge and exposure to entrepreneurial opportunities, thereby making them more likely to recognize future opportunity and use those experiences to enhance their entrepreneurial performance [36]. Taken together, these factors both empowered and enabled the IDEA Lab to explore the potential measurement of behavioural attitudes and mind-sets of participants in its programs to assist in enhancing specific characteristics and competencies such as self-confidence, creativity, achievement, and coach ability or mentoring.
Building upon the above-described research and data, the study described herein uses behavioural assessments to achieve three purposes. First, such assessments can help researchers ascertain the feasibility of assessing and comparing the entrepreneurial traits and attitudes among government employees engaged in an ideation and early-stage prototyping program known as the Ignite Accelerator. Second, they support development of baseline data on the entrepreneurial characteristics of innovators external to the federal government but nonetheless recruited to participate in internal Entrepreneur-in-Residence (EIR) and Innovator-in-Residence (IIR) programs 1 . Finally, they provide novel insights into how the data can be used or leveraged by managers, coaches, and mentors to design, target, or adjust specific program offerings toward the attitudes and mind-sets of intended beneficiaries such as (in this case) federal employees.

Features of Government Innovation Programs
There has been a growing interest in the development of measurement methodology to assess the impact of innovation programs in government agencies [18]. Government programs that foster innovation in mission-oriented activities are motivated by several factors. Among them are rapid, sometimes unpredictable changes in mission that require the need for new problem-solving approaches; the need to adapt business tools into practice to support expanded government services such as the increased utilization of data and business process automation; and expressed desires from executive leaders and Congressional authorities to render improved performance in operations.
Measurement of innovation activities in the private sector generally consists of global assessment measures that can be determined by parameters such as return on investment for the sale of goods and services, productivity in monetary terms (e.g., revenue generation), and customer satisfaction [20]. Such traditional measures, however, are not necessarily transferable to government and non-profit/publicsector organizations or environments because of the lack of such things as market forces, consumer marketplace sales, and traditional financial return on investment (i.e., profit motive) [37]. These differences highlight the need to develop and adapt enhanced assessment tools, methodologies, and metrics to measure the impact of innovation programs in public-sector environments (e.g., government).

About the programs in the study
Two innovation programs that are based within a large federal agency and that foster projects aimed at improving the operational outcomes of government programs and administrative functions were the focus of this study. The first program, known as the Ignite Accelerator focuses on prototyping, short-term (3 months) ideation and establishing proof of principle. The second program, known collectively as the Entrepreneur-in-Residence and Innovator-in-Residence (EIR+IIR) programs, encompasses larger scale, longer term (1-4 years) projects aimed at developing and deploying innovative solutions to problems deemed highly significant by the agency. 2 Projects encompassed by these two programs cover a wide range of topics from healthcare delivery to integration of information technology systems. The projects pursued through the auspices of these two programs utilize an ad hoc team approach to problem solving, which has been previously demonstrated to have greater impact on organizational innovation than individuals working alone [38]. Examples of specific projects from the Ignite Accelerator include those aimed at using natural language processing for government grant applications; enhancing the representation of disadvantaged secondary school students in science, technology, engineering, and mathematics (STEM); increasing the amount of data available for the review of new healthcare devices; and improving workflow processes related to the training of agency staff. Similarly, specific examples of outputs from the EIR+IIR program include the design and development of enhanced information technologies to support the logistics of the national organ donation procurement system, creation of technology to support the registration of public assistance programs, and design of a national public service aimed at improving prevention services.

Ignite Accelerator
The HHS Ignite Accelerator program was established to provide a mentored, learning experience in ideation and start up methodologies, with the goal of achieving a minimum viable product (MVP). The program provides a structured learning environment focused on developing knowledge around business stakeholders to create a usercentric solution using "lean start up" principles [39]. Because the program focuses on leveraging innovation to test probable solutions to acute organization problems, stakeholders can include colleagues, organizational leadership, and/or those in the ecosystem that are most prevalently or frequently affected by the problem. A key driver of the Ignite Accelerator's design was the desire to harness entrepreneurial spirit from within the organization (i.e., from entrepreneurs) to solve their problems and their colleagues' problems. This method is supported strongly throughout multiple service industries [40] and is likely to apply equally well to government innovation settings. Thus, competitions are held at multiple points throughout the year to select project teams that receive administrative structure (i.e., programmatic, quantitative milestones), administrative sponsorship (i.e., shared risk-bearing), training in needs assessment, access to internal networks and business tools and technologies, and experience with entrepreneurship management methodologies.

HHS IDEA Lab.
Candidate teams are formed in response to an internal call for applications to participate in the program. All projects considered by this program are early stage ideas to articulate, test, and establish proof of concept for specific new solutions, with most having no prior data or previous testing. Peers and mentors judge the applications, and all project teams selected for participation gather for a 3-day intensive course that focuses on community building and methods in lean start up and design. Throughout the following 3 months, the teams receive intensive coaching, facilitation, and instruction to enhance skills in storytelling, project scope definition, identification of stakeholders, and examination of other strategic elements of the problem under investigation. Coaches, who otherwise serve in professional innovation capacities (e.g., teaching innovative methods at local universities), interact regularly with teams through weekly status meetings, resource documents, and monthly team calls to share experiences. At the completion of the development period (3-4 months), the groups reconvene on an HHS Innovation Day to demonstrate the results of their individual projects in a format similar to the show "Shark Tank s ". 3 At this event, participants receive directed feedback from an expert panel of individuals familiar with or sometimes overseeing the organization in which the problem exists, as well as entrepreneurial experts and other ranking members of the HHS community. Participants can receive further support for the scaling and deployment of solutions judged to be at least minimally viable. Participants in successful projects may elect to compete for additional resources through a companion funding opportunity (known as "Ventures") that is focused on the further refinement, scaling, and eventual dissemination of the solution through multiple stakeholder agencies. Examples of past Ignite Accelerator projects that subsequently received longer-term venture support focused on the use of algorithms to estimate the prevalence of autism derived from medical records data extraction, a business framework for the economic return or secondary gain from investments in public health emergency resources, and open-source data and tools for sharing three-dimensional printable models related to biomedical science.
Overall, more than 100 project teams and more than 350 participants have participated in the Ignite Accelerator since its inception. A 2016 evaluation of the program outcomes indicated that approximately one-third of projects succeeded and progressed to further development and implementation, while another one-third was deemed successful but required further development before full-scale deployment. The remaining projects were deemed unsuccessful, a not surprising result given the objectives and higher-risk nature of the Ignite Accelerator [41].
For this feasibility study, 10 Ignite Accelerator participants voluntarily agreed to participate in a self-assessment of behaviour and mind-set attitudes before the start of their project and again at program completion. Of these, two individuals did not complete the post-program completion assessments, and one did not complete the program.

Entrepreneur and Innovator in Residence (EIR+IIR) Program
As a second component of the HHS IDEA Lab, the EIR and IIR program addresses the HHS innovation agenda by identifying external (i.e., outside government) talent needed to help internal (i.e., within government) projects through a specialized hiring process aimed at attracting highly innovative and entrepreneurial candidates to assist in high-visibility, solution-oriented government projects. Compared to the Ignite Accelerator projects, EIR+IIR projects are much more developmental (versus experimental) and delivery focused. Participants attend monthly meetings and maintain regular direct interaction with agency senior leadership. The EIR+IIR program consists of key systemic (versus acute) organizational problems that require solutions based on highly specialized and entrepreneurial insights obtained from talent outside the government (e.g., private sector). Through a specialized hiring authority for selection, entrepreneurs were on-boarded and assigned to specific projects derived from various HHS agencies in need of new approaches to solve historical and systemic problems. Each project underwent peer review and was approved on the basis that the problems and proposed solutions were determined to be sufficiently meritorious to warrant support under this program.
Since 2012, 70 teams from across HHS have expressed an interest in participating in the EIR+IIR program, and 55 have submitted applications. Applications are evaluated by technical reviewers for feasibility and innovation potential before undergoing final selection by program leadership. As of November 2016, the EIR+IIR program has hired 24 external entrepreneurs and innovators to work with 56 federal career staff on 15 high-priority projects across HHS. For this feasibility study, all 24 EIR+IIR participants voluntarily agreed to participate in a self-assessment of behaviour and mind-set attitudes before the start of their projects.
To assess the impact of the above-described programs (Ignite Accelerator and EIR+IIR) on HHS' organizational objectives and workforce culture, longitudinal data on workplace performance of participants in either program were curated and collectively analysed. The analytic framework employed herein examines the perspectives and attitudes of employees who participated in either program. The methods applied here could serve as an avenue for assessing the impact of innovation programs within a federal organization, as measured by and through the perspectives of its own employees. Thus, such data may be particularly useful for senior executives, leaders, and managers within a federal agency.

Objectives
This study was undertaken to evaluate an approach to augment the knowledge base about workplace participants in two representative innovation programs within HHS. The information was sought to demonstrate the behavioural impact(s) of these programs on the employees participating in them. Sponsors of participating employees often looked for objective evidence to support the notion that innovation training resulted in the professional growth of their staff. Similarly, mentors and coaches sought more information about the needs and attitudes of employees to assist in coaching and to improve aspects of cohesiveness and success. Together, these two program experiences serve as a testbed for improved understanding of approaches to enhance program performance and impact through the use of participant data. Further, as each program evolves, these data can guide programmatic design to enhance the ability to both capture and modify or "individualize" innovation and entrepreneurship skills and talents to fit specific workplace needs and situations.
Specifically, this study aims to test and provide proof of principle for the measurement of government employee entrepreneurial and innovative behaviour and competencies throughout their participation in innovation programs, by answering the following questions: How do participants' propensities toward entrepreneurial and innovative behaviours change from the start to completion of a government-led innovation program, and are the results measurable and repeatable?
Can a baseline population of internal entrepreneurs be created against which Ignite Accelerator participants' entrepreneurial behaviours can be compared, and does demonstrable similarity exist between the two cohorts?
Can government innovation programs administrators create a culture that allows for measurement and ranking of participants' performance in a way that enables comparison and validation against a population of known entrepreneurs?

Design
The study used a nonrandomized, observational design conducted with participants in the Ignite Accelerator, with the results blinded from program managers, employee supervisors, and other program participants. The population included past program participants (n=16) and, separately, a cohort (n=7) evaluated prior to the start and at the completion of the 3-month Ignite Accelerator experience. A comparative population of EIR and IIR program participants also volunteered to engage in the assessment (n=24).

Behavioural assessment
Extensive literature has documented the influence of past entrepreneurial and innovation experience and exposure on individuals' behaviours. The measurement of the behaviour, in terms of what should be measured and how, is currently less researched and therefore less established [42]. Psychobehavioral and value constructs, facets, and typologies were considered during study implementation including all present valuable research in understanding segments of human innovative and entrepreneurial propensities [43][44][45]. Collectively, the independent value-based research work. Evaluated the factors that lead to certain behaviour. These research findings explored the "deep beliefs" and histories that drive individuals' actions. The study discussed herein attempts to use those foundations to assess current behaviours, as a reflection on the research of Adams et al. [42] De Jong and Den Hartog [43]. Analysing entrepreneurial and innovative aspects of individuals in this way is considered a dissection between what is important to an individual (i.e., values) and what actions the individual takes (i.e., behaviours). In establishing this study, an assessment tool was sought that could realize measurement in parallel with the conduct of the innovation programs.
This study's assessment tool (PAIRIN; Denver, CO, USA) was developed as a form of the Adjective Checklist [46,47] and is widely used in entrepreneurial education. In this Adjective Checklist (ACL) assessment, participants are presented with a list of 300 adjectives, and asked to select the adjectives that describe how they perceive themselves to be most of the time. Participants can either select an adjective or not. Each adjective feature is then grouped into 1 or more of 37 original scales. A scale is a label that describes the adjectives that comprise it (e.g., Achievement, Self-Control). The commercial platform represents an expansion of the original scales and consists of 94 total attributes. An example of an attribute is "Resiliency," which consists of 75 adjectives. The adjectives of which "Resiliency" consists are either indicators or contra-indicators, which are used to determine the intensity of the attribute. Scores are derived from the number of adjectives influencing each attribute for which the participant selects a positive response. Those scores are then used to establish the individual's propensity to possess the various attributes. Individuals' scores across all scales and attributes are assessed for validity and given a validity score (i.e., potential for random or very negative answers).
In designing this study, considerations were made for various assessments before selecting the PAIRIN tool. The most important factor in tool selection was the ability to and likelihood of participants altering their responses for specific outcomes. Concerns have been raised of the Adjective Checklist and its validity [48]. Within the concern of validity, there were two important distinctions for consideration: (1) the intention of participants to alter their responses for specific outcomes based on perceived reaction to their scores and, (2) participants' ability to alter their responses in such a way that would allow them to achieve a desired outcome. With regard to participants' intention, it is known that individuals have, "'Enhanced' their scores in an attempt to look good," [49]. However, this analysis indicated that the majority of applicants in that study were honest. It is also noted that the subjects of the study discussed herein did not constitute job applicants or seekers, but rather were all fully and gainfully employed by the United States federal government. Each subject was informed that the questionnaires were being administered in order to acquire data that would influence the focus of the assistance and coaching provided to them and to future participants of the innovation programs described in our manuscript. Additionally, the supervisors themselves were not involved in any part of the administration of the programs and were neither queried about nor provided expected outcomes. To the second consideration of validity, the ability to alter responses for a desired outcome, it is believed that the Adjective Checklist and the PAIRIN tool's enhancement of the ACL have rendered the possibility near none. The indicator and contra-indicator status of any given adjective can be reversed across scales. One adjective's value within a given scale could be reversed in another scale, making it near impossible to fake the results. Also, the ACL has very low face validity. The complexity of the scoring for the ACL makes it nearly statistically impossible to detect how each adjective would impact a given outcome. While this argument renders the likelihood of participants intentionally altering responses as low, the PAIRIN tool has implemented algorithmic checks that alert analysts of possible degradation of validity. Seven checks, and subsequent system flags, indicate too few or too many adjectives selected and erratic, or random answering. Another two checks indicate if responses are too positive or too negative. All of the mentioned checks create a reliability score of up to 100%, and for this study, no data were used from participants with lower than 100 % validity.
The 94 attributes used in the assessment were consistent across all of the groups evaluated. Data from the group responses were analysed to determine differences between their pre-and post-participation scores and were compared against other populations to determine similarity and variance. Of the 94 attributes in the assessment tool, a portion of the study focused on a subset of 20 that comprise an "Entrepreneurial Spirit Target," which is based on a population of identified entrepreneurial individuals from a given field (Table 1) [50].
One aspect of the behavioural assessment tool output, a "match score," is determined by comparing individual participants' scores to the Entrepreneurial Spirit Target range. For each of the 20 attributes, a target range is established based on a sample size of more than 350 successful entrepreneurial individuals whose mean score for each attribute was either elevated or subdued as compared to a general population. The match score is based on an index determined by the degree to which a study participant's raw score fell in or out of the target range for each attribute. The index is determined by the sum of the number of attribute scores that fall within the target, and the numerical differences from the target. For this study, the match score is used to determine how closely an individual aligns with the individual attributes and ranges of the targets and to provide a hierarchy of which individuals match the target more than others.
The assessment tool's Entrepreneurial Spirit Target was used only for the match score analysis, and an additional target range was constructed using the assessment scores derived from the EIR+IIR cohort (EIR+IIR Entrepreneurial Target). This target range was intended to serve as an internal entrepreneur comparator for the study environment. For each of the 94 attributes, the minimum and maximum EIR+IIR raw scores were used to define the boundaries of the target range, and the average scores were used to define the highestranking attributes.

Population information
The participants for the Ignite Accelerator and the EIR+IIR programs were drawn equitably from employees across 12 agencies in HHS. Agencies with social service missions were represented as well as those with science and technology missions when adjusted for numbers of employees. A demographic summary of the population of employees indicates a broad range of responsibilities, with more than 300 skill types. The demographic characteristics of the workforce   cohorts described in this study were not significantly different from the workforce population. The sample used for this study is representative of the HHS workforce (Table 2).

Analysis performed on ignite accelerator participants' behavioural assessments
The analysis included a comparison of pre-and post-participation scores to assess any influences of the coaching and mentoring provided during the Ignite Accelerator program period. Pre-and post-participation assessment scores were compared to the EIR+IIR Entrepreneurial Target. At the completion of the cohort's mentoring program, Ignite Accelerator program directors ranked the participants based on their performance in the exercises and fulfilment of the requirements to create an MVP. That ranking was then compared to the match score to understand better the perception of the program's performance.

Data Analysis and Results
The Ignite Accelerator program participants' pre-and postprogram assessment mean raw scores as represented by the attributes of the Entrepreneurial Spirit Target. The degree of change between average scores in pre-and post-Ignite Accelerator is represented by boxes, in descending order from left to right, with the furthermost left representing the most positive movement, and the furthermost right representing the most negative movement. Entrepreneurial Spirit Target attributes are represented on the X-axis, with average scores on the Y-axis. Attributes with negative movement indicating a lower average score at post than previously scored in the pre-Ignite phase are represented below the 0-line (Figure 1).
The second analysis compared the Ignite pre-and post-scores to the EIR+IIR Entrepreneurial Target. The concept of using the EIR+IIR population to establish a separate reference value set is based on one element of our evaluation methods to understand environmental influences of innovation culture. For the Ignite Accelerator cohort, the minimum and maximum EIR+IIR population raw scores were considered the target ranges or each of the 94 attributes that represented the participants' target scores. Table 3 shows the raw Ignite pre-and post-(where applicable) scores, with the minimum and maximum EIR+IIR scores for each attribute (Figure 1).
In an alignment of the raw scores of the Ignite Accelerator participants to the EIR+IIR Entrepreneurial Target, a high correlation was found in the number of attributes where Ignite Accelerator participants' scores were within the EIR+IIR Entrepreneurial Target range. Pre-Ignite scores show in 90.7% of attributes, Ignite Accelerator participants were within the EIR+IIR Entrepreneurial Target range, and post-Ignite scores were within range in an average of 92.2% of the attributes. These results suggest that the program's recruitment efforts were successful in identifying and supporting highly entrepreneurial attitudes and mind-sets among the program participants. The common finding of the attributes from the Ignite Accelerator participants in the post-Ignite assessment aligning within the EIR+IIR Entrepreneurial Target suggests that potential growth of entrepreneurial and innovative characteristics occurred as a result the program.
In the comparison of the subjective rankings by the program directors, which were blinded to the behavioural assessments to the Entrepreneurial Spirit Target, results yielded 85% accuracy when compared to the derived match score, with only one subject out of the predicted rank order. Based on the results, there is preliminary evidence to suggest correlation of innovation program performance and behavioural features (Table 4).

Discussion
The meaningful assessment of the value gained from entrepreneurial experiences has been the subject of substantial research and debate. As far back as 1978, Kirzner [3] proposed that institutional influences on public entrepreneurs enable the opportunity for discovery. Similarly,    [25] examined the basis for social, political, economic, and technical influences on public sector entrepreneurship. These influences persist in today's government setting and serve as the roots that can enhance discovery and promote, change, or cultivate innovation.
The use of entrepreneurship education by government programs to stimulate increased discovery and economic activity has continued to expand [51,52]. However, the economic benefit of entrepreneurship education has been difficult to quantify.
To our knowledge, the study described herein is the first attempt to design a measurement and quantification system to assess the behavioural impact of entrepreneur training and education experiences of government programs at an individual level. The results demonstrate the feasibility of conducting behavioural assessments in a manner that is beneficial to the learning experience and respectful of the differences in talents and expertise among employees working in government programs. Further work on the optimization of the use of such tools toward targeted enrichment activities is needed, however, before such methods can be scaled and applied in a widespread fashion. Our belief that the use of quantitative and qualitative measures of individual attitudes and mind-sets is consistent with the core theory that a culture of measurement is necessary to promote a culture of innovation. The cardinal principles emphasized in the design and application of innovation interventions are to establish an iteration of an idea, test it, and reform it based on results and feedback.
Regarding the first two questions outlined in the objectives of this study, attempts were made to measure participants' attitudes and behaviours from pre-to post-Ignite and to align their behavioural attributes to those in a population of internal entrepreneurs. With the Ignite Accelerator cohort used for this study, the positive movement between Ignite participant scores and the alignment with the derived internal entrepreneur population indicate the potential for this assessment method to be repeated and to inform strategies for coaching and mentoring around a particular set of attributes. Further use of these measurement methods may be useful in facilitating the understanding of individual roles and interactions among project team members. The clustering of attributes from these specialized roles can be used to design innovation program curricula to enhance the entrepreneurial skills of subsets of government workers in specific roles (i.e., customer or beneficiary service, workflow processes, technology engineering and application, program management). Although this study looked primarily at the overall effects of innovation programs across all attributes, it is also important to focus on the attributes in which negative movement was observed from pre-to post-Ignite. Program directors can work with coaches to strategize possible ways to narrow the gaps seen in this analysis. For example, the two attributes with the most negative movement were personal power and assertiveness. Coaches in future rounds of the Ignite Accelerator can focus on the empowerment and decision-making of Ignite Accelerator participants, evaluate future assessments, and continue to refine strategies accordingly. With workplace trends moving toward increased automation and intelligent systems, there may one day be the ability to assess and provide coaches with feedback not only before and after programs, but also throughout to help maintain a course.
The final study objective explored whether program administrators could create a ranking of participant performance that aligned with a population of known entrepreneurs. The high correlation of the program directors' ranking with the match score further validates the ideals of success of the innovation program and correlates them to the Entrepreneurial Spirit Targets to be used in the future. This statistical significance gives credit to the innovation programs and makes the case that the innovation programs should continue to not only grow, but also use an assessment tool to understand better their participants, in every level and role. To see the Ignite participants' post-scores move to be more within range of the EIR+IIR program positively reinforces the concepts of mentoring and coaching and provides opportunity for future direction of innovation programs to have the same goals.

Conclusion
Beyond the implications for innovation programs such as those described in this study, the uses of behavioural evaluation tools could have broad implications for human resources in matching candidate skills and designing training and education programs for the workforce to build desired skills and competencies. Workers pursuing new employment opportunities may benefit from the use of behavioural psychology assessment tools (similar to those currently used in leadership or executive training programs) to assess the extent of the alignment of their innovation and entrepreneurial interests with those of organizations with which they seek employment opportunities. As future uses of behavioural assessments are conceived, tested, and implemented, their application could expand to measurement of not only individual programs, but also the individuals involved in such programs, thus maximizing efficacy and outcomes of innovation, as well as personal and professional growth.