AEB Online Calculator for Assessing Technology Maturity: IMATEC

This paper exposes initially the motivations for developing a TRL online calculator by AEB (Brazilian Space Agency; in Portuguese, Agência Espacial Brasileira). Such motivations demand the tool to be a way not only to assess the TRL condition of operational space missions and systems, but also as a method of evaluating and sharing proposals with new technological content. The TRL methodology and its relation to risk management are discussed. The development phases of the IMATEC Lite application are then described, with emphasis to the data handling structure capable of storing the user product model. A result of technology assessment using the software is detailed by presenting the PBS of the SERPENS 1 mission nanosatellite. The prospects of using the tool for judging the acceptability of new missions and projects are discussed, as well as some of the encountered difficulties.


INTRODUCTION
Developing a TRA calculator involves: (i) the creation of a product breakdown structure (PBS) in terms of which a given system or mission is described up to a pre-determined minimum level (Bilbro 2007;NASA 2013;Nolte 2005); (ii) the use of queries for the user to assess the technological maturity of each element; and (iii) an automatic integrator to summarize the product TRL as a function of the input (Altunok and Cakmak 2010). The output is given as a final report with the entire map of TRL linked to the product.
Although the TRL methodology seems to be universally accepted as a clear indication of the state of technological maturity of the components used in a given space mission or product, there is considerably less evidence on its systematic usage as part of a suite of management tools (Smith 2005), or as a way to assess new proposals. In fact, TRL alone can hardly be used to assess the full system maturity (Sauser et al. 2006). However, there is a relationship between TRA and the overall risk level (Mankins 2009a) of a technology development project, which manifests itself in the risk associated with schedule management (Dubos et al. 2007), for example. Such correspondence states that TRL scale inversely correlates with the disseminated risk of the ongoing mission or product development stage, even though TRL alone captures a fraction of the overall risk only. More generically, if used with care, the TRL scale can help not only to characterize the development phase, but also as an important goal descriptor of the readiness status of the several components to be used in a new mission or development proposal. This is precisely the context of the TRL calculator presented here, in which the notion of risk assumes a specific meaning associated with the state of readiness of a given technology to fulfill a certain function.

MOTIVATIONS
AEB is a Brazilian authority in charge of running the (civil) Brazilian Space Program (PEB; in Portuguese, Programa Espacial Brasileiro). Its roles are defined primarily by Act. 8854 of February 10th, 1994(Monserrat-Filho 2010. According to Article 3 of this rule, AEB oversees the elaboration and update of the National Program of Space Activities (PNAE; in Portuguese, Programa Nacional de Atividades Espaciais), a task involving the adoption of future space missions in a time span of 10 years. The execution of PNAE is described in the arrangement of the National System for Developing Space Activities (SINDAE; in Portuguese, Sistema Nacional de Desenvolvimento das Atividades Espaciais; Decree 1953 of July 10th 1996) composed of several federal institutions dedicated to space activities like the National Institute of Space Research (INPE; in Portuguese, Instituto Nacional de Pesquisas Espaciais; Space and Ground Segment) and Department of Aerospace Science and Technology (DCTA; in Portuguese, Departamento de Ciência e Tecnologia Aeroespacial, launch segment).
As a central body of SINDAE, the two fundamental tasks of AEB are: (i) making decisions about future space missions and the need of developing their associated technology; and (ii) supervising the execution of defined space missions. Both activities are good instances for the application of TRA, because, while (ii) applies specific management tools in assessing the several intermediary goals of a development schedule, (i) previous inherited technology has to be continuously assessed in order to decide about its convenient application to a future space mission or project.
The Brazilian federal government has recently stressed the need of all federal institutions to comply with procedures of risk control and mitigation prior to any decision related to their strategic goals. Such need is declared in the Normative Ruling MP/ CGU 001 of May 10th 2016, which states that "organs and entities of the executive authority should implement, maintain, monitor, and review internal management controls with the aim of identifying, assessing and managing all risks that could possibly impact the fulfillment of goals established by the public authority" (article 3). MP/CGU 001 is, in fact, a mandatory regulation imposing compliance with risk management, as accomplished by: Material and formal activities, such as policies, procedures, technique and tools implemented by the administration to reduce risk and to ensure the fulfillment of organizational objectives and public policies. These activities are either preventive (reduce the occurrence of risk events) or detectives (allow the prior identification of occurrences of risk events) and may be implemented in either manual or automatic ways (article 11-3).
The definition above states the legal framework justifying the creation of a TRL tool as a possible solution. We further emphasize that the sought tool should address the serious issues of risk mitigation in all phases of a space project, as a way of identifying all major technological risks right from the start. However, finding such risks, disseminated in the components of a new mission proposal is only partially achieved using TRA. One should also foster other good practices specifically created to the space industry such as, for example, the use of mission adoption procedures (as described in part by, for example, ECSS 2017b).
Besides risk control, other motivations may be invoked to justify the development of a standalone TRL tool. All freely available versions on the web of TRL tools consist of XLS files requiring a MS Excel compatible interface and appropriate O/S. They seem to be based on a very original idea (Bilbro 2007), but many of these tools exhibit the distinct ways in which TRL is evaluated. They are mixed up with other measures, such as 'manufacturing levels' (Nolte 2005), 'programmatic readiness levels' or 'system readiness level' (Kujawski 2013), mainly at the lowest levels of the PBS. Moreover, the output is given in terms of similar worksheets, in which the results are neither completely summarized nor properly formatted. More importantly, by relying on MS Excel worksheets, these tools are prone to several disadvantages, such as: lack of control, and vulnerability to fraud; difficulties in managing data security; susceptibility to human errors of several types and to troubleshooting/testing, and so on. But, from the point of view of software development (Wirth 1976, Jackson 2001, the lack of a data structure (DS) to represent the PBS is one of the main reasons justifying the creation of new software dedicated to TRL assessment. The worksheets also do not show any standardized application of TRL questions, except at the high levels defined by ECSS and NASA standards (Mankins 1995). These features, in fact, reveal the variety of internal guidance and customization during their creation in accordance to the client's intention.
To further justify the simplicity of the IMATEC tool presented here, we remember that the challenge of TRA is not simply to create a TRL calculator, but to apply the methodology in a consistent way (Kujawski 2013). Such consistency implies that the dependence on user opinion -which ultimately is used to judge a given maturity level for a subsystem or component -should be minimized, which is only achievable by starting from a simple interface for TRL definition. As stated before, such interface is desirable mainly in the design phase of a mission (or technology project proposal), when manufacturing and programmatic details are not fully available. Hence, we add the designation 'lite' to our TRL tool, as part of a broader suite of tools to encompass manufacturing and programmatic aspects in the future.

BACKGROUND ON THE TRL SCALE AND PBS
The classical definition of the TRL scale (ISO 2013) describes it as a ladder with nine steps (Table 1, Mankins 1995), starting from a modification of the original NASA seven-level scale. This table also shows the general development stages from 'basic technology research' to 'system test, launch and operation' as a function of TRL level. The development path goes from the innovation phase (in a 'academic context') to fully functional, almost 'service level stage' , typical of industrial environments. TRL 6 and 7 mark the end of the development phase, when the system is tested in the 'relevant environment' . Many launched-only-once systems typically finish their development lifecycle at these stages (as is the case of many nanosatellite missions, see Section 'Case Study: SERPENS 1'). However, some confusion exists in the definition of a 'relevant environment' in the literature. For ISO (2013), it is the "minimum subset of the operational environment that is required to demonstrate critical functions of the element performance in its operational environment" (Definition 2.16). This same norm defines 'operational environment' as the "set of natural and induced conditions that constrain the element from its design definition to its operation" (Section 2.11). In this work, this later definition will be adopted.
The intuitive notion of 'technology risk' evolves as we move to the later stages in the TRL scale and reduces to zero after successful operation of the product. However, when we apply the TRL scale to estimate the degree of risk of a given development, this intuitive notion relates mostly to the residual risk associated with the technological development and nothing else. This means that the scale does not express all the other risks (financial, operational, interface-related etc.), which are clearly relevant in the development program (Mankins 2009b). As stated in ISO: System test, launch and operation 9 Actual system "flight proven" through successful mission operations The time or effort to move from one TRL to another are technology dependent and are not linearly connected to the TRL scale. Experience shows that they can vary widely depending on the element and mission under consideration. Therefore, while the TRL scale is an appropriate tool for assessing the technology maturity status at a given point in time, it gives no indication of the effort and cost to be spent for reaching the next level (2013).
Hence, the use of TRL will not exclude an exhaustive application of other risk assessment techniques as is usual in many risk management estimations (PMI 2013), even at the system level (Sauser et al. 2010;Kujawski 2013). This is another manifestation of the 'nonlinearity' in the risk-TRL relation, because a given technology -once matured in a context -might be completely inappropriate in another, introducing additional risk.
Part of the TRA work involves the analytical description of the target mission or object (the project "product"), which is accomplished by creating a PBS (Bach and Hameri 1997; PMI 2013). Figure 1 is a four-level description of a product skeleton down to the "component level". Clearly, this is a simplification, because, in general, more intermediary levels may accurately describe the system under development. The degree of detail and number of components are often an engineering choice and are essentially defined in the earlier phase of a mission during system concept studies (ECSS 2017b). Usually (ISO 2013;ECSS 2014;NASA 2014;ECSS 2017c), for TRL evaluation, the definitions for each of these elements are: 1. System: equipment, method or facility capable of performing a functional task. 2. Subsystem: the first assembly of functionally related or interconnected parts. For example, the electrical power subsystem. 3. Assembly or unit: a complete or separable functional item at a lower level in the PBS. e.g., an Amplifier. 4. Component: a single unit which is destroyed if disembodied (e.g., a resistor).
The resulting maturity of the product is a function of all component contributions in terms of readiness to fulfill the main goal. The probability of mission success (in terms of complying with the defined goals) is a function of the success of all its constituent parts. Hence, the risk of failure (in a statistical view) of the whole system is heavily dependent on the risk of failure of all the components and on the way they are integrated -which is often a criticism raised against the exclusive use of TRL as an assessing tool (Hoboken et al. 2010). The TRL methodology evaluates systematically the system so that the overall readiness degree at a given level is smaller than or equal to the smallest degree of any of its constituents. Going up in the PBS, such logical role implies that the product TRL will always be smaller than or equal to the smallest TRL found in any of its parts, in accordance with the intuitive notion of overall probability of failure. The less capable part is the 'risk bottleneck' responsible for most of the potential failures.
Creating a PBS and assessing the maturity level of all its components is a systematic way of exhaustively estimating the risk of failure at a given stage of development associated with a system component. The failure probability naturally evolves with time as each component undergoes many tests, first in simulation laboratories, then in the several other relevant and operational environments. Despite the limitations of TRL described in the literature (Valerdi and Kohl 2004) and its restriction to technological risk, there are clear advantages: • It allows better estimation of the system technological uncertainties, because evaluating a given component TRL is equivalent to making a risk assessment in a certain sense. • The higher the TRL of a given component, the smaller is the contribution of that component to slippage of the project schedule. • TRL allows the project manager to identify the components in need of substantial investment as those still in the early development stages (smaller TRL). • With proper care regarding the interfaces, TRL may represent a convenient way of describing the actual conformity of a given technology to the mission or project requirements. On the contrary, in poorly defined missions, requirements may be organized in order to satisfy the current stage of technological readiness, creating a false image of 'proper fitness' of a given component to those requirements. The TRA process is summarized in Fig. 2. It is important to emphasize that the result of a product or system evaluation ('end block') is a 'snapshot' of a given technology stage. As it evolves, new evaluations become necessary, thus forming a continuous flow of assessments intended to accurately describe the status of a technological path.
Create the PBS structure of the product under assessment. The proposed PBS has four levels: product, subsystem, assembly and component Evaluate the TRL of each assembly. The TRL of each assembly cannot be larger than the smallest value of its components Evaluate the TRL of each subsystem. The TRL of each sybsystem cannot be larger than the smallest value of its assemblies.
Evaluate the TRL of each system. The TRL of each system cannot be larger than the smallest value of its subsystems.
Identify all components, assemblies and subsystems wich do not comply with the level required by the program or system. <<End>> of TRA.
Evaluate the TRL each component.
Is a new technology available?

THE AEB IMATEC LITE AS A TRL ASSESSING TOOL
A Portuguese translation of the acronym TRL is "nível de maturidade tecnológica". To emphasize the scale as an index, the calculator was named 'IMATEC' , where 'I' stands for 'index' and 'MATEC' for technological maturity. The Portuguese version described here (called 'IMATEC Lite') will be followed in the future by the public release of the tool for an initial assessment by SINDAE users, Brazilian academic institutions, and other national research funding agencies.
IMATEC Lite implements a TRL query by adapting to Portuguese the several questions proposed by Bilbro (2007), and, subsequently, by creating a DS for the user to enter and store the PBS. The questionnaire follows the questions available in the XLS file TRL_Worksheet_11-30-10.xls (NASA 2014), which provides different questions on the system and subsystem levels, and similar questions to components and assemblies. Thus, in order to save space, these questions will not be reproduced here.
As it is apparent in the PBS (Fig. 1), the application input is represented optimally by a classic DS, in which the questions are properties associated with the class members filling the product component hierarchy. The advantages of using the data organization schemes are well documented (Mehta and Sahni 2004;Goodrich and Tamassia 2006), because they address two major problems related to how data is to be stored in the application and what operations will be performed on them. A schematic representation of the user interaction with the IMATEC tool is shown in Fig. 3.
Since the 'mental PBS' on the user's mind (Fig. 3) might not correspond to the hierarchy of Fig. 1, the user is naturally constraint to represent his product in accordance to that model. The product creation/edition phase orients the user to create a distinct model in accordance to this hierarchy, which is then stored in the product DS. The second phase is the product assessment, in which the hierarchy is associated with a set of queries admitting only true/false answers. An 'if-then' logic is created internally to guide the appearance of the query according to the user's past assessment on a given hierarchy level.
Comprehensibly, the hierarchy is filled up from the lowest to the highest component levels. When all questions on a given level are answered, a new query is opened for the next level. The TRL value of a given level will never be larger than the one found among its constituent´s components on the level immediately below. Thus, an assembly will only be available to evaluation when all its components were finished and so on. After reaching the last level, a final report containing the overall IMATEC rate of the system is calculated. The use of a DS and the 'if/then' logic greatly adds flexibility to the tool, which may be improved with other technological measures (i. e., 'manufacturing readiness'), without any loss to the PBS hierarchy.

Mental PBS map
User Product creation/edition Product DS Product assesssment Report results The application (the initial 'new product screen' is shown in Fig. 4) was conceived in such a way that neither the user needs to install any software, nor the system provider needs to store any product information as given by the user. External product data are assessed and saved in the user's own system through a simple ASCII file, which may be reloaded, thus saving the IMATEC assessment for later access and edition.  Following the process illustrated in Fig. 3, the application has four major user features: The product creation is the module responsible to build a PBS starting from a default DS, and which contains the hierarchical levels, such as subsystems, assemblies, and components.
Using the product edit window, the user may freely add more elements to the product and to each of its elements, as well as change names and write information on key items, implementation methods, and justification. The user then saves and exports a file containing all the input information. The output file uses JavaScript Object Notation (JSON) (Severance 2012), which can be reloaded using the same application if the user decides to modify or continue the analysis.
Some of the 'true/false' buttons for the questions are shown in Fig. 4, which exhibits the assessment window (Fig. 5). At the end of a product evaluation, the user gets a full report generated by the report result module. The output document summarizes what the user has just entered and displays the final product IMATEC level, but also lists all intermediary levels in accordance to the input hierarchy. An interesting interaction between the user and the interface is expected, because the tool makes the user think about its system architecture sometimes from a different perspective, especially when the readiness levels of its elements were not a relevant issue in the system design phase. The tool works as a systematic product descriptor, providing a valuable task in the early phases of a new project.
The software architecture is based on the concept of a front-end solution. Thus, all PBS related information is entered and manipulated from the data provided to the front-end modules. The only back-end functionality is the register/login features. Personal identification is stored in a simple MongoDB database (Banker 2011). The application runs under nodeJS (Cantelon et al. 2014) and the modules use AngularJS (Darwin and Kozlowski 2013;Jain et al. 2015). The visual identity is based on Bootstrap CSS framework. The software development process was based on Agile and Scrum methodologies (Schwaber and Beedle 2002), with minor software artifacts being delivered after every main 'sprint.

CASE STUDY: SERPENS 1 MISSION
Because SERPENS 1 subsystems have different readiness levels, the test was a good application of the IMATEC methodology implemented by the proposed Lite calculator. In the Appendix, we detail the resulting PBS output report generated by IMATEC Lite, and the user justification for each resolved element. In the following, we introduce the SERPENS 1 mission. SERPENS 1 (COSPAR designation 1998-067GX; Fig. 6) derives its name from 'Space System for Conducting Research and Experiments with Nanosatellites' in Portuguese (Sistema Espacial para Realização de Pesquisa e Experimentos com Nanossatélites). It is, in fact, the first satellite of a series of cubesats for the so-called SERPENS educational program. The program primary goal is to give students and young engineers the opportunity to participate in a real satellite mission (Gonçalves and Veras 2016), in order to foster international technical cooperation. As such, SERPENS 1 was the result of a multinational effort involving Brazil, Spain, Italy, and the USA. Nationally, several universities and institutions took part in SERPENS 1 (such as INPE; Fernandes et al. 2016), under the coordination of the University of Brasília (Figueiró 2014;de Oliveira et al. 2015;Gonçalves and Veras 2016), with over 100 students involved.
SERPENS 1 is a 3U CubeSat divided into two sectors, called A and B. Sector A is a technological demonstrator with the aim of testing space technologies developed in Brazil by the educational program. The main payload of Sector B is an UHF transceiver designed to collect, store, and transmit environmental data, being part of the HumSat constellation (Woellert et al. 2011;Tubío-Pardavila et al. 2014). Each sector may be further broken down into the following subsystems: • On-Board Data Handling (OBDH). • Electrical Power Subsystem (EPS). • Telemetry, Tracking, and Command (TTC). • Payload.
The ADCS (Attitude Determination and Control System), the structure, and the wiring systems are common to the two sectors. Many of the SERPENS 1 subsystems and components are based on COTS (commercial off-the-shelf) elements, following the CubeSat integration practice. With approximately 4 kg of mass, SERPENS was launched onboard a Japanese H-IIB rocket and injected into a ~ 400 km orbit from the International Space Station (ISS) in September 2015. It successfully operated until March 27, 2016 when, due to atmospheric drag typically found in low orbits, it reentered the atmosphere. As described in the Discussion, only Sector B operated as expected. Sector A experienced a failure and was not able to establish a communication with the ground.
By presenting an IMATEC-based description of SERPENS 1 (see Appendix), relevant goals of the TRA process are achieved, such as to decompose a complete and closed space system component in its TRL description, thus proving the excellence of the TRL tool. The calculator test was completed just after the debugging stages of the IMATEC tool, when other 'toy models' were used. The resulting PBS structure was built down to the assembly level only ( Fig. 7; see Appendix). The justification for the three-level PBS is twofold: (i) because the satellite is made of COTS mostly, three levels are sufficient to characterize the TRL description of the entire system; (ii) lack of space here prohibits the presentation of a full PBS tree (which is limited to Fig. 7). In a first straightforward application of the calculator, SERPENS 1 IMATEC reached level 6. Some comments about this outcome are presented in the Discussion. The user-calculator interaction allowed further detailing of the SERPENS PBS, which suggested important interface enhancements and improvements in the way the tool should be used, and its results interpreted.

DISCUSSION
As shown in the proposed SERPENS 1 PBS structure, the existence of a redundant system (Sector A) allowed us to interpret the TRA process for this system in two different ways. SERPENS 1 may be regarded as a single 'technological product' in which case the final IMATEC is 6. In accordance with ISO (2013), SERPENS 1 has "verified the critical functions of the elements, demonstrated performance in the relevant environment and displayed a representative model in form, fit, and function" as achieved milestones. At the same time, most of the SERPENS 1 structure and Sector B can be viewed as a 'bus' for testing Sector A, which unfortunately failed its first test in the operational environment -a condition for reaching IMATEC 7. In fact, some spurious signals received during the commissioning phase were sufficient to rule out an absolute failure of Sector A, possibly due to the extreme freezing of its batteries and an error in the design of the Battery Charge Regulator board. Thus, an industrial platform provided the 'relevant environment' for testing a still immature technology item. According to this interpretation, SERPENS 1 mission served as a necessary stage for advancing Sector B level toward the full operational environment. However, the main requirement of the SERPENS 1 mission was not technological: the presence of Sector A as a contribution of many Brazilian universities added a component of 'technological development and, therefore, some degree of risk. Nevertheless, the observed failure was counterbalanced many times by the gains achieved by other aspects, even prior to the satellite launch -for example, during the simulation tests at LIT (INPE Laboratory of Integration and Tests).
These open interpretations expose some of the already cited limitations of TRL (Sauser et al. 2006;2009) as an instrument for assessing any kind of risk associated with the system. Since "the whole is greater than the sum of its parts", Sectors A and B working independently showed a "low interface readiness level", because Sector A could not be accessed by any way other than by its own. In this sense, there was no causal relationship between both subsystems, a feature not easily captured by the methodology. Hence, the final IMATEC 6 level is exclusively a measure of the individual technology level related to a single component and not the whole system or mission.

CONCLUSIONS
This paper presents a tool developed by AEB with the goal of providing a platform to evaluate the TRL of products or systems associated with space components. The software tool was named IMATEC Lite to emphasize the technological scale as an 'index of technology readiness, using well established notions of the TRL. A case study involving a nanosatellite was assessed using the tool, and the results have helped to interpret the outcome of the automatic TRL assessment process thus created. The results achieved and the tests performed have demonstrated that IMATEC Lite is a useful tool to clients and stakeholders of future space mission proposals, especially when an evaluation of technology maturity is necessary (ECSS 2017c). One of the lessons learned from the many tests using the calculator was that it is hard for the user to separate his/her inner notion of risk -associated with the system -and make a judgment only in terms of the whole technology degree implied by that system. The user should be oriented to think about the answers to the many TRL questions exclusively in terms of the technology stage of the specific component -which, somehow, lessens the importance of the interfaces with which that component is logically or functionally associated. The IMATEC Lite tool proved to be an essential help in this sense. Amongst the gains enabled by the tool, the easiness of product definition, the facility in storing, sharing, and augmenting the data structure, and the portability of the output report may be listed. Improvements in the current version of the IMATEC calculator should be considered in order to add other capabilities to the evaluation process of distinct system components, such as software, hardware, and their interface. In fact, supplementary questions may be provided to orient the evaluation in accordance with component types (i. e., by the creation of specific software-hardware interface queries). Another improvement is to adapt the tool to treat redundant elements and to allow for proper interpretation of the TRL value obtained when these elements are integrated fully. Other partially applied concepts, such as 'manufacturing readiness level' or 'programmatic readiness' may be easily integrated to the DS already established for IMATEC Lite.

APPENDIX: COMPILING THE OUTPUT REPORT OF SERPENS 1
In view of the previous knowledge used to build SERPENS 1 (Figueiró 2014;de Oliveira et al. 2015;Gonçalves and Veras 2016), the following information is relevant to the PBS creation: • Because subsystems and assemblies of Sector B correspond to the same elements presented in the HumSat family (Woellert et al. 2011), their IMATEC level was set equal to 9.
• Other subsystems of Sector B and components specially developed for SERPENS 1 were successfully validated in orbit. So, they were rated IMATEC 7.
• Subsystems and assemblies of Sector A, as direct applications of COTS solutions that were already flight proven, were rated IMATEC 9; • Because subsystems and components of Sector A were custom built for SERPENS 1, they were tested before launch, but were not demonstrated in space, due to a failure in the sector. Hence, they were set to IMATEC 6 level; • All elements that underwent environmental and functional tests at LIT were rated IMATEC 6. In the list below, RBF is an acronym for "Remove Before Flight".
• Product name: Nanosatellite Serpens 1; • IMATEC of the product: IMATEC 6; • Key Technology Items: an embedded computer system, energy supply systems, radio communication system, and design to meet ISS interface requirements.

HARDWARE (HDWE): IMATEC 9
Justification: The onboard computer hardware is based on COTS. No further subdivisions.

SFWE: IMATEC 9
Justification: vendor upgraded the software to meet the subsystem requirements. No further subdivisions.

TTC_A: IMATEC 6
Justification: environmental and functional tests performed at LIT/INPE. The TTC_A subsystem has the following assemblies: