Comparison of adaptive neuro-fuzzy inference system (ANFIS) and Gaussian processes for machine learning (GPML) algorithms for the prediction of skin temperature in lower limb prostheses

Monitoring of the interface temperature at skin level in lower-limb prosthesis is notoriously complicated. This is due to the flexible nature of the interface liners used impeding the required consistent positioning of the temperature sensors during donning and doffing. Predicting the in-socket residual limb temperature by monitoring the temperature between socket and liner rather than skin and liner could be an important step in alleviating complaints on increased temperature and perspiration in prosthetic sockets. In this work, we propose to implement an adaptive neuro fuzzy inference strategy (ANFIS) to predict the in-socket residual limb temperature. ANFIS belongs to the family of fused neuro fuzzy system in which the fuzzy system is incorporated in a framework which is adaptive in nature. The proposed method is compared to our earlier work using Gaussian processes for machine learning. By comparing the predicted and actual data, results indicate that both the modeling techniques have comparable performance metrics and can be efficiently used for non-invasive temperature monitoring. © 2016 The Author(s). Published by Elsevier Ltd on behalf of IPEM. This is an open access article under the CC BY license. ( http://creativecommons.org/licenses/by/4.0/ )


Introduction
Many amputees complain of increased temperature and perspiration within their prosthetic socket [1,2] . The most accurate temperature reading would be obtained by placing the sensor directly in contact with the skin, however, this would create practicality issues with prosthetic use in a domestic situation such as, protruding lead wiring, consistent positioning of sensors and possible skin irritation and discomfort. On the other hand, embedding sensors and wires in to the hard prosthetic socket during the manufacturing process for prosthetic sockets would eliminate any issues described earlier. In addition, there would be no damage to the device during the donning and doffing and its longevity would not be marred. We describe the route wherein the in-socket residual limb temperature can be accurately predicted by monitoring the temperature between the liner and the socket using artificial neurofuzzy inference system (ANFIS). The predictive modeling results are then compared with Gaussian processes for machine learning (GPML) previously developed by Peery et al. [3] . * Corresponding author. ANFIS are a class of adaptive networks that incorporate both neural networks and fuzzy logic principles. Neural networks are supervised learning algorithms which utilize a historical dataset for the prediction of future values. In fuzzy logic, the control signal is generated from firing the rule base. This rule base is drawn on historical data and is random in nature. This implies that the controller's output is also random which may prevent optimal results. The use of ANFIS can make the selection of the rule base more adaptive to the situation. In this technique, the rule base is selected utilizing the neural network techniques via the back propagation algorithm. To enhance its applicability and performance, the properties of fuzzy logic, i.e. approximating a non-linear system by setting IF-THEN rules is inherited in this modeling technique. This integrated approach, makes ANFIS to be a universal estimator [4] .
The goal of the Gaussian process technique on the other hand is to infer a continuous function f ( x ) from a training set of inputoutput pairs in supervised learning context. A, Gaussian process is a collection of random variables, any finite number of which have joint Gaussian distributions [5] . The key assumption in Gaussian process modeling is that the data can be represented as a sample from a multivariate Gaussian distribution. Therefore, it could be totally specified by the mean and covariance function. A Gaussian process model can be thought of as a prior probability distribution over functions in Bayesian inference. This enables deducing the hyperparameters for the model which are an indication of the precision and relevance of the input parameters for predicting the output. Thus, the aim in Gaussian process modeling is to select the model parameters for which the probability of the training data is maximized [5] .In this paper, we investigate the use of ANFIS and Gaussian process algorithms in the context of predicting the residual limb skin temperature of the amputee.

Method
It has been recorded by Klute et al. [6] that environmental factors like temperature, humidity, and also the activity level of an amputee affect their residual limb temperature. Also, it was suggested by Klute et al. [7] that some prosthetic materials act as insulators as restrict the transfer of heat and maybe the cause of thermal discomfort in the residual limb. Thus, if the thermal properties of the prosthetic materials are known then the residual limb temperature can be predicted by monitoring the temperature between the liner and the socket. In order to investigate the temperature profile correlation of the residual limb and the socket-liner interface, a 68 year-old male transitibial traumatic amputee weighing 70 kg was asked to take part in a laboratory test. The details of investigation were similar to as described by Mathur et al. [3] . In summary, the subject wore a 6 mm Polyurethane liner (Otto-Bock Technogel) with a 4 mm thick socket made of thermosetting lay-up material and was dressed in shorts and t-shirt without any extra layer of clothing on the prosthesis. According to Klute et al. [7] the thermal conductivities of materials used in the prosthesis of the amputee subject were -Polyurethane liner 0.19 W/m °K and Thermosetting socket material 0.14 W/m °K.
To monitor and record the residual limb and liner-socket temperatures, four K -type thermocouples via a data logger (type HH1384; Omega Engineering) were used. One thermocouple was taped on the lateral side of the limb and the other on the medial. The remaining two thermocouples were taped on the corresponding lateral and medial positions on the liner-socket interface. The schematic and the actual setup of the prosthesis with the sensor placement are indicated in Fig. 1 . Data from these four thermocouples was recorded at a sampling rate of 0.5 Hz at a defined ambient temperature (dataset A). This was repeated again after two months to confirm the influence of ambient temperature on the residual limb skin temperature (dataset B). Details of the 35 min experimental protocol are as indicated in Table 1.
The temperature profiles of the liner and the residual limb skin were recorded in a climate controlled chamber with zero wind velocity and 40% humidity levels for ambient temperatures of 10 °C, and then the same protocol was repeated for 15 °C, 20 °C, and 25 °C. The results indicated that for any given ambient temperature, the liner temperature profile follows that of the in-socket residual limb temperature. This suggested a possibility to apply supervised machine learning algorithms to model the residual limb temperature of the amputee as a function of liner temperature. Time averaging of 5 s is done on the recorded data to help in identifying the trend better and smooth out the fluctuations. Different modeling techniques for machine learning were utilized and the results from the Gaussian processes model and ANFIS technique are compared in this study. Since, the temperature profiles of the residual limb are almost similar for the ambient temperature pairs of 10 °C, 15 °C and 20 °C, 25 °C, the predictive model at ambient temperatures of 10 °C and 25 °C are only discussed in this study.

Adaptive neuro fuzzy inference strategy
ANFIS belongs to a family of hybrid system, called as the term 'neuro fuzzy networks' [8] inheriting the properties of both neural networks and fuzzy logic. Neural networks can easily learn from the data. However, it is difficult to interpret the knowledge acquired by it, as meaning associated with each neuron and each weight it quite complex to comprehend. In contrast, fuzzy logic itself cannot learn from the data. But fuzzy-based models are easily understood as it utilizes linguistic terms rather than numeric and the structure of IF-THEN rules. Linguistic variables are defined as variables whose values are words or sentences in a natural a b  language with associated degrees of membership. The fuzzy set in which linguistic variables belongs is an extension of a 'crisp' set where an element could have full or no membership. However, fuzzy sets allow partial membership as well, which implies that an element may partially belong to more than one set [9] . In other words, for a crisp set, the membership level of an element x in set But for a fuzzy set A the membership function μ A (x ) can take values in the interval [0, 1].The basic structure of the developed ANFIS controller for the prediction of residual limb skin temperature consists of four parts, which are, fuzzification, rule base, inference engine and the de-fuzzification blocks as seen in Fig. 2.
In the ANFIS controller, the crisp input signal (liner temperature in our case) is converted to fuzzy inputs by the membership function. The membership function pattern used in our ANFIS model is Gaussian. The fuzzy inputs along with the Gaussian membership function are then fed into the neural network block. The neural network block consists of a rule base which is connected to the inference engine. Back propagation algorithm is used to train the inference engine for the proper selection of rule base. Once trained, proper rules can be generated and fired from the neural network block to yield optimal output. The linguistic output from the neural network block is then converted into crisp output (residual limb skin temperature) by the defuzzifier unit [10] . The structure of the neuro-fuzzy model consists of different adaptive layers. Each of these layers has nodes with an associated network of transfer functions, through which the fuzzy inputs are processed. The output from these nodes are then combined to yield a single crisp output as the configuration of the ANFIS permits only one output of the model. This crisp output is fedback as input to the model and compared with the set value. If there is any deviation, the error signal so generated becomes the input to the ANFIS controller, thereby maintaining stability in the system [11] .
ANFIS, supports the Takagi-Sugeno based systems [12] . The structure of the adaptive network is composed of five network layers i.e. layer 1 to layer 5 (with nodes and connections) as shown in Fig. 3 . Assuming that the system is defined to have two inputs x 1 and x 2 , one output z and fuzzy set A 1 , A 2 , B 1 , B 2 ; then for a first order Takagi-Sugeno fuzzy model, having two IF-THEN rules in the common rule set, can be written using the following Eqs.
Layer 2: The nodes in this layer are fixed and labeled as O 2 ,i . The output of each node is the product of all the incoming signals as in the Eq. (6) .
The output of each node represents the firing strength of a rule. Also, known as the membership layer, it acts on the input variables from layer 1 as membership functions to represent them in their fuzzy sets.
Layer 3: Each node in this layer calculates the ratio of the individual rule's firing strength to the sum of all rules firing strengths as in the Eq. (7) . w i represents the normalized firing strength. Hence, this layer is also known as the rule layer.
Since, each node in this layer calculates the normalized weights, the output signal can be thought of as the normalized firing strength of a given rule. Layer 4: This layer known as the defuzzification layer. It calculates the individual output values y from the inferring of rules from the rule base. Individual nodes of this layer are connected to the respective normalization node in layer 3 and also receive the input signal. Each node of this layer is adaptive in nature with the node function given by the Eq. (8) where p i , q i , r i is a set of consequent parameters of rule i .
Layer 5: This layer is known as the output layer. It has only one node and it calculates the sum of all the outputs coming from the nodes of the defuzzification layer to produce the overall ANFIS output as in Eq. (9) .
This architecture of the adaptive network is used to develop the ANFIS model for the prediction of in-socket residual limb temperature and is discussed in the next section.

Model generation and prediction
The ANFIS model is designed using MATLAB's Fuzzy Logic Toolbox and the GUI editor which was used for analyzing its performance [14] . The optimized sets of rules were generated using the grid partition method. The architecture of the realized AN-FIS model had the following specifications; number of nodes: 84, number of linear parameters: 20, number of nonlinear parameters:    Fig. 8. Predicted residual limb skin temperature from ANFIS model is shown along with the actual skin temperature at lateral and medial sides in (a) and (b), respectively, at ambient temperature of 10 °C. The actual measured residual limb temperature is indicated as the checking data whereas the predicted residual limb temperature is the FIS output. The axis labels imply: index is the time in seconds and the output is the temperature in Celsius.

Obtained results
Our aim was to use the model to predict the residual limb skin temperature from the liner temperature and compare the results with direct skin temperature measurements. Hence, the input to the model is the liner temperature obtained from running the experiment in the climate chamber and the output of the model would be the predicted residual limb skin temperature. To test the predictive capability of a model we developed, the model is first 'trained' on one set of data and then is 'tested' on previously unseen data we collected independently. Finally, the results are compared for its accuracy.
It is seen from that the skin temperature is dependent on the ambient temperature. Hence, individual Gaussian process models and ANFIS models for the lateral and medial side of the residual limb were designed, using the principle as described in the previous section for ambient temperatures of 10 °C and 25 °C.
The actual skin temperature obtained by the ANFIS model is shown in Figs. 8 and 9 for two very different ambient temperatures of 10 °C and 25 °C, respectively. For both these experiments, done at different ambient temperatures, our predictive model using ANFIS provides a simple, effective, and practical approach to determine the unknown skin temperature of the subject within the prosthetic device from the actual liner measurements. Both the developed predictive models lead to results which have an accuracy of ±0.5 °C. However, this study needs to be extended on a greater population in order to define a generic behavior.

Discussion and test results
In this study, both the GPML and ANFIS were trained on the entire dataset A and tested for predicting the residual limb skin a b Fig. 9. Predicted residual limb skin temperature from ANFIS model is shown along with the actual skin temperature at lateral and medial sides in (a) and (b), respectively, at ambient temperature of 25 °C. The actual measured residual limb temperature is indicated as the checking data whereas the predicted residual limb temperature is the FIS output. The output label implies temperature in Celsius.  Tables 2 and 3. The MAE indicates how close the predictions are to the eventual outcomes which is given by As seen in Eq. (10) , the mean absolute error can be defined as the average of absolute errors; the absolute error given by where f i is the prediction and y i the true value. It should be noted that in MAE, all the individual errors have equal weight in the average, making it a linear score. In order to have a reliable statistical comparison between the mathematical models, both the MAE and RMSE can be used together to ascertain the variation in errors in a given set of predictions. Calculation of RMSE involves squaring the difference between the predicted and corresponding observed values, averaging it over the sample and then finally taking its average. This can be written as RMSE has a quadratic error rule, where the errors are squared before being averaged. As a result, a relatively high weight is given to large errors [15] . This could be useful when large errors are undesirable in a statistical model. From Table 2 it can be deduced that for the Gaussian model the MAE and RMSE is slightly lower as compared to ANFIS. But in order to discriminate between the models for their predictive performance, the error metrics should be capable to differentiate amongst the model results. In this context, the MAE might be affected by large average error values by ignoring some large errors. The RMSE is generally better in reflecting the model performance differences [16] as it gives higher weight to the unfavorable conditions. The difference between the RMSE of the Gaussian model and ANFIS is not immense and hence both the models have comparable performance metrics.
Another measure of goodness-of-fit of the model is the R 2 criteria. Higher values are indicative that the predictive model fits the data in a better way. By definition, R 2 is the proportional measure of variance of one variable that can is predicted from the other variable. Thus ideally the values of R 2 to approach one is always desirable. However, a high R 2 tells you that the curve came very close to the points but in reality it does not always indicate the model quality [17] . From Table 3 , both Gaussian and ANFIS models have similar R 2 values which are indicators that in both the modeling techniques, the prediction capability is similar. Using the R 2 criteria in conjunction with the MAE and RMSE, it can be fairly deduced that the Gaussian and ANFIS models can be accurately used for the prediction of residual limb temperature.

Conclusion
This study addresses the challenges of non-invasively measuring the in-socket residual limb temperature by comparing two different modeling techniques, namely ANFIS and Gaussian processes. The temperature profile of the residual limb skin is dependent on the ambient temperature and the activity level of the subject. The data obtained at ambient temperatures of 10 °C and 25 °C were used to develop an ANFIS model. The results from the AN-FIS model were encouraging. These were compared with the previously developed Gaussian model. The performance metrics of both the models indicate that they are very similar in their predictive ability with an accuracy of ±0.5 °C. However this approach has certain limitations as well. The residual limb temperature profile will differ for every amputee as there are variations in physiological responses (such as differences in capillary dilatation) and variations in properties of the skin parameters (such as thickens/composition of the skin layers). Because of the varying residual limb temperature profile in individuals, these machine learning algorithms have to be personalized by training them with individual datasets for each of the amputee subjects. This study which was conducted on one amputee subject a number of times, verified the success of proposed approach with an accuracy of ±0.5 °C. Thus, this work will be used to figure out the envelope in estimating the statistical power i.e. how many people are needed to make the model clinically significant and will be useful in extending it on a greater population in order to define a generic behavior. Also the temperature profile of the residual limb is dependent on the ambient temperature; it puts a constraint on drawing up a generalized model for all ambient temperatures. This could potentially be resolved by using interpolation or extrapolation techniques in the model at a given temperature to predict the residual limb temperature profile at another ambient temperature provided that the activity state of the subject is known.