Detection of apple firmness with a novel loudspeaker-based excitation device

: Firmness is one of the important indices to evaluate the internal quality of fruit. In this study, a noncontact loudspeaker-based detection system was developed to evaluate apple firmness. The structural parameters of the excitation device were modified in the single-factor experiments, and the best combination of structural parameters was that the inner diameter of the gasket was 40 mm; the distance between fruit surface and loudspeaker was 95 mm. Besides, the proper posture style was that the apple was placed with its stem upward. After the modification of the Laser Doppler Vibrometer (LDV) method, the vibration response signals of 48 apples were measured to establish the firmness prediction model. The results showed that the better prediction performance of stiffness was obtained in multiple models. The Back Propagation Neural Network (BPNN) model had the best prediction performance by using parameters of elasticity index (EI), the peak value at the second resonance frequency f 2 ( A 2 ), and peak area S , with a correlation coefficient of prediction ( r p ) of 0.914; root mean square error of prediction (RMSEP) of 0.491 N/mm. Therefore, the proposed detection system is feasible to nondestructively detect apple firmness, which has the potential to be applied in online detection. Eng, 2022; 15(1): 260–266.


Introduction
Firmness is one of the key factors to evaluate the internal quality of the apple, which is closely related to the mechanical properties of its flesh [1,2] . Accurate detection of firmness is indispensable in the fruit supply chain. In the harvest, firmness is used to confirm the proper harvest time and ripeness for edibility. In the grading process, firmness is the basis for classification. As for transportation, firmness is considered a standard to select proper methods of transportation and packaging. In terms of storage, firmness helps to select the proper storage temperature and time. For the sale, firmness is important to evaluate the taste and freshness of fruit, which deeply affect consumer purchasing behaviors [3,4] .
Fruit firmness detection methods can be divided into two classes, destructive and nondestructive methods. The frequently used destructive method is Magness-Taylor (MT) puncture test, which was considered a standard in industries. In the puncture test, a penetrometer records the force by penetrating the fruit tissue at a specific speed for a certain depth [5] . However, the main disadvantages of this method are time-consuming, labor-intensive, and local measurements.
Over the past few years, many nondestructive techniques have been developed for firmness assessment, such as acoustic vibration [6] , spectroscopy [7] , and ultrasonic [8] . Among them, the acoustic vibration method is one of the most frequently used methods to nondestructively evaluate the firmness of the entire fruit based on vibration parameters [9,10] . Based on existing studies, a series of vibration parameters were extracted to evaluate the fruit firmness, such as f 2 m, f 2 m 2/3 , and f 2 m 2/3 ρ 1/3 [11][12][13] , where f is the resonance frequency, m and ρ are the mass and the density of fruit. Generally, a detection system mainly contained an excitation device, detection sensor, and signal analysis software [6] . To satisfy the requirement of fast and nondestructive inspection, many noncontact excitation devices and detection sensors were introduced, such as a speaker and a Laser Doppler Velocimeter (LDV). For example, Fumuro et al. [14] used a portable vibration system to evaluate the optimal harvest time of pitaya. During the measurement, the sample was sandwiched between a small speaker and a receiver by fingers. Similarly, Kataoka et al. [15] developed a portable device to detect tomato firmness, which consisted of a smartphone, a microphone, and a speaker. The smartphone provided the swept sine signal from 20 to 10 000 Hz in 1 s to excite the fruit by a speaker and captured the response signal by a microphone. The LDV is another alternative noncontact sensor to obtain the vibration velocity of the samples based on the Doppler shift of the reflected laser beam, which has the merits of high sensitivity, high precision, quick dynamic response and wide detection range [16] . In the early time, Muramatsu et al. [17] first utilized an LDV to inspect the firmness of different fruit, such as apples, kiwifruits and pears. The measurement results received by an LDV had higher precision than the accelerometer, especially in the frequency range of 800-1600 Hz. Lately, Abbaszadeh et al. [18] developed an LDV-based system to estimate the firmness of watermelon. In the detection, the shaker was used to excite the bottom of the watermelon, and the LDV vertically recorded the response signals of the top of the sample. The results revealed that the stepwise multiple linear regression model (SMLR) had better prediction performance of sensory score by using 24 phase shifts as the inputs (r p =0.999; RMSEP=0.035). Though the acoustic vibration method could be applied in firmness detection of fruits, few researchers investigated the factors that influenced the measurement results, such as posture styles and detection points. Therefore, the repeatability of the developed detection system should be evaluated to obtain reliable response signals during the measurement.
In this study, a novel detection system based on a loudspeaker and an LDV was developed to nondestructively inspect apple firmness after harvest. The objectives of this study were 1) to develop a loudspeaker-based device to realize noncontact excitation; 2) to determine the optimal combination of structural parameters of the excitation device; 3) to establish different regression models to predict apple firmness.

Apple samples
In total, sixty Fuji apples (Malus domestica cv. Fuji, produced in Shandong Province, China) were selected from a local fruit orchard, which had a round shape, firm and juicy flesh, rich nutrition ingredients, and good storage ability. After harvest, all apples were placed in a laboratory for 12 h at 20°C and 60% relative humidity (RH) before detection. Twelve were used to evaluate the performances of the developed excitation device with different structure parameters. The remainder was used to establish the prediction models of apple firmness, where 32 samples were in the calibration set, and 16 samples were in the validation set. Before the test, each sample was randomly coded, and their physical parameters were measured. The mean value and standard deviation of mass, height, and equator diameter were (280±31.41) g, (79.47±4.28) mm, and (74.53±3.78) mm, respectively. Due to small variations of all physical parameters, these samples were nearly uniform in size.

Design of the noncontact excitation device
A loudspeaker-based device was developed to excite apple (Figure 1), which contained a sound-hole gasket, a metal cylinder, a high-power loudspeaker and a base. A gasket was installed on the top of the metal cylinder, and the loudspeaker was mounted on the base. Due to the limited excitation force produced by the loudspeaker, the intensity of the response signals was relatively low. Thus, the response signal was easily affected by the environmental noise and electric power supply. To strengthen the excitation force, different structural parameters were considered, such as the inner diameter of gasket (d) and distance between fruit surface and loudspeaker (h). According to the structure of excitation device, the biggest d and the largest h were 40 mm and 155 mm, respectively. Therefore, the inner diameters of the gaskets were made in three sizes: 20 mm, 30 mm, and 40 mm. And the distance between fruit surface and loudspeaker was adjusted from Note: d: diameter of the gasket, 20 mm, 30 mm, and 40 mm; h: distance between fruit surface and loudspeaker, 95-155 mm. Figure 1 Schematic diagram of the loudspeaker-based excitation device 95 mm to 155 mm by revolving the base. Through the single-factor experiments, the signal-to-noise ratio (SNR) of the response signal was used to evaluate the performance of different structural parameters.

Vibration response signal measurement
The detection system for measuring the vibration response signal of apples mainly consisted of a noncontact excitation device based on a 30 W loudspeaker (CS622C, Dayton Enterprises, USA), a microphone (40AE, M+P Enterprises, Germany), an LDV (LV-S01, Sunny Instruments Singapore Pte., Ltd., Singapore), a data acquisition (DAQ) module (USB-4431, National Instrument, Austin, USA), a power amplifier and a personal computer (PC) ( Figure 2). In the measurement, a reflective film was stuck on the surface of an apple to enhance the laser reflection and placed the apple on the tensioned string bag to freely vibrate. The distance between fruit surface and loudspeaker could be adjusted by revolving the base. The loudspeaker produced a swept sine wave signal (frequency 250-2000 Hz in 1 s) to excite the apple. The sound signal recorded by the microphone was as input signal (X in ). Meanwhile, the LDV obtained the response signal from apple surface which was considered as the output signal (X out ). The units of input and output signal were Pa and m/s, respectively. Through Fast Fourier Transform (FFT), these signals were transformed from the time domain to the frequency domain, and their ratio was regarded as the frequency response function (FRF), and the unit of its amplitude was m/(s·Pa) ( Figure 3). Then, four vibration parameters were extracted from the FRF: the second resonance frequency (f 2 ), the peak value at f 2 (A 2 ), the peak width at half height (w), and the peak area (S=A 2 w). In addition, the elasticity index (EI=f 2 2 m 2/3 ) and stiffness coefficient (SC=f 2 2 m) were calculated, where m is the mass of fruit. Note: LDV: Laser Doppler Velocimeter; DAQ: Data Acquisition; PC: Personal Computer. X in is the sound signal recorded by the microphone; X out is the response signal obtained from the apple surface by LDV. Figure 2 A loudspeaker-based detection system to measure the vibration of apple Note: f 2 is the second resonance frequency, Hz; A is the peak value, m/(s·Pa); A/2 is the half of the peak value, m/(s·Pa); w is the peak width at half-height, Hz. Figure 3 Typical frequency response function of an apple obtained by the loudspeaker-based detection system

Firmness measurement
The standard penetrometer (TA-XT2i, Stable Micro Systems Ltd., England) was used to destructively measure the fruit firmness. In this section, three peeled detection points with equal intervals on the equator of the apple were selected. At each site, a flat-tip cylindrical probe (P/5) of 5 mm diameter penetrated the fruit flesh. The penetration velocity and depth were 1 mm/s and 8 mm, respectively. Three destructive firmness indices were extracted from the Force-Deformation curve (F-D curve), which were stiffness (Stif, which is the slope of the curve before the rupture point), MT firmness (maximum force), and flesh firmness (FF, which is the mean force from 2 to 8 mm) (Figure 4). The average values of each index were calculated and used in the following analysis [4] .

Experimental procedure
Twelve apples were selected in the single-factor experiment to modify the measurements. First, the apples were excited by the excitation device with different gaskets and distances between fruit surface and loudspeaker. The inner diameters of the gaskets were 20, 30, and 40 mm. The selected distances were 95, 115, 135, and 155 mm. Each test was repeatedly performed three times, and the best combination of structure parameters was determined by analyzing the SNR of the response signals. After a modification of the excitation device, the effects of different posture styles and detection points on response signals were also investigated. As shown in Figure 5, the apple can be placed on the string bag in three different postures: the stem is upward (Posture style A), the calyx is upward (Posture style B), and the stem-calyx is horizontal (Posture style C). For each posture, three detection points with equal intervals were selected as a group to compare with other groups using the SNR of the response signals. Subsequently, the repeatability of the modified detection system was evaluated based on the coefficient of variation (CV) of all vibration parameters.
According to the modified detection system, the response signals of 48 apples were measured; then, a puncture test was performed for each apple to obtain the destructive firmness indices.   The signal-to-noise ratio is an index to compare the level of the measured signal to the level of noise, which was calculated using Equation (1). Max where, a and b were data points, Hz; f(a) is the response signal in the time domain; f(b) is the background noise. In this study, the SNR of response signals was used to determine the proper structural parameters of the loudspeaker-based excitation device and posture of the apple during the measurement.

Repeatability analysis
The repeatability was used to describe the degree of consistency among the results of successive measurements. In this section, the repeatability of different detection points was evaluated by the coefficient of variation (CV) of the vibration parameters, which was defined as the ratio of standard deviation (SD) and mean value (Mean) of repeated measurements. In general, a value of CV below 10% shows that the detection system had good repeatability [19] .

Correlation analysis
Correlation analysis was used to understand the direction and strength of the relationship between two individual variables [20] . In this section, the correlation coefficients (r) among vibration parameters were firstly investigated to avoid the collinearity problem of the input variable, which was calculated in Equation (2). Then, the relationships among the destructive firmness indices obtained from F-D curves and the remaining vibration parameters extracted from FRF were assessed to select output variable of firmness prediction model.
where, x i and y i are the i-th measurement of variables X and Y (i=1,2, …, n); x and y are the mean values of X and Y.

Modeling analysis
To establish the prediction models of apple firmness, various modeling methods were introduced in this study. The Stepwise Multiple Linear Regression (SMLR) method and Partial Least Square Regression (PLSR) method were two main types of linear modeling methods.
SMLR selects significant independent variables and removes those that are not important based on the variance contribution in a linear regression model to avoid multicollinearity [21] .
PLSR reduces the correlated input variables and finds the latent variables by the covariance analysis. Nonlinear modeling methods were also applied for the quantitative analysis of the apple firmness, such as Support Vector Regression (SVR) and Back Propagation Neural Network (BPNN) methods. The SVR method is a supervised learning method. The main characteristics of this method are to maintain the maximal margin and minimize the error [22] . The BPNN method is a widely-used nonlinear network for supervised learning, which is trained by the backpropagation of the error. A neural network mainly consists of three parts: input layer, hidden layers, and output layer. There are several neurons in each layer, which are connected with other layers. In the calculation process, this method adjusts the network weights and threshold according to the prediction error until the predictive value is close to the desired output [23] . Besides, one hidden layer was utilized to reduce the training time.
And the activation functions of the hidden layer and output layer were tangent-sigmoid and pure functions. The learning efficiency and the error range were 0.1 and 0.0004, respectively.

Parameter optimization of the loudspeaker-based excitation device
The response signals in the frequency domain of the single-factor experiments are shown in Figure 6. The second, third, and fourth resonance frequencies (f 2 , f 3 , and f 4 ) were almost identical, while the amplitudes of the response signals were different, which might be due to the variance of excitation force caused by different structure parameters of the exciter. a. Selected distance was 95 mm b. Inner diameter of the selected gasket was 40 mm. Note: f 2 , f 3 , and f 4 are the second, third, and fourth resonance frequencies, respectively, Hz; Distance means the distance between fruit surface and loudspeaker, mm. An offset of 0.2 m/(s·Pa) was added to make each signal visible. Figure 6 Response signals in the frequency domain of apples excited by the excitation device with different structure parameters To modify the excitation device, the SNR values of the response signals in the time domain were used to evaluate the performance of different structural parameters. The SNR values at different inner diameters of gaskets are shown in Figure 7a.
The SNR nearly linearly increased with the increase in the inner diameter of the gasket. The highest SNR and lowest standard error were observed when the inner diameter of the gasket was 40 mm. In addition, the SNRs at different distances between the fruit surface and loudspeaker are shown in Figure 7b. Due to the maximum SNR and minimum standard error, 95 mm was selected as the proper level. Thus, the proper combination of d and h was 40 mm and 95 mm, respectively.

Effects of the posture styles and detection points
After a modification of the excitation device, the effects of different posture styles of the apple on the measurement results were investigated. Figure 8 shows the response signals for three different posture styles. The response signals for Posture styles A and B almost overlapped, and their second to fourth resonance frequencies were clearly visible. For Posture style C, there was a significant difference among all three resonance frequencies, particularly the fourth resonance frequency. In addition, the amplitude of its response signal was relatively low compared to those of other posture styles. As shown in Figure 9, Posture style A obtained a slightly higher SNR than that of Posture style B. However, Posture style C had the lowest SNR and highest standard error, which was unstable during the measurement.
Note: An offset of 1.0 m/(s·Pa) was added to make each signal visible.  To study the effects of different detection points on the response signal, three points with equal intervals were selected as a group to compare. As shown in Figure 5, each detection point was measured three times, and four vibration parameters (f 2 , A 2 , w, S) were used to describe the characteristics of the response signals.
The CV values of these parameters were proposed to evaluate the repeatability and stability of different detection points for each posture style (Figure 10). Good repeatability was found in each group (CV<5%), which indicates that the effects of different detection points with identical posture styles were negligible. In addition, Posture styles A and B had similar performances in four vibration parameters, and they were much better than those of Posture style C. Due to the highest SNR value and good repeatability, Posture style A was selected for subsequent experiments.
a. The second resonance frequency b. Peak value at the second resonance frequency c. Peak width at half height d. Peak area Figure 10 Coefficients of variation of four vibration parameters at different detection points

Repeatability evaluation of the modified detection system
The repeatability of the modified detection system was evaluated by a comparison of the CV values of all vibration parameters (f 2 , A 2 , w, S, EI, and SC) of twelve apples ( Figure 11). In this section, each sample was placed on the string bag with Posture style A, and the measurement was repeated three times. It could be seen that f 2 had the best repeatability (CV<1%) compared with the others. In addition, EI and SC exhibited identical CV values, which revealed the repeatability of f 2 2 . The worst result of S was observed since its value had great fluctuation caused by both A and w. In general, all CV values of twelve apples were less than 10%, which indicates that these parameters could be accurately extracted through the modified detection system and selected as latent variables in firmness prediction models.

Quantitative analysis of apple firmness
Based on the modified detection system, six variables of forty-eight apples were obtained to predict apple firmness. To diminish the collinearity effect of these variables, the intercorrelations of six variables are listed in Table 1. The correlation analysis results demonstrated that f 2 , EI, and SC were closely cross-correlated (p≤0.01) with r of 0.629-0.827. In addition, w was strongly correlated with S (r=0.571), EI (r=−0.611), and SC (r=−0.641). S was moderately correlated with A 2 (r=0.514), EI (r=−0.499) and SC (r=−0.446). It could be seen that . Therefore, f 2 and w would not be introduced as input variables in the following analysis. is the second resonance frequency, Hz; A 2 is the peak value, m/(s·Pa); w is the peak width at half-height, Hz; S is the peak area, Hz·m/(s·Pa); EI is the elasticity index, Hz 2 ·kg 2/3 ; SC is the stiffness coefficient, Hz 2 ·kg; * means significant correlation at the level of 0.05; ** means significant correlation at the level of 0.01.
The relationships between destructive firmness indices (Stif, MT, and FF) and two vibration parameters (EI and SC) are listed in Table 2. It could be seen that the stiffness of apple was both closely related with EI and SC, which was appropriate to be an output variable in firmness prediction models.  Table 3 revealed the prediction performances of different simple linear regression models. All variables (EI, SC and S), except A 2 , were strongly correlated with fruit stiffness. The best prediction performance was obtained using EI as the input variable, whose r p and RMSEP were 0.763 and 0.670 N/mm, respectively.  The performances of different multiple regression models in predicting the stiffness are listed in Table 4. Compared to the performances of the simple linear regression models, multiple models had better prediction ability. In addition, the SMLR model using SC, A 2 , and S had the worst performance in both the calibration set and validation set. Furthermore, the highest correlation coefficient r p was observed in the BPNN model based on EI, A 2 , and S (r p =0.914; RMSEP=0.491 N/mm).