A new probability forecasting model for cotton yarn futures price volatility with explainable AI and big data

Cotton, cotton yarn, and other cotton products have frequent price volatility, increasing the difficulty for industry participants to develop rational business decision plans. To support cotton textile industry decision-makers, we apply data mining methods to extract the main influencing factors affecting cotton yarn futures prices from big data and build a probabilistic forecasting model for cotton yarn price volatility with uncertainty assessment. Based on Explain-able Artificial Intelligence (XAI) and data-driven perspectives, we use the LassoNet algorithm to extract 18 features most relevant to the target variable from the massive data and visualize the importance values of the selected features to improve the reliability. Moreover, by combining conformal forecasting (CP) with quantile regression (QR), the uncertainty measure of the point estimation results of the long and short-term memory (LSTM) model is applied to improve the application value of the model. Finally, SHAP (SHapley Additive exPlanations) is introduced to analyze the SHAP values of the input features on the output results and to explore in depth the interaction and mechanism of action between the input features and the target variables to improve the explainability of the model. Our model provides a “ big data-forecasting model-decision support ” decision paradigm for real-world problems.


| INTRODUCTION
Forecasting plays a crucial role in the domain of operational research.It allows businesses to anticipate future trends and optimize their strategies and resources, thereby improving their competitiveness and reducing potential losses.In the field of operational research, forecasting is used to make informed decisions about production lines, logistics, credit analysis, and supply chain management, helping companies achieve maximum efficiency and cost-effectiveness (Chai et al., 2024;Fan et al., 2023;Karimi & Zaerpour, 2022;Sarlo et al., 2023;Sroginis et al., 2023).Furthermore, forecasting is of utmost importance for the cotton textile market, as it enables market participants to make informed decisions regarding production, procurement, and other aspects based on expected market trends (Wang, He, et al., 2021).Cotton yarn, as an important product in the cotton textile supply chain, is crucial to the operation of upstream and downstream industries (Venkataraman et al., 2016).Considering this crucial role, the study will delve into the forecasting of cotton yarn futures prices as a case study, providing more decision support for decision-making in the field of operations research and market participants.
The cotton textile industry is a deeply rooted basic industry in China.China is the largest producer and exporter of textiles and apparel (Muhammad et al., 2021).According to data from the National Bureau of Statistics and the World Trade Organization (WTO), in 2020, the operating income of large-scale enterprises in the cotton textile industry reached 967.95 billion Yuan, accounting for 21.4% of the national textile industry.The total profit amounted to 33.65 billion yuan, representing 16.3% of the national textile industry.Textile and apparel exports reached 296 billion US dollars, holding a substantial share of 36.9% in the global textile export market, maintaining its position as the world's leading exporter.In recent years, the uncertainty of the cotton textile industry has greatly increased because of the dual pressure of international geopolitical conflicts, volatility in commodity prices, slow global economic recovery, as well as the unsmooth operation of supply chains, and the need for further recovery in the domestic consumer market.This has led to violent volatility in commodity prices (Muhammad et al., 2021), such as cotton and cotton yarn, in the industrial chain.The price volatility of cotton products will not only affect the normal operation of the cotton textile market but may also affect other markets, thereby affecting the healthy operation of the entire national economy.Therefore, it is necessary and meaningful to article the volatility pattern of cotton yarn prices.With the rapid development of the big data era, data mining and AI technology can reveal potential distribution patterns from the huge amount of data (Bouteska et al., 2023;Jalota et al., 2023;Meng et al., 2022), so this paper follows the data-driven decision paradigm of "big data-forecasting model-decision support" to provide decision support for OR stakeholders and market participants.
Cotton yarn futures price volatility forecasting is an important area of operational research.By analyzing and modeling various factors such as cotton cultivation, harvesting, processing, and sales, researchers can forecast the trends in cotton yarn prices (Xu & Zhang, 2022).This is crucial for both textile manufacturers and investors as it provides key information about future market changes, supporting more informed decision-making.For textile manufacturers, the ability to adjust raw material procurement strategies based on anticipated price forecasting is paramount.This allows them to minimize the risks associated with high-cost purchases or shortages caused by low prices.In dynamic market conditions, manufacturers can adapt more flexibly to supply and demand forecasting, enhancing production efficiency, ensuring stable operation of production lines, and effectively reducing production costs.Investors, on the other hand, rely on these forecasting insights as crucial references for making informed trading decisions.They can dynamically adjust their investment portfolios based on market expectations, seize favorable trading opportunities, and consequently achieve higher returns.For investors, a deep understanding of the trends in cotton yarn futures prices signifies a better grasp of market dynamics, facilitating more prudent, data-driven investment decisions (Marfatia et al., 2022).
In order to reduce uncertainty, scholars have conducted multi-angle research on commodity price volatility, but relatively less in the cotton textile field.Current research mainly relies on methods such as autoregressive moving average (ARMA) models, back-propagation (BP) neural networks, and basic machine learning models.There is limited research that utilizes deep learning methods (Jiang et al., 2021;Venkataraman et al., 2016;Wu et al., 2018).Additionally, cotton yarn price prediction predominantly focuses on accurately forecasting the price for a specific future day rather than predicting the distribution range of cotton yarn prices.For instance, Venkataraman et al. (2016) combined ARMA and K-nearest neighbors (KNN) to predict cotton yarn prices.In this research, the ARMA algorithm was used to forecast the influencing factors of cotton yarn price fluctuations, such as cotton prices, yarn production, and other features' future prices.Subsequently, the KNN algorithm utilized the predicted values of these influencing factors to forecast future cotton yarn prices.Wu et al. (2018) collected 13 factors influencing cotton yarn price fluctuations and investigated the predictive performance of the mean impact value, genetic algorithms, and BP neural networks in predicting cotton yarn prices in China.They concluded that the proposed predictive model demonstrated high accuracy.In comparison to forecasting studies in the financial domain, the development of cotton yarn price prediction has been relatively slow (Du et al., 2022;Hu et al., 2020;Li et al., 2017;Vidal & Kristjanpoller, 2020).The existing research on price volatility mainly includes two aspects: (1) analysis of influencing factors and (2) price volatility forecasting.In terms of analyzing influencing factors, econometric methods are mainly used (Guo et al., 2019;Karasu et al., 2020).The above methods involve only a few variables and cannot fully explore the influencing factors of the target variable from a big data perspective (Besbes & Mouchtaki, 2023).In terms of constructing volatility forecasting models, most existing research is based on point forecasting at the numerical level, that is, providing accurate forecasting results (Herwartz, 2017;Hu et al., 2020;Jabeur et al., 2021;Kim & Won, 2018;Lee & Ryu, 2019;Li et al., 2019;Tschora et al., 2022).From the results of existing articles, we find that point forecasting models, whether they are econometric models or deep neural network models that can handle complex information, have some degree of discrepancy between their forecasting results and actual values.There are two reasons for this finding.First, the market environment is complex and changeable, and there is a large amount of uncertainty that cannot be eliminated by quantification.Second, when collecting and processing data, we started from a big data perspective and collected the factors related to the target variable as much as possible, but there is still a gap with the actual influencing factors (Hüllermeier & Waegeman, 2021;Jalota et al., 2023).From the analysis of the above two aspects, we can know that precise and fixed-point forecasting results are unreliable and difficult to achieve.In consideration of the limitations of point forecasting, researchers have extended the forecasting model to interval forecasting models and probability forecasting models (Baştürk et al., 2019;Hajek et al., 2020;Jensen et al., 2022;Li et al., 2022;Liu & Ma, 2022;Sun et al., 2018).Unlike point forecasting, the output results of interval forecasting and probability forecasting are not a determined point estimation value but an interval range with uncertain estimates, which not only quantifies the price volatility range but also corrects the point forecasting results.Therefore, in contrast to point forecasting, probability forecasting is more in line with the actual situation and has a higher practical value (Hüllermeier & Waegeman, 2021;Zhang et al., 2020).
In the research, for the analysis of influencing factors on cotton yarn futures prices, we combined residual neural networks with regularization (Lasso) from a datadriven perspective to construct a LassoNet algorithm that can handle complex, nonlinear regression.The algorithm was used to extract 18 most relevant factors from a collection of 148 potential influencing factors.Different from the commonly used principal component analysis (PCA) feature extraction method, LassoNet returns specific features, not only extracts features but also provides importance scores of the features and visualizes them, thus improving the interpretability of the research (Jalota et al., 2023;Lemhadri et al., 2021).In terms of analyzing price volatility patterns, we proposed a CP-QR-LSTM probability forecasting model, which combines CP and QR to calibrate the point estimator output by LSTM, generating a forecasting interval with a given confidence level.This mixed model quantifies the uncertainty of point estimation results, and the final output is the distribution range of the point estimation value under certain probability conditions.Specifically, the primary focus of this article is to address the following research questions: RQ1: What is the pattern of cotton yarn futures price volatility driven by data (three types of data)?
RQ2: What is the relationship between data empowerment of XAI and cotton yarn price futures volatility?
To address these research questions, we propose a data-driven research paradigm based on big data and XAI, which is called the "big data-forecasting modeldecision support" framework.First, to identify the factors that influence cotton yarn feature prices, we use the LassoNet algorithm to extract highly correlated feature variables from massive datasets related to the target variable.The important scores of these features were visualized to help OR stakeholders and market participants to understand and analyze the reasons for cotton yarn futures price volatility.Second, the CP-QR-LSTM probability forecasting model was constructed to explore the patterns of cotton yarn futures price volatility under data-driven contexts.This model provides uncertain probability measurements of future cotton yarn futures price distribution.Finally, after the model training, we introduce the SHapley Additive exPlanations (SHAP) algorithm to quantify the contribution of each input feature to the LSTM point estimate results.On this basis, we further reveal the interaction relationship between feature variables and the target variables to further improve the explainability of the proposed model.It aims to provide reasonably accurate price volatility forecasting schemes for the cotton textile industry and related enterprises, helping stakeholders in the cotton textile market make decisions and plans in highrisk and high-uncertainty markets and enhancing their competitiveness.
The other sections of this article are organized as follows: first, a literature review and research framework were established; second, the required data for model validation was collected, and the model was tested; third, important findings were drawn; finally, a summary and discussion were presented.

| LITERATURE REVIEW AND RESEARCH FRAMEWORK
In this article, we will answer the proposed research questions through two main themes: price volatility forecasting and explainable AI.In this section, we will discuss the existing literature on these two themes and propose our own research framework.

| Research on price volatility
The forecasting of commodity price volatility has received extensive research attention in recent times in operational research and management science (Chen et al., 2021;Karimi & Zaerpour, 2022).In literature, predictive models can be broadly classified into three categories, as shown in Table 1.
Initially, traditional econometric models (Herwartz, 2017) were widely used for price forecasting.These models typically analyze linear relationships between variables based on strict assumptions and piecewise functions.However, with a better understanding of the nonlinear relationships between factors that influence commodity prices and their financial properties, it has been shown that traditional econometric models are unable to meet the needs of highly complex datasets (Chen et al., 2021;Hajek & Abedin, 2020;Ma et al., 2019;Svetunkov et al., 2023).
On the other hand, machine learning models have deep network hierarchies that can handle nonlinear and complex structures, learn from data, and reflect actual commodity price volatility patterns.Prior articles (Li et al., 2017;Li et al., 2019) combine LSTM and convolutional neural networks (CNN) to explore the impact of unstructured data on oil and stock price volatility trends.Results find that LSTM and CNN are superior to traditional linear methods in terms of forecasting accuracy and performance.However, they only compared single models and not hybrid models.
Finally, hybrid models, which combine econometric and machine learning models (Kim & Won, 2018;Venkataraman et al., 2016), artificial neural networks and machine learning (Hu et al., 2020), or multiple machine learning models (Svetunkov et al., 2023;Vidal & Kristjanpoller, 2020), have shown significant improvements in forecasting performance compared with single models.However, existing articles have some limitations.Many articles focus on point data, such as predicted closing prices (Jabeur et al., 2021;Liu & Ma, 2022) and volatility (Vidal & Kristjanpoller, 2020) without exploring the uncertainty of predicted results, leading to weak robustness and effectiveness of the models.
Considering these limitations, few articles have started expanding forecasting objects to include interval data (Hajek et al., 2020;Kang et al., 2022;Li et al., 2022), functional data, and density distribution (Baştürk et al., 2019), among other fields.In the limited interval forecasting or probabilistic forecasting literature, most articles granulate the original time series datasets directly into interval datasets based on fuzzy granulation theory (Goltsos et al., 2022;Sun et al., 2022), and the quality of granulation is difficult to measure and prone to large errors.There are also articles based on Gaussian regression and quantile regression, among others (Liu et al., 2015;Panagiotelis et al., 2023;Wang, Wang, et al., 2021), but this type of article is weak in terms of coverage and cannot meet the decision-making needs.

| eXplainable Artificial Intelligence (XAI)
With the growing demand for XAI techniques, many different studies have recently been conducted, which can be mainly divided into two types: (1) model explainable and explainable and (2) model uninterpretable but explainable, as shown in Table 2. Model explainable and explainable techniques focus on constructing models that are inherently transparent and understandable.This means that the internal workings of the model are designed to be user-friendly, allowing users to delve into how the model arrives at specific predictions (Jiang et al., 2024;Lisboa et al., 2023).On the other hand, model uninterpretable but explainable techniques acknowledge that the underlying model might be complex or a "black box."These methods generate explanations after the model has made predictions, revealing why certain decisions were made without necessarily disclosing the complexity of the model (Dwivedi et al., 2023).Existing literature on explainable models mostly uses federated learning and rule extraction methods.For example, researchers embed complex "black-box" models into decision trees or regression trees that are inherently explainable by using techniques such as federated learning and rule extraction to extract explainable regression and classification models from "black-box" models and demonstrate the effectiveness of extracting models on 20 public datasets (Haffar et al., 2023;Johansson et al., 2022).However, the aim of this article on explainability is to explore the important factors affecting volatility in cotton yarn futures prices.Therefore, on the interpretability level, we will focus on the uninterpretable but explainable models.Common methods of XAI techniques in this category include (1) SHAP and (2) Local Interpretable Model Agnostic Explanations (LIME).In this context, the SHAP method is based on cooperative game theory and aims to fairly distribute the "value" of each feature among the contributing features in a prediction.It provides a way to attribute the contribution of each feature to a specific prediction instance, offering a comprehensive understanding of how each feature influences the model's output.SHAP values highlight the impact of each feature on the target variable, revealing whether a specific feature contributes positively or negatively to the prediction (Lundberg & Lee, 2017;Yang et al., 2023).On the other hand, LIME generates local explanations by

Classification
Research methods Research content Reference

Federated learning
Based on Federated Learning technology, we can construct a surrogate model of black-box models and use the combination of these two models to interpret the black-box model.

Rule extraction
We can use explainable models with weaker functionality to approximate opaque models and extract explainable regression models from these opaque models.

SHAP
The SHAP method is proposed to verify the benefit of various gold prices.

Jabeur et al., 2021
Results show that SHAP are beneficial for the collaboration between human workers and artificial intelligence.training a locally simplified model around a specific instance.It perturbs input data and observes changes in the predicted outcomes, thus understanding the impact of different features on the model's decision for a specific instance.This approach makes it easy to comprehend the local behavior of the model (Dwivedi et al., 2023).As shown in Table 2, SHAP's biggest advantage is its ability to reflect the degree of influence of features and indicate their positive or negative impact.Researchers have used SHAP to visualize the Shapley values of different input features to help decision-makers understand the forecasting of complex machine learning (Jabeur et al., 2021;Yang et al., 2022).Results show that the combination of machine learning methods with SHAP can significantly improve model explainability, but the computational cost of SHAP is high when there are many features.Results have found that the LIME method can improve the overall explainability of models, but it relies on an alternative linear model, explaining the local impact of each variable through multiple random perturbation terms of instances, and has high instability (ElShawi et al., 2021;Pan & Stark, 2022;Wang et al., 2022).The methods mentioned above can improve model explainability to varying degrees, but they are all performed after model training.In actual situations, there are many factors affecting cotton yarn futures prices, so there are many features in this article.Directly inputting these variables into the forecasting model will surely increase computational costs.Traditional feature extraction methods based on linear regression, such as Lasso, Pearson correlation coefficient, and PCA, have limitations in handling nonlinear problems (Fan et al., 2014;Lockhart et al., 2014).First, the above traditional feature extraction methods generally target linear problems and perform poorly on nonlinear problems.Second, the returned values after feature processing are not individual features (such as PCA), which makes commonly used explainable methods such as SHAP and LIME ineffective.
Therefore, this paper will use LassoNet and SHAP algorithms to explain the causal relationships between the feature variables and the target variables before and after model training, respectively.First, the LassoNet feature sparsity method was introduced before model training to analyze the degree of influence between the target and feature variables.LassoNet not only meets the demand for feature extraction but also reduces computational effort.Moreover, the algorithm is also able to calculate the importance scores of the input variable, quantify the importance of each variable on the target variable, and improve the interpretability of the whole article.Second, after the model training, the SHAP algorithm is introduced to analyze the marginal contribution rate of the input features to the forecasting results.

| Research framework
In the big data scenario, there are numerous and dynamically variable factors affecting cotton yarn futures price fluctuations, resulting in large volatility in the cotton yarn futures market.Considering that it is impossible to collect all the data related to cotton yarn futures prices in actual research, this paper provides a data-driven research paradigm to explore the price volatility of cotton yarn futures in this dilemma, that is, "big data-forecasting modeldecision support."The framework of this article is shown in Figure 1.
In this article, we divide influencing factors into three types of data from the big data perspective: trading data, measurement data, and interaction data, which contain characteristic variables that are dynamic.Second, XAI is introduced to extract the typical features affecting the fluctuation of cotton yarn futures prices interpretably from the dynamically changing feature variables, which do not change with the amount of original data, and thus can reflect the fluctuation pattern of cotton yarn futures prices to a certain extent.Next, the CP-QR-LSTM model is built to extract the fluctuation pattern of cotton yarn futures from the extracted typical features and predict the distribution of cotton yarn futures prices in the future period.Because of the incompleteness of the data collected in the context of big data, the proposed model gives a range of future price distributions at different confidence levels to improve the generalization and stickfiltering ability of the model.The final model will produce the volatility range of cotton yarn futures prices under certain confidence level conditions to provide decision support for OR stakeholders and market participants.

| Data collection
In order to explore the volatility pattern of cotton yarn prices, we selected the daily closing prices of cotton yarn futures (CCYc1_C) as the research target, which were obtained from the Zhengzhou Commodity Exchange (www.czce.com.cn).Research shows that cotton yarn, as the most direct downstream product of cotton, is not only affected by the supply and demand relationship in the cotton textile market.It also has financial and political attributes (Besbes & Mouchtaki, 2023;Huchet & Fam, 2016).To better simulate the cotton yarn futures trading environment, we collected 148 feature variables that may affect the volatility of cotton yarn futures prices from three aspects: commodity attributes, financial attributes, and political attributes.Our data collection is not only limited to trading data but also includes interaction data and measurement data.The time span of the data is from August 21, 2017, to March 17, 2023, for a total of 301,180 records.Table 3 shows the descriptive statistical analysis of some of the data.In addition, we summarize data classification and source information in Table 4.

| Data analysis
To achieve the forecasting target and make the collected datasets match the model, three steps of data pre-processing are performed before model validation, as shown in Figure 2.Moreover, we introduce the LassoNet algorithm and SHAP algorithm to analyze how features affect the forecasting results from both data and feature perspectives, respectively, during the experiment.In this section, only data-based XAI is covered.Featurebased XAI will be elaborated on in Section 4.3.

| Data analysis
Missing value imputation stage Through analysis of the original data, we found that missing values are generally caused by holidays.In actual trading, the problem of missing data caused by holidays is generally supplemented based on the previous day's trading data.Therefore, in this article, the upward-filling method will be adopted to fill in the missing values in the original datasets.

Normalization stage
The dimensional and dimensions of various features in the original datasets are different, which leads to imbalanced weights of feature variables in the model, which will affect the accuracy of the research.In order to reduce that influence, the StandaraScaler class is used to normalize the feature values to the same dimension before modeling.The mathematical principle of the normalization process is shown in Formula (1).
where x denotes the mean of the feature value and σ denotes the standard deviation of the feature value.
After normalization, the mean of the feature value in the original datasets is 0, and the standard deviation is 1.

Feature selection stage
In many research problems, there are some feature variables in the original datasets that are unrelated to the forecasting target.Feature selection can not only achieve a deep understanding of the relationship between input features and forecasting results before model validation and improve the interpretability of the model but also eliminate redundant or noisy features, thus reducing computational costs and improving the generalization ability of the model.Considering that there is a nonlinear relationship between cotton yarn futures prices and feature variables, traditional feature extraction methods based on linear regression, such as Lasso, Pearson correlation coefficient, and PCA, have limitations in dealing with nonlinear problems (Fan et al., 2014;Lockhart et al., 2014).Therefore, we introduced an arbitrary feed-forward network and residual layer into the Lasso regression model, forming a feature extraction model, LassoNet, that can handle highly nonlinear regression (Lemhadri et al., 2021).Moreover, the process of Lass-oNet feature extraction is also the process of analyzing the causal relationship between input features and forecasting results, and the specific implementation process is described in Section 3.2.2.

| Data-based XAI
We use LassoNet as an interpretable technique for the model, which can mine the features with strong influence on the target variables from a large amount of data, give explanations, and provide strong support for decision-making.The LassoNet algorithm is achieved by combining Lasso and neural networks, as shown in Figure 3, where the green part represents the residual layer, and the black part represents a feed-forward network.It can effectively select the features most relevant to the output variable and shrink the weights of other features to zero.In addition, the LassoNet algorithm can rank the importance of each feature by training the datasets, calculating the weight of each feature, and visualizing the results.
In addition to the target variable of cotton yarn futures prices, we collected 147 exogenous variables.If all variables are input into the model, it would be difficult to identify the features that significantly impact results.LassoNet utilizes Lasso to reduce model complexity while retaining important features, which can not only reduce training time and computational costs but also effectively identify the key variables in the input data.The working principle of LassoNet is shown in Equation (2).
In the formula, L θ, w ð Þ¼ nonlinear components.When M !0, LassoNet reduces to linear regression with Lasso regularization, whereas when M !∞, it becomes a standard feed-forward neural network.Algorithm 1 presents the working mechanism of LassoNet.
In our article, we set M ¼ 10 and obtained the relationship between candidate feature variables and loss values, as shown in Figure 4. Experimental results show that when extracting 18 features from the original datasets, the loss value can be minimized.Using more than 18 features decreases the accuracy of the experimental results.Therefore, we chose 18 features from the original datasets as input features in our article.
In the above experiments, we concluded that loss minimization could be achieved when 18 features are input, but it is impossible to know exactly which 18 features.Moreover, the feature-based explainable technique can solve this problem.LassoNet can compute the absolute values of the input feature coefficients to determine their importance, with higher coefficients indicating greater importance.By leveraging this approach, we can identify the features that have the greatest impact on the target variable and visualize them, which is critical for data interpretation and deeper analysis.Since the original dataset contains 148 features in total, it cannot be clearly displayed on a single image, so we only show the top 18 sorted feature variables in terms of importance, as shown in Figure 5.
From the feature importance ranking graph, it can be seen that the factors affecting cotton yarn futures price volatility are quite complex, including not only cotton yarn futures prices themselves but also the prices of their upstream and downstream products (e.g., CCFc1_c and CTAc1_C), as well as macroeconomic indicators (e.g., LCOK3_C and SZI_F) and market sentiment indicators (VIX_L, ETU_SCA, etc.).

| Construction of probability forecasting models
In this article, we combined CP (Lei et al., 2018), QR (Liu et al., 2015), and LSTM (Du et al., 2022) to develop a hybrid probability forecasting model called CP-QR-LSTM, which was applied to investigate the volatility patterns of cotton yarn futures prices.The CP-QR-LSTM model combines the strengths of CP, QR, and LSTM (Jensen et al., 2022), enabling it to handle the hidden complexities in time series data while providing reliable and effective interval forecasting with confidence levels.Furthermore, since the model takes historical errors into account and dynamically updates the threshold needed for quantile forecasting, it can deceptively handle potential Heteroskedasticity or noise issues (Schecter et al., 2022).The research framework is illustrated in Figure 6.
In the above framework, the LSTM model learns hidden features in time series data using the back-propagation algorithm and provides point estimates corresponding to a one-time step into the future.These point estimates can provide some time series trend information, which is often misunderstood as precise values.However, in practical forecasting tasks, precise point forecasting cannot be achieved.We hope to provide a reliable confidence interval while giving point forecasting to reflect the uncertainty of forecasting results (Schecter et al., 2022).In this article, we trained three CP-QR models simultaneously and combined them into a single model (named CP-QR model) to calibrate the point estimates output by the LSTM model and generate forecast intervals that satisfy the specified confidence level.Next, we will introduce the working principle of CP-QR in detail.
CP is a non-parametric method for predicting confidence levels, which can ensure that predicted results are effectively covered within a confidence interval (Lei et al., 2018).The mathematical principle of the CP algorithm is as follows.First, the given datasets X R d are divided into three datasets, which are the LSTM point estimation training set ξ 1 , the calibration set ξ 2 , and the test set ξ 3 .For results forecasted Y nþ1 by d h lstm in ξ 1 , we require a forecast interval c C α x ð Þ to satisfy the following equation.
where α is the miscoverage rate, R is residual set in ξ 2 , and Q 1Àα R,ξ 2 ð Þis the 1 À α ð Þth quantile of R. QR is a quantile regression method that models uncertainty in predicted results and can deal with heteroskedasticity data exhibiting local variability (Liu et al., 2015).First, the purpose of QR is to estimate the conditional quantile function of Y given X at a specified α, which is defined as By utilizing the advantages of these two algorithms, the proposed CP-QR algorithm is a calibration method based on confidence levels and quantile regression, which can perform conditional probability analysis on the distribution of forecasting results of the LSTM point forecasting model.The forecast interval c C α x ð Þ of CP-QR-LSTM satisfies the following equation.Algorithm 2 is the pseudo-code of the CP-QR-LSTM model.

| Model validation
In order to demonstrate the effectiveness and accuracy of the proposed CP-QR-LSTM model, this section validates the model in two stages, as shown in Figure 7.In the first stage, we visualize the results of the proposed probability forecasting model and the point forecasting (LSTM-Attention) model to verify the effectiveness of the model.In the second stage, the proposed model is compared with the CP-QR-TCN (TCN: Temporal Convolutional Network) probability forecasting model and the CP-QR-RF (RF: Random Forest) probability forecasting model to validate the accuracy of the model.

| Validity test
In this article, we use mean absolute percentage error (MAPE), root mean square error (RMSE), and mean absolute error (MAE) as evaluation metrics.The ranges of these three indicators are all (0, +∞), and the larger the indicator value, the worse the model's performance.The calculation formulas are shown in Equations ( 8), ( 9), and (10), respectively.
where b y i is the forecast value, y i is the real value, and m is the total amount of data.
In this article, we built an LSTM model with an attention mechanism as the point forecasting model, and its quantitative forecasting results are shown in Table 5.The forecasting horizon in Table 5 represents the forecasting step length.Without loss of generality, we analyze the forecasting errors for five different forecasting step lengths: 1, 7, 14, 21, and 28.The final average of the errors of five different steps is calculated as the evaluation criterion.Here, in order to test the effectiveness of data mining in Section 3.2, we input the missing values and the original standardized dataset into the model for comparison tests.From Table 5, we conclude that the features mined by the LassoNet algorithm can better reflect the fluctuation pattern of cotton yarn futures prices, which greatly improves the accuracy of models.Nevertheless, the point forecasting model still has a large inaccuracy.
In order to intuitively verify the effectiveness of probability forecasting, we visualize experimental results of LSTM-Attention and CP-QR-LSTM, as shown in Figures 8  and 9. From Figure 8, we can see that the predicted values and actual values do not completely overlap, and there is even a significant error between the two around Time = 100, which leads poor robustness of the model.The reason for the above problem is due to the inherent limitations of point forecasting, which generates only a deterministic result and does not consider the uncertainties caused by complex market environments, data collection, and data processing, thus failing to provide uncertainty for the predicted results (Chen et al., 2023;Hüllermeier & Waegeman, 2021;Mullins & Sabherwal, 2022).Unlike point forecasting, the CP-QR-LSTM model delves into the distribution of point estimates generated by LSTM and calculates the forecasting interval at a given confidence level.Figure 9 shows the forecasting results at a confidence level of 90%, and the model's results are not a fixed value but rather within a certain range, providing an uncertainty measure for the predicted results.Therefore, in practical applications, probability forecasting has higher effectiveness, practical value, and potential than point forecasting.

| Accuracy test
The forecasting interval coverage probability (PICP) and normalized average width (PINAW) were selected as evaluation metrics for CP-QR-LSTM in this article.Their calculation formulas are shown in Equations ( 11) and ( 12), respectively.In this article, we aimed to ensure that the PICP reached the confidence level we set while minimizing the execution interval width, that is, minimizing the PINAW as much as possible. where where R ¼ y max À y min .
As shown in Table 6, in order to verify the accuracy of the CP-QR-LSTM model, we compared it with the CP-QR-TCN and CP-QR-RF models.To reduce the randomness of the experimental results, we conducted experiments under the conditions of 90.0% and 95.0% confidence intervals, respectively, and took the average value of multiple experimental results as our basis for comparison.From Table 6, it can be seen that all three models are able to reach the corresponding confidence level, which is due to the introduction of the CP (conformal forecasting) algorithm in the models, which improves the coverage rate of the models.Because the LSTM algorithm has advantages in capturing long-term dependencies and nonlinear modeling of time series, the proposed model is significantly better than the other two comparison models in terms of coverage width.Specifically, under the condition of a 90.0%confidence interval, the coverage width of CP-QR-LSTM is reduced by about 61.43% compared with CP-QR-TCN and by about 44.31% compared with CP-QR-RF; under the condition of a 95.0% confidence interval, the coverage width of CP-QR-LSTM is reduced by about 44.13% compared with CP-QR-TCN and by about 3.09% compared with CP-QR-RF.Therefore, results show that the CP-QR-LSTM model has high accuracy in predicting the volatility patterns of cotton yarn futures prices.

| Feature-based XAI
SHAP approximates the importance of local features by using the Shapley values defined in cooperative game theory (Lundberg & Lee, 2017), and it interprets the results of models by evaluating the marginal contribution margin to the result of the model while visualizing the results.SHAP works as shown below.Moreover, 2 presents the working mechanism of SHAP.SHAP interprets the model output by calculating the Shapley value for each feature, as shown in Equation ( 13).
The Shapley value φ j of j can be calculated for all possible combinations of features and then weighted and summed, as shown in Equation ( 14): where s is the subset of input features, x is vector of instance eigenvalues, p is feature's number, val x s ð Þ is the output of s, and integral is calculated for features not included in the s, as shown in Equation ( 15).And Algorithm 3 is the pseudo-code of SHAP in this article.The remainder of this section introduces the SHAP algorithm for feature interpretable analysis of the point estimation part of the CP-QR-LSTM.
First, Figure 10 shows the SHAP feature importance analysis chart, which ranks the SHAP values of the factors affecting cotton yarn futures prices according to their elements.In Figure 10a, the ranking is based on the magnitude of the absolute value of SHAP values, Figure 10b directly shows the degree and direction of the influence characteristics of each instance.From Figure 10, it can be found that the differences in the characteristics of CCYc1_U (highest of cotton yarn futures), CCYc1_L (lowest of cotton yarn futures), and CCYc1_O (opening of cotton yarn futures) have a more obvious impact on the model.This is the same as the top three important features calculated by LassoNet in Section 3.2.2. Figure 10b indicates that all three main features have a positive effect on CCYc1_C (closing of cotton yarn futures).When these characteristics have larger values, they correspond to higher SHAP values, which in turn are associated with higher CCYc1_C.
Compared with the three types of features mentioned above that have more significant effects, the effects of other features are difficult to observe visually in Figure 10.Therefore, we plot the interaction dependence of the two features by changing the specific features in the SHAP algorithm.Figure 11a,b shows the interaction dependence plots of CCFc1_L (lowest of cotton futures), FEST_O (FTSE 100 opening), and CCYc1_L, where the red dots represent the higher values of CCYc1_L (lowest price of cotton yarn futures), where the red dots represent the higher values of CCYc1_L and the blue dots represent the lower values of CCYc1_L.When the values of CCFc1_L and FEST_O increase, CCYc1_L increases, and the SHAP value also increases, indicating that CCFc1_L and FEST_O indirectly affect the CCYc1_C by interacting with CCYc1_L, and the effect is positive.Similarly, Figure 11c,d   Figure 12 shows the marginal contribution of the main features to the predicted values using SHAP analysis for the 23rd randomly selected estimate after the training of LSTM point estimation.
We can see from Figure 12 that the baseline value (mean predicted value) for the 29th sample is 22935.78.All the red bars on the left side of the model output represent features that have a positive effect on the deviation from the baseline value (mean predicted value), including INR/CNY_U, ....., and CCY1c1_U.Each of these features causes an increase in prediction, and the length indicates the extent of the effect.Similarly, all blue bars on the right side of the model output represent features that have a negative impact on the deviation of the predicted value from the baseline value.The length of the bar indicates the contribution of that element, indicating that CCYc1_L is the feature that causes a decrease in prediction results, with the largest effect originating at CCYc1_l = 22545.0.
Consequently, we can clearly understand the extent and direction of input features on the forecasting results, which significantly improves the interpretability of the "black box" model.

| Robustness test
To validate the strong generalization capabilities of the proposed probability forecasting model, we selected an open-source dataset from the meteorological domain to assess the model's predictive performance.The descriptive statistics for some variables in the datasets are presented in Table 7, where T (degC) represents the prediction target.
As the dataset is collected on an hourly basis, in our study, we set a sliding window of 24 h to predict the temperature change range within the next 6 h.The experimental results are presented in Table 8 and Figure 13.Table 8 displays the evaluation results of the model constructed in this study at 90% and 95% confidence levels.It is evident from the table that the model exhibits good coverage in meteorological forecasting, with relatively narrow prediction intervals.
Figure 13 illustrates the visual results of the testing set at a 90% confidence level, including actual values, point estimates, and prediction intervals.From the graph, we observe that the model developed in this research demonstrates applicability in meteorological forecasting with effective coverage.

| Theoretical contribution
The probability forecasting model (CP-QR-LSTM) proposed in this paper is used to quantify the volatility  patterns and uncertain distributions of cotton yarn futures.Based on the self-properties of cotton yarn futures prices and a data-driven perspective, we collected 148 possible factors that may affect the target variable forecasting and extracted 18 effective features for model input using XAI techniques.Combined with existing research, our article has the following theoretical significance.
The contributions of this article are significant for forecasting in the field of operational research.Accurate forecasting is essential for making informed decisions and optimizing resources in various fields such as finance, economics, logistics, and supply chain management.The background and significance of forecasting in operational research are related to its ability to predict future outcomes based on past data and trends, helping decision-makers plan for future scenarios and minimize risks.The contributions theory of this article can be summarized in four points: (1) the data processing stage selects the LassoNet algorithm for explainable analysis.The LassoNet algorithm is based on L1 regularization and residual neural networks, which can effectively screen features and provide variable importance rankings and explanations.This modeling idea aims to screen out the variables most relevant to the dependent variable among many independent variables in the cotton yarn market data, thereby improving model interpretability.This feature selection approach also has promotional significance in practical applications.(2) Using CP-QR-LSTM to analyze the uncertainty of future cotton yarn futures price distributions.LSTM can obtain high-precision point forecasting results, but when a reliable forecasting range is required, it is difficult to support probability forecasting.Therefore, in this article, combining the CP-QR algorithm transforms the LSTM forecasting results into a probability distribution form with a confidence level.In particular, our proposed probabilistic forecasting framework can be applied not only to the LSTM point estimator built in this paper but also to point forecasting models from different domains in existing articles, which makes a small contribution to advancing the development of probabilistic forecasting methods based on deep learning models.This article also provides decision-makers with more accurate and reliable forecasting ranges, which can inform optimal resource allocation and risk management strategies.(3) The introduction of the SHAP algorithm to analyze the influence of features on forecasting results significantly improves the explainability of models.This is important in operational research because it allows decision-makers to understand how different factors contribute to forecasting results and make more informed decisions based on the insights provided by these models.(4) This paper provides a "big data-probabilistic predictive model-decision support" data-driven decision paradigm from the perspective of big data and XAI, which provides a reference for decision-making in big data scenarios.This approach is highly relevant to operational research because it enables decision-makers to leverage the huge amounts of data available to them to make more accurate and informed decisions.The integration of probabilistic forecasting and XAI methods in this paradigm offers decision-makers a transparent and reliable way to understand how different factors contribute to forecasting results and use these insights to inform their decisions.

| Practical value
We summarize the practical value of this article from three perspectives: people, organizations, and technology.
From the people's perspective, this paper provides decision support for stakeholders in the cotton textile industry.Stakeholders in the cotton textile industry and related fields can find real value in the proposed models.By providing more accurate and reliable predictions of price volatility, these models can help stakeholders, including shareholders, investors, and regulators, make informed decisions and manage risk in a volatile market environment.LassoNet, CP-QR-LSTM, and SHAP models, through their feature selection, interpretable enhancement, and explainable analysis, enable stakeholders to understand the factors that contribute to price volatility and adjust their strategies accordingly.Furthermore, the integration of big data and artificial intelligence techniques into price volatility prediction provides stakeholders with a more modern and effective approach to decision-making and risk management.
From the perspective of organizations, this article provides a more reasonable price volatility forecasting scheme for the cotton textile industry and related enterprises, helping them make decisions and plans in highrisk and high-uncertainty markets and enhancing their competitiveness.In terms of optimizing enterprise decisions, the proposed probability forecasting model can predict volatility that is more application-oriented than traditional point forecasting methods, reducing the serious consequences of incorrect decisions for enterprises.In terms of improving enterprise competitiveness, as the market competition intensifies, cotton textile industry participants need to adjust their decisions based on market volatility.Through feature selection, interpretability enhancement, and other means, the model proposed in this article enables enterprise managers to effectively understand market changes and formulate scientific and reasonable decisions, thereby enhancing the core competitiveness of enterprises.In terms of digital transformation and intelligent management of enterprises, the predicted results of the proposed model can provide important references for large-scale data analysis and management by enterprises, which is conducive to the in-depth development of digital strategies.Introducing modern computer technology into price volatility forecasting not only improves decision-making quality and direction but also responds to changes more accurately and quickly, which is beneficial to shorten enterprise response time and enhance market sensitivity.
From a technical perspective, this article combines LSTM deep networks with CP-QR algorithms to provide a new and more accurate price volatility forecasting model, utilizing new technologies such as big data and artificial intelligence to their fullest potential, possessing strong practical value.The proposed model can not only be applied to cotton yarn futures prices but can also be generalized to other datasets with time series features.
Overall, this article provides practical value not only for individuals and organizations in the cotton textile industry but also for the various stakeholders involved in the operation and development of the industry.

| CONCLUSION
In the context of big data, the volatility of cotton yarn futures prices exhibits characteristics such as high dimensionality, complexity, and nonlinearity.Additionally, existing machine learning prediction models often lack interpretability.Given these challenges, this study addresses two research questions: RQ1: What is the pattern of cotton yarn futures price volatility driven by data (three types of data)?RQ2: What is the relationship between XAI data empowerment and volatility in cotton yarn futures prices?
For the first research question, this study, through a review of the literature and expert experience, collected a total of 148 potential influencing factors from the perspectives of trading data, interaction data, and measurement data.The aim was to simulate the actual fluctuation environment of cotton yarn prices.Subsequently, a LassoNet network was constructed to explore the influencing factors of cotton yarn futures prices, enhancing interpretability by visualizing their importance rankings.The research integrated LSTM, CP, and QR to build a probability forecasting model, measuring the uncertainty of future cotton yarn futures price distribution.The inclusion of the CP algorithm ensured the confidence level of the model's output results.As shown in Table 7, all three experimental models, under the constraint of the CP algorithm, achieved the expected confidence levels.Notably, this study used deep learning networks as point estimators, improving the accuracy of point estimation compared to TCN and RF algorithms.This led to a reduction in the PINAW indicator value, further narrowing future uncertainty.
For the second research question, given the intrinsic lack of interpretability of the LSTM point estimator, this paper introduces the SHAP algorithm to analyze the marginal contribution of each input feature to the point estimation results.With the inclusion of the SHAP interpretability method, the model achieves a comprehensive analysis of the interaction between input features and the target variable from both a global and local perspective.This provides a high level of confidence in the predictive results.The study reveals that, in addition to historical trading data of cotton yarn futures having a significant impact on price fluctuations, factors such as CCFc1_L, FEST_O, EUR/CNY_O, and VIX_L, among others, also influence the volatility of cotton yarn prices.Their inclusion enhances the accuracy of predictions to a certain extent.
In addition, to visually compare the differences between point predictions and probability forecasts, we visualized the experimental results.As observed in Figures 8 and 9, probability forecasting demonstrates greater practical applicability.It not only provides point estimation values but also quantifies the uncertainty of their distribution, aligning more closely with human decisionmaking behavior.
A limitation of this study lies in the focus on the distribution of point prediction results when constructing the probability forecasting model.Consequently, a single model was utilized in constructing the point prediction model.Future research could consider further improvements on the point prediction model to enhance predictive accuracy.
to perform explainability analysis on the results of AI medical diagnosis.Elshawi et al., 2021 LIME technology is applied to the early stage of an engineering change to explain the reasons for the change request.After evaluation by industry experts, LIME improves the transparency of the model.Pan & Stark, 2022 LIME is used to derive and evaluate the predicted results of model forecasting, improving the transparency of the established model.Wang et al., 2022 Þrepresents the loss function and λ θk k 1 represents the Lasso.The LassoNet algorithm contains two hyper-parameters λ and M. A larger λ value leads to sparser models and M is used to control the relative strength between linear and LassoNet structure diagram.

F
I G U R E 4 The relationship between the number of features and the loss value.F I G U R E 5 Feature importance ranking.

8
Visualization of point forecasting results of the LSTM-Attention model.F I G U R E 9 Visualization of probability forecasting results of CP-QR-LSTM model.T A B L E 6 Assessment results of probability forecasting indicators.
affecting CCYc1_C, and the effect is negative.Through the feature dependence chart, we find that the factors that have a positive effect on the CCYc1_C include CCFc1_F (lowest of cotton futures), FEST_O (the opening price of FTSE 100), CTAc1_U (highest of PTA futures), 2SK3_O (opening of US soybean futures), (a) Absolute value ordering of SHAP values (b) ordering of SHAP values F I G U R E 1 0 SHAP feature importance analysis.U R E 1 1 SHAP dependence plots.LCOK3_C (closing of London Brent crude oil futures), CTc2 _L (highest value of US cotton futures No. 2 [CTc2]), IXIZ_L (lowest value of the Nasdaq Composite), and INR/CNY_U (highest value of the Indian rupee/yuan).Features that have a negative effect include EUR/CNY_O (EUR/CNY opening value), VIX_L (VIX Panic Index minimum), TEU_SCA (Twitter EPU [TEU_SCA]), SZI_F (SZI component index gain/loss), and MCOTc1_L (Indian cotton futures minimum).

F
I G U R E 1 2 Explanation of the 23rd instance by SHAP.T A B L E 7 Descriptive statistical analysis of robustness test datasets.
Overview of price volatility research.
Flowchart of the proposed framework.Descriptive statistical analysis of the datasets.
Data classification and sources.
T A B L E 4

6
Architecture diagram of CP-QR-LSTM model.Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/for.3185by Swansea University, Wiley Online Library on [29/08/2024].See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions)on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License T A B L E 5 Evaluation results of point forecasting indicators.
shows the interaction dependence plots of EUR/CNY_O (opening value of EUR/CNY), VIX_L (lowest value of VIX panic index), and CCYc1_L.Unlike the above two features, EUR/CNY_O and VIX_L affect the CCYc1_C by influencing CCYc1_U, indirectly Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/for.3185by Swansea University, Wiley Online Library on [29/08/2024].See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions)on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License