Abstract
Considering data insufficiency in metal additive manufacturing (AM), transfer learning (TL) has been adopted to extract knowledge from source domains (e.g., completed printings) to improve the modeling performance in target domains (e.g., new printings). Current applications use all accessible source data directly in TL with no regard to the similarity between source and target data. This paper proposes a systematic method to find appropriate subsets of source data based on similarities between the source and limited target datasets. Such similarity is characterized by the spatial and model distance metrics. A Pareto frontier-based source data selection method is developed, where the source data located on the Pareto frontier defined by two similarity distance metrics are selected iteratively. This method is integrated into an instance-based TL method (decision tree regression model) and a model-based TL method (fine-tuned artificial neural network). Both models are then tested on several regression tasks in metal AM. Comparison results demonstrate that (1) the source data selection method is general and supports integration with various TL methods and distance metrics, (2) compared with using all source data, the proposed method can find a subset of source data from the same domain with better TL performance in metal AM regression tasks involving different processes and machines, and (3) when multiple source domains exist, the source data selection method could find the subset from one source domain to obtain comparable or better TL performance than the model constructed using data from all source domains.
Similar content being viewed by others
Data availability
The datasets about the melt pool width in DED-LB/p and the relative densities of parts fabricated by different machines are open-accessed. The dataset about the melt pool width in DED-LB/w is available upon reasonable request from the authors.
References
Aboutaleb, A. M., Bian, L., Elwany, A., Shamsaei, N., Thompson, S. M., & Tapia, G. (2017). Accelerated process optimization for laser-based additive manufacturing by leveraging similar prior studies. IISE Transactions, 49(1), 31–44. https://doi.org/10.1080/0740817X.2016.1189629
Aharoni, R., & Goldberg, Y. (2020). Unsupervised domain clusters in pretrained language models. arXiv preprint: arXiv:2004.02105
Akhavan, J., Lyu, J., Mahmoud, Y., Xu, K., Vallabh, C. K. P., & Manoochehri, S. (2023). Dataset of in-situ coaxial monitoring and print’s cross-section images by direct energy deposition fabrication. Scientific Data, 10(1), 776. https://doi.org/10.1038/s41597-023-02672-4
Bischl, B., Binder, M., Lang, M., Pielok, T., Richter, J., Coors, S., et al. (2023). Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 13(2), 1–43. https://doi.org/10.1002/widm.1484
Bisheh, M. N., Wang, X., Chang, S. I., Lei, S., & Ma, J. (2023). Image-based characterization of laser scribing quality using transfer learning. Journal of Intelligent Manufacturing, 34, 2307–2319. https://doi.org/10.1007/s10845-022-01926-z
Brion, D. A. J., Shen, M., & Pattinson, S. W. (2022). Automated recognition and correction of warp deformation in extrusion additive manufacturing. Additive Manufacturing, 56, 102838. https://doi.org/10.1016/j.addma.2022.102838
Cheng, G. H., Wang, G. G., & Hwang, Y. M. (2021). Multi-objective optimization for high-dimensional expensively constrained black-box problems. Journal of Mechanical Design. https://doi.org/10.1115/1.4050749
Dai, X., Karimi, S., Hachey, B., & Paris, C. (2019). Using similarity measures to select pretraining data for NER. arXiv preprint: arXiv:1904.00585
Dai, X., Karimi, S., Hachey, B., & Paris, C. (2020). Cost-effective selection of pretraining data: a case study of pretraining BERT on social media. arXiv preprint: arXiv:2010.01150
Ding, D., He, F., Yuan, L., Pan, Z., Wang, L., & Ros, M. (2021). The first step towards intelligent wire arc additive manufacturing: An automatic bead modelling system using machine learning through industrial information integration. Journal of Industrial Information Integration, 23, 100218. https://doi.org/10.1016/j.jii.2021.100218
Ding, Y., Ding, P., Zhao, X., Cao, Y., & Jia, M. (2022). Transfer learning for remaining useful life prediction across operating conditions based on multisource domain adaptation. IEEE/ASME Transactions on Mechatronics, 27(5), 4143–4152. https://doi.org/10.1109/TMECH.2022.3147534
Dongshang. (2023). Stainless steel density. http://www.dsstainlesssteel.com/stainless-steel-density/. Accessed 13 Oct 2023.
Ferreira, R. D. S. B., Sabbaghi, A., & Huang, Q. (2020). Automated geometric shape deviation modeling for additive manufacturing systems via Bayesian neural networks. IEEE Transactions on Automation Science and Engineering, 17(2), 584–598. https://doi.org/10.1109/TASE.2019.2936821
Gao, J., Zhao, X., Chen, B., Yan, F., Guo, H., & Tang, R. (2023). AutoTransfer: instance transfer for cross-domain recommendations. In SIGIR 2023—Proceedings of the 46th international ACM SIGIR conference on research and development in information retrieval, Taipei, Taiwan, China (pp. 1478–1487). Association for Computing Machinery. https://doi.org/10.1145/3539618.3591701
Guo, H., Pasunuru, R., & Bansal, M. (2019). AutoSEM: automatic task selection and mixing in multi-task learning. arXiv preprint: arXiv:1904.04153
Jamnikar, N., Liu, S., Brice, C., & Zhang, X. (2021). Machine learning based in situ quality estimation by molten pool condition-quality relations modeling using experimental data. arXiv preprint: arXiv:2103.12066
Kang, Z., Yang, B., Yang, S., Fang, X., & Zhao, C. (2020). Online transfer learning with multiple source domains for multi-class classification. Knowledge-Based Systems, 190, 105149. https://doi.org/10.1016/j.knosys.2019.105149
Karimi, S., Dai, X., Hassanzadeh, H., & Nguyen, A. (2017). Automatic diagnosis coding of radiology reports: a comparison of deep learning and conventional classification methods. In BioNLP 2017—SIGBioMed workshop on biomedical natural language processing, proceedings of the 16th BioNLP Workshop, Vancouver, Canada (pp. 328–332). Association for Computational Linguistics. https://doi.org/10.18653/v1/w17-2342
Kim, H., Lee, H., & Ahn, S. H. (2022). Systematic deep transfer learning method based on a small image dataset for spaghetti-shape defect monitoring of fused deposition modeling. Journal of Manufacturing Systems, 65, 439–451. https://doi.org/10.1016/j.jmsy.2022.10.009
Kitahara, A. R., & Holm, E. A. (2018). Microstructure cluster analysis with transfer learning and unsupervised learning. Integrating Materials and Manufacturing Innovation, 7(3), 148–156. https://doi.org/10.1007/s40192-018-0116-9
Knüttel, D., Baraldo, S., Valente, A., Wegener, K., & Carpanzano, E. (2022). Transfer learning of neural network based process models in direct metal deposition. Procedia CIRP, 107, 863–868. https://doi.org/10.1016/j.procir.2022.05.076
Lange, L., Strötgen, J., Adel, H., & Klakow, D. (2021). To share or not to share: predicting sets of sources for model transfer learning. arXiv preprint: arXiv:2104.08078
Leung, H. C., Leung, C. S., & Wong, E. W. M. (2019). Fault and noise tolerance in the incremental extreme learning machine. IEEE Access, 7, 155171–155183. https://doi.org/10.1109/ACCESS.2019.2948059
Li, Z., Liu, F., Yang, W., Peng, S., & Zhou, J. (2022). A survey of convolutional neural networks: Analysis, applications, and prospects. IEEE Transactions on Neural Networks and Learning Systems, 33(12), 6999–7019. https://doi.org/10.1109/TNNLS.2021.3084827
Lin, Y. H., Chen, C. Y., Lee, J., Li, Z., Zhang, Y., Xia, M., et al. (2019). Choosing transfer languages for cross-lingual learning. arXiv preprint: arXiv:1905.12688
Liu, M., Song, Y., Zou, H., & Zhang, T. (2020). Reinforced training data selection for domain adaptation. In ACL 2019—57th annual meeting of the association for computational linguistics, Florence, Italy (pp. 1957–1968). Association for Computational Linguistics. https://doi.org/10.18653/v1/p19-1189
Liu, R., Liu, S., & Zhang, X. (2021a). A physics-informed machine learning model for porosity analysis in laser powder bed fusion additive manufacturing. International Journal of Advanced Manufacturing Technology, 113(7–8), 1943–1958. https://doi.org/10.1007/s00170-021-06640-3
Liu, S., Stebner, A. P., Kappes, B. B., & Zhang, X. (2021b). Machine learning for knowledge transfer across multiple metals additive manufacturing printers. Additive Manufacturing, 39, 101877. https://doi.org/10.1016/j.addma.2021.101877
Lu, H., Wu, J., Ruan, Y., Qian, F., Meng, H., Gao, Y., & Xu, T. (2023). A multi-source transfer learning model based on LSTM and domain adaptation for building energy prediction. International Journal of Electrical Power and Energy Systems, 149, 109024. https://doi.org/10.1016/j.ijepes.2023.109024
MatWeb. (2023). Duplex stainless steel 2209. https://www.matweb.com/search/datasheet.aspx?matguid=e4df7ef1593f4f518bd3b26667a0aa56. Accessed 13 Oct 2023.
Mehta, M., & Shao, C. (2022). Federated learning-based semantic segmentation for pixel-wise defect detection in additive manufacturing. Journal of Manufacturing Systems, 64, 197–210. https://doi.org/10.1016/j.jmsy.2022.06.010
Mikolov, T., Grave, E., Bojanowski, P., Puhrsch, C., & Joulin, A. (2017). Advances in pre-training distributed word representations. arXiv preprint: arXiv:1712.09405
Milner, B. B., Gradl, P., Snedden, G., Brooks, M., Pitot, J., Lopez, E., et al. (2021). Metal additive manufacturing in aerospace: A review. Materials and Design, 209, 110008. https://doi.org/10.1016/j.matdes.2021.110008
Ngatchou, P., Zarei, A., & El-Sharkawi, M. A. (2005). Pareto multi objective optimization. In Proceedings of the 13th international conference on intelligent systems application to power systems, Arlington, VA, USA (pp. 84–91). IEEE. https://doi.org/10.1109/ISAP.2005.1599245
Olleak, A., & Xi, Z. (2020). Calibration and validation framework for selective laser melting process based on multi-fidelity models and limited experiment data. Journal of Mechanical Design, Transactions of the ASME, 142(8), 1–13. https://doi.org/10.1115/1.4045744
Ontañón, S. (2020). An overview of distance and similarity functions for structured data. Artificial Intelligence Review, 53(7), 5309–5351. https://doi.org/10.1007/s10462-020-09821-w
Pandita, P., Ghosh, S., Gupta, V. K., Meshkov, A., & Wang, L. (2022). Application of deep transfer learning and uncertainty quantification for process identification in powder bed fusion. ASCE-ASME Journal of Risk and Uncertainty in Engineering Systems, Part B: Mechanical Engineering, 8(1), 1–12. https://doi.org/10.1115/1.4051748
Pandiyan, V., Drissi-Daoudi, R., Shevchik, S., Masinelli, G., Le-Quang, T., Logé, R., & Wasmer, K. (2022). Deep transfer learning of additive manufacturing mechanisms across materials in metal-based laser powder bed fusion process. Journal of Materials Processing Technology. https://doi.org/10.1016/j.jmatprotec.2022.117531
Pang, Y., Cao, Y., Chu, Y., Liu, M., Snyder, K., MacKenzie, D., & Cao, C. (2020). Additive manufacturing of batteries. Advanced Functional Materials, 30(1), 1–22. https://doi.org/10.1002/adfm.201906244
Pardoe, D., & Stone, P. (2010). Boosting for regression transfer. In Proceedings of the 27th international conference on machine learning, Haifa, Israel (pp. 863–870). Association for Computing Machinery. https://doi.org/10.5555/3104322.3104432
Qin, J., Hu, F., Liu, Y., Witherell, P., Wang, C. C. L., Rosen, D. W., et al. (2022). Research and application of machine learning for additive manufacturing. Additive Manufacturing. https://doi.org/10.1016/j.addma.2022.102691
Rahmani Dehaghani, M., Sahraeidolatkhaneh, A., Nilsen, M., Sikström, F., Sajadi, P., Tang, Y., & Wang, G. G. (2024). System identification and closed-loop control of laser hot-wire directed energy deposition using the parameter-signature-property modeling scheme. Journal of Manufacturing Processes, 112, 1–13. https://doi.org/10.1016/j.jmapro.2024.01.029
Ren, J. (2018). Two-stage-TrAdaboost.R2. https://github.com/jay15summer/Two-stage-TrAdaboost.R2. Accessed 20 Jan 2020.
Ren, J., & Wang, H. (2019). Surface variation modeling by fusing multiresolution spatially nonstationary data under a transfer learning framework. Journal of Manufacturing Science and Engineering, Transactions of the ASME, 141(1), 011002. https://doi.org/10.1115/1.4041425
Ren, J., Wei, A. T., Jiang, Z., Wang, H., & Wang, X. (2021). Improved modeling of kinematics-induced geometric variations in extrusion-based additive manufacturing through between-printer transfer learning. IEEE Transactions on Automation Science and Engineering, 19(3), 2310–2321. https://doi.org/10.1109/TASE.2021.3063389
Ren, K., Chew, Y., Zhang, Y. F., Fuh, J. Y. H., & Bi, G. J. (2020). Thermal field prediction for laser scanning paths in laser aided additive manufacturing by physics-based machine learning. Computer Methods in Applied Mechanics and Engineering, 362, 112734. https://doi.org/10.1016/j.cma.2019.112734
Ruder, S., & Plank, B. (2017). Learning to select data for transfer learning with Bayesian optimization. arXiv preprint: arXiv:1707.05246
Sabbaghi, A., & Huang, Q. (2018). Model transfer across additive manufacturing processes via mean effect equivalence of lurking variables. The Annals of Applied Statistics, 12(4), 2409–2429. https://doi.org/10.1214/18-AOAS1158
Sabbaghi, A., Huang, Q., & Dasgupta, T. (2018). Bayesian model building from small samples of disparate data for capturing in-plane deviation in additive manufacturing. Technometrics, 60(4), 532–544. https://doi.org/10.1080/00401706.2017.1391715
Schönemann, P. H. (1966). A generalized solution of the orthogonal procrustes problem. Psychometrika, 31(1), 1–10. https://doi.org/10.1007/BF02289451
Senanayaka, A., Tian, W., Falls, T. C., & Bian, L. (2023). Understanding the effects of process conditions on thermal–defect relationship: a transfer machine learning approach. Journal of Manufacturing Science and Engineering. https://doi.org/10.1115/1.4057052
Shin, S. J., Lee, J. H., Sainand, J., & Kim, D. B. (2024). Material-adaptive anomaly detection using property-concatenated transfer learning in wire arc additive manufacturing. International Journal of Precision Engineering and Manufacturing, 25, 383–408. https://doi.org/10.1007/s12541-023-00924-2
Tang, Y., Rahmani Dehaghani, M., & Wang, G. G. (2023a). Comparison of transfer learning based additive manufacturing models via a case study. arXiv preprint: arXiv:2305.11181
Tang, Y., Rahmani Dehaghani, M., & Wang, G. G. (2023b). Review of transfer learning in modeling additive manufacturing processes. Additive Manufacturing, 61, 103357. https://doi.org/10.1016/j.addma.2022.103357
Tian, J., Han, D., Li, M., & Shi, P. (2022). A multi-source information transfer learning method with subdomain adaptation for cross-domain fault diagnosis. Knowledge-Based Systems, 243, 108466. https://doi.org/10.1016/j.knosys.2022.108466
Tian, Y., Sehovac, L., & Grolinger, K. (2019). Similarity-based chained transfer learning for energy forecasting with big data. IEEE Access, 7, 139895–139908. https://doi.org/10.1109/ACCESS.2019.2943752
Van Houdt, G., Mosquera, C., & Nápoles, G. (2020). A review on the long short-term memory model. Artificial Intelligence Review, 53(8), 5929–5955. https://doi.org/10.1007/s10462-020-09838-1
Vasco, J. C. (2021). Additive manufacturing for the automotive industry. In J. P. Davim & K. Gupta (Eds.), Additive manufacturing (pp. 505–530). Elsevier Inc. https://doi.org/10.1016/B978-0-12-818411-0.00010-0
Wang, B., Qiu, M., Wang, X., Li, Y., Gong, Y., Zeng, X., et al. (2019). A minimax game for instance based selective transfer learning. In Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining, Anchorage, AK, USA (pp. 34–43). Association for Computing Machinery. https://doi.org/10.1145/3292500.3330841
Wang, J., Li, Y., Gao, R. X., & Zhang, F. (2022a). Hybrid physics-based and data-driven models for smart manufacturing: Modelling, simulation, and explainability. Journal of Manufacturing Systems, 63, 381–391. https://doi.org/10.1016/j.jmsy.2022.04.004
Wang, Z., Yang, W., Liu, Q., Zhao, Y., Liu, P., Wu, D., et al. (2022b). Data-driven modeling of process, structure and property in additive manufacturing: A review and future directions. Journal of Manufacturing Processes, 77, 13–31. https://doi.org/10.1016/j.jmapro.2022.02.053
Willia, R. J. (1992). Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8(3), 229–256. https://doi.org/10.1023/A:1022672621406
Xie, Y., Li, B., Wang, C., Zhou, K., Wu, C. T., & Li, S. (2023). A Bayesian regularization network approach to thermal distortion control in 3D printing. Computational Mechanics, 72(1), 137–154. https://doi.org/10.1007/s00466-023-02270-6
Xie, Y., Li, S., Wu, C. T., Lyu, D., Wang, C., & Zeng, D. (2022). A generalized Bayesian regularization network approach on characterization of geometric defects in lattice structures for topology optimization in preliminary design of 3D printing. Computational Mechanics, 69(5), 1191–1212. https://doi.org/10.1007/s00466-021-02137-8
Yao, Y., & Doretto, G. (2010). Boosting for transfer learning with multiple sources. In Proceedings of the IEEE computer society conference on computer vision and pattern recognition, San Francisco, CA, USA (pp. 1855–1862). IEEE. https://doi.org/10.1109/CVPR.2010.5539857
Yuan, Y., Chen, Z., Wang, Z., Sun, Y., & Chen, Y. (2023). Attention mechanism-based transfer learning model for day-ahead energy demand forecasting of shopping mall buildings. Energy, 270, 126878. https://doi.org/10.1016/j.energy.2023.126878
Zhang, K., Xiong, C., Liu, Z., & Liu, Z. (2020). Selective weak supervision for neural information retrieval. In The web conference 2020—proceedings of the world wide web conference, WWW 2020, Taipei, Taiwan, China (pp. 474–485). Association for Computing Machinery. https://doi.org/10.1145/3366423.3380131
Zhang, H., Choi, J. P., Moon, S. K., & Ngo, T. H. (2021). A knowledge transfer framework to support rapid process modeling in aerosol jet printing. Advanced Engineering Informatics, 48, 101264. https://doi.org/10.1016/j.aei.2021.101264
Zhou, Z., Shen, H., Liu, B., Du, W., & Jin, J. (2021). Thermal field prediction for welding paths in multi-layer gas metal arc welding-based additive manufacturing: A machine learning approach. Journal of Manufacturing Processes, 64, 960–971. https://doi.org/10.1016/j.jmapro.2021.02.033
Zhu, J., Jiang, Q., Shen, Y., Qian, C., Xu, F., & Zhu, Q. (2022). Application of recurrent neural network to mechanical fault diagnosis: A review. Journal of Mechanical Science and Technology, 36(2), 527–542. https://doi.org/10.1007/s12206-022-0102-1
Zhu, X., Jiang, F., Guo, C., Xu, D., Wang, Z., & Jiang, G. (2023). Surface morphology inspection for directed energy deposition using small dataset with transfer learning. Journal of Manufacturing Processes, 93, 101–115. https://doi.org/10.1016/j.jmapro.2023.03.016
Acknowledgements
The authors gratefully acknowledge funding from the Natural Sciences and Engineering Research Council (NSERC) of Canada [Grant numbers: RGPIN-2019-06601] and the in-kind support of University West (Dr. Morgan Nilsen and Dr. Fredrik Sikström) under the Eureka! SMART project (S0410) titled “TANDEM: Tools for Adaptive and Intelligent Control of Discrete Manufacturing Processes.” Meanwhile, the authors acknowledge the Ph.D. candidate Javid Akhavan at Stevens Institute of Technology, United States, for his help in processing their image datasets.
Author information
Authors and Affiliations
Contributions
Yifan Tang: writing—original draft preparation, validation, methodology, investigation, formal analysis, conceptualization. Mostafa Rahmani Dehaghani: resources, methodology, formal analysis, conceptualization. Pouyan Sajadi: resources, methodology, formal analysis, conceptualization. G. Gary Wang: writing—review & editing, supervision, resources, project administration, methodology, funding acquisition, conceptualization.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflicts of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Tang, Y., Rahmani Dehaghani, M., Sajadi, P. et al. Selecting subsets of source data for transfer learning with applications in metal additive manufacturing. J Intell Manuf (2024). https://doi.org/10.1007/s10845-024-02402-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10845-024-02402-6