Skip to main content

Advertisement

SpringerLink
Log in

Decoding human taste perception by reconstructing and mining temporal-spatial features of taste-related EEGs

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

For humans, taste is essential for perceiving the nutrient content or harmful components of food. The current method of taste sensory evaluation relies on artificial sensory evaluation and an electronic tongue. The former has strong subjectivity and poor repeatability, and the latter is not sufficiently flexible. To decode people's objective taste perception, a strategy for acquiring and recognizing four classes (sour, sweet, bitter, and salty) in taste-related electroencephalograms (EEGs) was proposed. First, according to the proposed experimental paradigm, the taste-related EEGs of subjects under different taste stimulations were collected. Second, to avoid insufficient training of the model due to the small number of EEG samples, a temporal and spatial reconstruction data augmentation (TSRDA) method was proposed, effectively augmenting taste-related EEGs by reconstructing the important features in temporal and spatial dimensions. Third, a multiview channel attention (MVCA) module was introduced into a designed convolutional neural network to extract the important features of the augmented EEG. The proposed method had an accuracy of 99.56%, F1 score of 99.48%, and kappa value of 99.38%, showing the method's ability to successfully decoded sour, sweet, bitter, and salty EEG signals. In conclusion, combining TSRDA with EEG technology provides an objective and effective method for the sensory evaluation of food taste.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Data availability

The datasets generated during and/or analysed during the current study are not publicly available due to privacy but are available from the corresponding author on reasonable request.

References

  1. Vivek K, Subbarao KV, Routray W, Kamini NR, Dash KK (2019) Application of fuzzy logic in sensory evaluation of food products: a comprehensive study. Food Bioproc Technol 13:1–29. https://doi.org/10.1007/s11947-019-02337-4

    Article  Google Scholar 

  2. Stasi A, Songa G, Mauri M, Ciceri A, Diotallevi F, Nardone G, Russo V (2018) Neuromarketing empirical approaches and food choice: A systematic review. Food Res Int 108:650–664. https://doi.org/10.1016/j.foodres.2017.11.049

    Article  Google Scholar 

  3. Zheng W, Men H, Shi Y, Ying Y, Liu J, Liu Q (2022) Computational model of taste pathways: A biomimetic algorithm for electronic tongue based on nerve conduction mechanism. IEEE Sens J 22:6859–6870. https://doi.org/10.1109/JSEN.2022.3152057

    Article  Google Scholar 

  4. Dogan S, Tuncer I, Baygin M, Tuncer T (2023) A new hand-modeled learning framework for driving fatigue detection using EEG signals. Neural Comput & Applic 35:14837–14854. https://doi.org/10.1007/s00521-023-08491-3

    Article  Google Scholar 

  5. Wang H, Zhan X, Liu L, Ullah A, Li H, Gao H, Wang Y, Hu R, Li G (2022) Unsupervised cross-user adaptation in taste sensation recognition based on surface electromyography. IEEE Trans Instrum Meas 71:2509611. https://doi.org/10.1109/TIM.2022.3160834

    Article  Google Scholar 

  6. Linforth RS (2000) Developments in instrumental techniques for food flavour evaluation: future prospects. J Sci Food Agric 80:2044–2048. https://doi.org/10.1002/1097-0010(200011)80:14%3c2044::AID-JSFA753%3e3.0.CO;2-Z

    Article  Google Scholar 

  7. Wang ZM, Zhang JW, He Y, Zhang J (2022) EEG emotion recognition using multichannel weighted multiscale permutation entropy. Appl Intell 52:12064–12076. https://doi.org/10.1007/s10489-021-03070-2

    Article  Google Scholar 

  8. Zhong X, Gu Y, Luo Y, Zeng X, Liu G (2023) Bi-hemisphere asymmetric attention network: recognizing emotion from EEG signals based on the transformer. Appl Intell 53:15278–15294. https://doi.org/10.1007/s10489-022-04228-2

    Article  Google Scholar 

  9. Xu DQ, Li MA (2023) A dual alignment-based multi-source domain adaptation framework for motor imagery EEG classification. Appl Intell 53(9):10766–10788. https://doi.org/10.1007/s10489-022-04077-z

    Article  Google Scholar 

  10. Tasci I, Tasci B, Barua PD, Dogan S, Tuncer T, Palmer EE, Fujita H, Acharya UR (2023) Epilepsy detection in 121 patient populations using hypercube pattern from EEG signals. Information Fusion 96(2023):252–268. https://doi.org/10.1016/j.inffus.2023.03.022

    Article  Google Scholar 

  11. Tasci G, Gun MV, Keles T, Tasci B, Barua PD, Tasci I, Dogan S, Baygin M, Palmer EE, Tuncer T, Ooi CP, Acharya UR (2023) QLBP: Dynamic patterns-based feature extraction functions for automatic detection of mental health and cognitive conditions using EEG signals. Chaos, Solitons Fractals 172:113472. https://doi.org/10.1016/j.chaos.2023.113472

    Article  Google Scholar 

  12. Kroupi E, Vesin J-M, Ebrahimi T (2016) Subject-independent odor pleasantness classification using brain and peripheral signals. IEEE Trans Affect Comput 7(4):422–434. https://doi.org/10.1109/TAFFC.2015.2496310

    Article  Google Scholar 

  13. Andersen CA, Kring ML, Andersen RH, Larsen ON, Kjær TW, Kidmose U, Moller S, Kidmose P (2019) EEG discrimination of perceptually similar tastes. J Neurosci Res 97:241–252. https://doi.org/10.1002/jnr.24281

    Article  Google Scholar 

  14. Anbarasan R, Carmona DG, Mahendran R (2022) Human taste-perception: Brain Computer Interface (BCI) and its application as an engineering tool for taste-driven sensory studies. Food Eng Rev 14:408–434. https://doi.org/10.1007/s12393-022-09308-0

    Article  Google Scholar 

  15. Crouzet SM, Busch NA, Ohla K (2015) Taste quality decoding parallels taste sensations. Curr Biol 25(7):890–896. https://doi.org/10.1016/j.cub.2015.01.057

    Article  Google Scholar 

  16. Hashida JC, Silva ACD, Souto S, Costa EJX (2005) EEG pattern discrimination between salty and sweet taste using adaptive Gabor transform. Neurocomputing 68:251–257. https://doi.org/10.1016/j.neucom.2005.04.004

    Article  Google Scholar 

  17. Chandran KS, Perumalsamy M (2019) EEG – Taste classification through sensitivity analysis. Int J Electr Eng Educ. https://doi.org/10.1177/0020720919833036

    Article  Google Scholar 

  18. Men H, Liu M, Shi Y, Xia XX, Wang TZ, Liu JJ, Liu QJ (2022) Interleaved attention convolutional compression network: An effective data mining method for the fusion system of gas sensor and hyperspectral. Sens Actuators, B Chem 355:131113. https://doi.org/10.1016/j.snb.2021.131113

    Article  Google Scholar 

  19. Kang SY, Zhang QL, Li ZY, Yin CB, Feng NH, Shi Y (2023) Determination of the quality of tea from different picking periods: an adaptive pooling attention mechanism coupled with an electronic nose. Postharvest Biol Technol 197:112214. https://doi.org/10.1016/j.postharvbio.2022.112214

    Article  Google Scholar 

  20. Xia X, Liu X, Zheng W, Jia X, Wang B, Shi Y, Men H (2023) Recognition of odor and pleasantness based on olfactory EEG combined with functional brain network model. Int J Mach Learn Cybern 14:2761–2776. https://doi.org/10.1007/s13042-023-01797-7

    Article  Google Scholar 

  21. Xia X, Shi Y, Li P, Liu X, Liu J, Men H (2023) FBANet: An Effective Data Mining Method for Food Olfactory EEG Recognition. IEEE Transactions on Neural Networks and Learning Systems. https://doi.org/10.1109/TNNLS.2023.3269949

    Article  Google Scholar 

  22. Lawhern VJ, Solon AJ, Waytowich NR, Gordon SM, Hung CP, Lance BJ (2018) EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces. J Neural Eng 15:056013. https://doi.org/10.1088/1741-2552/aace8c

    Article  Google Scholar 

  23. Seal A, Bajpai R, Agnihotri J, Yazidi A, Herrera-Viedma E, Krejcar O (2021) DeprNet: A Deep Convolution Neural Network Framework for Detecting Depression Using EEG. IEEE Trans Instrum Meas 70:2505413. https://doi.org/10.1109/TIM.2021.3053999

    Article  Google Scholar 

  24. Zhao XQ, Zhang HM, Zhu GL, You FX, Kuang SL, Sun LN (2019) A multi-branch 3D convolutional neural network for EEG-based motor imagery classification. IEEE Trans Neural Syst Rehabil Eng 27(10):2164–2177. https://doi.org/10.1109/TNSRE.2019.2938295

    Article  Google Scholar 

  25. Wang F, Zhong SH, Peng JF, Jiang JM, Liu Y (2019) Data augmentation for EEG-based emotion recognition with deep convolutional neural networks. Lecture Notes Artificial Intelligence 10705:82–93. https://doi.org/10.1007/978-3-319-73600-6_8

    Article  Google Scholar 

  26. Zhang Y, Guo Y, Yang P, Chen W, Lo B (2020) Epilepsy seizure prediction on EEG using common spatial pattern and convolutional neural network. IEEE J Biomed Health Inform 24(2):465–474. https://doi.org/10.1109/JBHI.2019.2933046

    Article  Google Scholar 

  27. Pei Y, Luo ZG, Yan Y, Yan HJ, Jiang J, Li WG, Xie L, Yin ER (2021) Data augmentation: Using channel-level recombination to improve classification performance for motor imagery EEG. Front Hum Neurosci 20:645952. https://doi.org/10.3389/fnhum.2021.645952

    Article  Google Scholar 

  28. Schirrmeister RT, Springenberg JT, Fiederer LDJ, Glasstetter M, Eggensperger K, Tangermann M, Hutter F, Burgard W, Ball T (2017) Deep learning with convolutional neural networks for EEG decoding and visualization. Hum Brain Mapp 38(11):5391–5420. https://doi.org/10.1002/hbm.23730

    Article  Google Scholar 

  29. Al-Saegh A, Dawwd SA, Abdul-Jabbar JM (2021) CutCat: An augmentation method for EEG classification. Neural Netw 141:433–443. https://doi.org/10.1016/j.neunet.2021.05.032

    Article  Google Scholar 

  30. Yun S, Han D, Oh SJ, Chun S, Choe J, Yoo Y (2019) CutMix: Regularization strategy to train strong classifiers with localizable features. In: IEEE/CVF International Conference on Computer Vision (ICCV). 6022–6031. https://doi.org/10.1109/ICCV.2019.00612.

  31. Woo S, Park J, Lee J-Y, Kweon IS (2018) CBAM: Convolutional block attention module. In: 15th European Conference on Computer Vision (ECCV). https://doi.org/10.1007/978-3-030-01234-2_1.

  32. Hu J, Shen L, Albanie S, Sun G, Wu E (2020) Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell 42(8):2011–2023. https://doi.org/10.1109/TPAMI.2019.2913372

    Article  Google Scholar 

  33. Li X, Hu X, Yang J (2019) Spatial group-wise enhance: improving semantic feature learning in convolutional networks. arXiv: Computer Vision and Pattern Recognition, arXiv e-prints. https://doi.org/10.48550/arXiv.1905.09646.

  34. Wang Q, Wu B, Zhu P, Li P, Zuo W, Hu Q (2020) Eca-net: Efficient channel attention for deep convolutional neural networks. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 11531–11539. https://doi.org/10.1109/CVPR42600.2020.01155.

  35. Wallroth R, Hochenberger R, Ohla K (2018) Delta activity encodes taste information in the human brain. Neuroimage 181:471–479. https://doi.org/10.1016/j.neuroimage.2018.07.034

    Article  Google Scholar 

  36. Delorme A, Makeig S (2004) EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods 134:9–21. https://doi.org/10.1016/j.jneumeth.2003.10.009

    Article  Google Scholar 

  37. Xu GW, Shen XA, Chen SR, Zong YS, Zhang CY, Yue HY, Liu M, Chen F, Che WL (2019) A deep transfer convolutional neural network framework for EEG signal classification. IEEE Access 7:112767–112776. https://doi.org/10.1109/ACCESS.2019.2930958

    Article  Google Scholar 

  38. Laurens VDM, Hinton G (2008) Visualizing Data using t-SNE. J Mach Learn Res 9:2579–2605

    Google Scholar 

  39. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2020) Grad-CAM: Visual explanations from deep networks via gradient-based localization. Int J Comput Vision 128(2):336–359. https://doi.org/10.1007/s11263-019-01228-7

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the Science and Technology Development Plan of Jilin Province under Grant YDZJ202101ZYTS135.

Author information

Authors and Affiliations

  1. School of Automation Engineering, Northeast Electric Power University, Jilin, 132012, China

    Xiuxin Xia, Yuchao Yang, Yan Shi, Wenbo Zheng & Hong Men

  2. Institute of Advanced Sensor Technology, Northeast Electric Power University, Jilin, 132012, China

    Xiuxin Xia, Yuchao Yang, Yan Shi, Wenbo Zheng & Hong Men

Authors
  1. Xiuxin Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuchao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wenbo Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hong Men
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: Xiuxin Xia; Methodology: Xiuxin Xia; Formal analysis and investigation: Xiuxin Xia, Yuchao Yang, Yan Shi; Writing—original draft preparation: Xiuxin Xia; Writing—review and editing: Xiuxin Xia; Funding acquisition: Hong Men; Resources: Yuchao Yang, Wenbo Zheng; Supervision: Hong Men.

Corresponding author

Correspondence to Hong Men.

Ethics declarations

Ethical standards

There was no potential conflicts of interest in the research. The research was in line with the revised Declaration of Helsinki, and the protocol had been approved by the Northeast Electric Power University Scientific Research Ethics and Science and Technology Safety and Committee. The subjects all signed the informed consent form.

Competing interests

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xia, X., Yang, Y., Shi, Y. et al. Decoding human taste perception by reconstructing and mining temporal-spatial features of taste-related EEGs. Appl Intell 54, 3902–3917 (2024). https://doi.org/10.1007/s10489-024-05374-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05374-5

Keywords

Search

Navigation