Skip to main content
SpringerLink
Log in

The need for external validation in machine olfaction: emphasis on health-related applications

  • Review
  • Published:
Analytical and Bioanalytical Chemistry Aims and scope Submit manuscript

Abstract

Over the last two decades, electronic nose research has produced thousands of research works. Many of them were describing the ability of the e-nose technology to solve diverse applications in domains ranging from food technology to safety, security, or health. It is, in fact, in the biomedical field where e-nose technology is finding a research niche in the last years. Although few success stories exist, most described applications never found the road to industrial or clinical exploitation. Most described methodologies were not reliable and were plagued by numerous problems that prevented practical application beyond the lab. This work emphasizes the need of external validation in machine olfaction. I describe some statistical and methodological pitfalls of the e-nose practice and I give some best practice recommendations for researchers in the field.

State-of-the-art electronic noses feature digitally embedded multivariate predictive system: either pattern recognition systems or quantitative predictors

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Dragonieri S, Schot R, Mertens BJ, Le Cessie S, Gauw SA, Spanevello A, Resta O, Willard NO, Vink TJ, Rabe KF, Bel EH, Sterk PJ (2007) An electronic nose in the discrimination of patients with asthma and controls. J Allergy Clin Immunol 120:856–862

    Article  Google Scholar 

  2. Montuschi P, Mores N, Trové A, Mondino C, Barnes PJ (2013) The electronic nose in respiratory medicine. Respi Int Rev Thorac Dis 85:72–84

    Google Scholar 

  3. Greulich T, Hattesohl A, Grabisch A, Koepke J, Schmid S, Noeske S, Nell C, Wencker M, Jörres RA, Vogelmeier CF, Köhler U, Koczulla AR (2013) Detection of obstructive sleep apnea by an electronic nose. Eur Respir J 42:145–155

    Article  Google Scholar 

  4. Lazar Z, Fens N, van der Maten J, van der Schee MP, Wagener AH, de Nijs SB, Dijkers E, Sterk PJ (2010) Electronic nose breathprints are independent of acute changes in airway caliber in asthma. Sensors 10:9127–9138

    Article  CAS  Google Scholar 

  5. Biller H, Holz O, Windt H, Koch W, Müller M, Jörres RA, Krug N, Hohlfeld JM (2011) Breath profiles by electronic nose correlate with systemic markers but not ozone response. Respir Med 105:1352–1363

    Article  Google Scholar 

  6. Chapman E, Thomas PS, Stone E, Lewis C, Yates DH (2012) A breath test for malignant mesothelioma using an electronic nose. Eur Respir J 40:448–454

    Article  Google Scholar 

  7. Hattesohl A, Jörres R, Dressel H, Schmid S, Vogelmeier C, Greulich T, Noeske S, Bals R, Koczulla AR (2011) Discrimination between COPD patients with and without alpha 1-antitrypsin deficiency using an electronic nose. Respirology 16:1258–1264

    Article  Google Scholar 

  8. Valera J, Togores B, Cosio B (2012) Use of the electronic nose for diagnosing respiratory diseases. Arch Bronconeumología 48:187–188

    Google Scholar 

  9. Fens N, Zwinderman A, van der Schee M, de Nijs SB, Dijkers E, Roldaan AC, Cheung D, Bel EH, Sterk PJ (2009) Exhaled breath profiling enables discrimination of chronic obstructive pulmonary disease and asthma. Am J Respir Crit Care Med 180:1076–1082

    Article  CAS  Google Scholar 

  10. Di Natale C, Macagnano A, Martinelli E, Paolesse R, D'Arcangelo G, Roscioni C, Finazzi-Agrò A, D'Amico A (2003) Lung cancer identification by the analysis of breath by means of an array of non-selective gas sensors. Biosens Bioelectron 18:1209–1218

    Article  Google Scholar 

  11. Machado R, Laskowski D, Deffenderfer O, Burch T, Zheng S, Mazzone PJ, Mekhail T, Jennings C, Stoller JK, Pyle J, Duncan J, Dweik RA, Erzurum SC (2005) Detection of lung cancer by sensor array analyses of exhaled breath. Am J Respir Crit Care Med 171:1286–1291

    Article  Google Scholar 

  12. Fens N, de Nijs S, Peters S, Dekker T, Knobel HH, Vink TJ, Willard NP, Zwinderman AH, Krouwels FH, Janssen HG, Lutter R, Sterk PJ (2011) Exhaled air molecular profiling in relation to inflammatory subtype and activity in COPD. Eur Respir J 38:1301–1309

    Article  CAS  Google Scholar 

  13. Pennazza G, Marchetti E, Santonico M, Mantini G, Mummolo S, Marzo G, Paolesse R, D'Amico A, Di Natale C (2008) Application of a quartz microbalance based gas sensor array for the study of halitosis. J Breath Res 2:017009

    Article  Google Scholar 

  14. Persaud K (2005) Medical applications of odor-sensing devices. The Int J Lower Extrem Wounds 4:50–6

    Article  Google Scholar 

  15. Kodogiannis V, Lygouras J, Tarzynski A, Chowdrey H (2008) Artificial odor discrimination system using electronic nose and neural networks for the identification of urinary tract infection. IEEE Trans Inf Technol Biomed 12:707–703

    Article  Google Scholar 

  16. Covington J, Wedlake L, Andreyev J, Ouaret N, Thomas MG, Nwokolo CU, Bardhan KD, Arasaradnam RP (2012) The detection of patients at risk of gastrointestinal toxicity during pelvic radiotherapy by electronic nose and FAIMS: a pilot study. Sensors 12:13002–13018

    Article  CAS  Google Scholar 

  17. Marco S, Gutierrez-Galvez A (2012) Signal and Data Processing for Machine Olfaction and Chemical Sensing: A Review. IEEE Sensors J 12:3174–3184

    Article  Google Scholar 

  18. Gutierrez-Osuna R (2002) Pattern analysis for machine olfaction: a review. IEEE Sensors J 162(1-2):46–55

    Google Scholar 

  19. Gutiérrez, A, Marco, S, Eds. (2009). Biologically Inspired Signal Processing for Chemical Sensing (Studies in Computational Intelligence. Vol. 188. Springer

  20. Persaud KC, Marco S, Gutierrez-Galvez A, Eds (20139, Neuromorphic Olfaction, in Frontiers in Neuroengineering, CRC Press

  21. Ioannidis J (2005) Why most published research findings are false. PLoS Med 2:e124

    Article  Google Scholar 

  22. Cornfield J (1966) Sequential trials, sequential analysis and the likelihood principle. Am Statistian 20:18–23

    Google Scholar 

  23. Defernez M, Kemsley EK (1997) The use and misuse of chemometrics for treating classification problems. TrAC Trends Anal Chem 16:216–221

    Article  Google Scholar 

  24. Marco S, Ortega A, Pardo A, Samitier J (1998) Gas identification with tin oxide sensor array and self-organizing maps: adaptive correction of sensor drifts. Instrum Meas Trans IEEE 47:316–321

    Article  CAS  Google Scholar 

  25. Padilla M, Perera A, Montoliu I, Chaudry A, Persaud K, Marco S, (2010) Drift compensation of gas sensor array data by orthogonal signal correction. Vol. 100, pp. 28–35

  26. Ziyatdinov A, Marco S, Chaudry A, Persaud K, Caminal P, Perera A (2010) Drift compensation of gas sensor array data by common principal component analysis. Sensors Actuators B: Chem 146:460–465

    Article  CAS  Google Scholar 

  27. Martinelli E, Magna G, Vito SD, Di Fuccio R, Di Francia F, Vergara A, Di Natale C (2013) An adaptive classification model based on the Artificial Inmune System for chemical sensor drift mitigation. Sensors Actuators B: Chem 177:1017–1026

    Article  CAS  Google Scholar 

  28. Di Carlo S, Falasconi M, Sanchez E, Scionti A, Squillero G, Tonda A (2011) Increasing pattern recognition accuracy for chemical sensing by evolutionary based drift compensation. Pattern Recog Lett 32:1594–1603

    Article  Google Scholar 

  29. Gutierrez-Osuna R (2000) Drift reduction for metal-oxide sensor arrays using canonical correlation regression and partial least squares. Proceedings of the 7th International Symposium on Olfaction and Electronic Nose. IOP Press, London, pp 147–152

    Google Scholar 

  30. Artursson T, Eklov T, Lundstrom I, Martensson P, Sjostrom M, Holmberg M (2000) Drift correction for gas sensors using multivariate methods. J Chemom 14:711–723

    Article  CAS  Google Scholar 

  31. Knobloch H, Turner C, Spooner A, Chambers M (2009) Methodological variation in headspace analysis of liquid samples using electronic nose. Sensors Actuators B: Chem 139:353–360

    Article  CAS  Google Scholar 

  32. Kuske M, Rubio R, Romain AC, Nicolas J, Marco S (2005) Fuzzy k-NN applied to molds detection. Sensors Actuators B: Chem 106(1):52–60

    Article  CAS  Google Scholar 

  33. Kuske M, Padilla M, Romain AC, Nicolas J, Rubio R, Marco S (2006) Detection of diverse mold species growing on building materials by gas sensor arrays and pattern recognition. Sensors Actuators B: Chem 119(1):33–40

    Article  CAS  Google Scholar 

  34. Adam G, Lemaigre S, Romain A-C, Nicolas J, Delfosse P (2013) Evaluation of an electronic nose for the early detection of organic overload of anaerobic digesters. Bioprocess Biosyst Eng 36:23–33

    Article  CAS  Google Scholar 

  35. Quality Assurance of Pharmaceuticals: A Compendium of Guidelines and Related Materials (2007) World Health Organization

  36. Gujral P, Amrhein M, Wise BM, Bonvin D (2010) Framework for explicit drift correction in multivariate calibration models. J Chemom 24(7/8):534–543

    Article  CAS  Google Scholar 

  37. Nimsuk N, Nakamoto T (2008) Study on the odor classification in dynamical concentration robust against humidity and temperature changes. Sensors Actuators B: Chem 134(1):252–257

    Article  CAS  Google Scholar 

  38. Kashwan, KR, Bhuyan M (2005) Robust electronic-nose system with temperature and humidity drift compensation for tea and spice flavor discrimination. IEEE Asian Conference on Sensors and the International Conference on new Techniques in Pharmaceutical and Biomedical Research, 5-7 September 2005, Kuala Lumpur, Malasya, pp. 154–158

  39. Romain AC, Nicolas J, Andre P (1997) In situ measurement of olfactive pollution with inorganic semiconductors: limitations due to humidity and temperature influence. Semin Food Anal 2:283–296

    CAS  Google Scholar 

  40. Tomic O, Eklöv T, Kvaal K, Haugen JE (2004) Recalibration of a gas-sensor array system related to sensor replacement. Anal Chim Acta 512:199–206

    Article  CAS  Google Scholar 

  41. Marco S, Pardo A, Davide FA, Di Natale C, D'Amico A, Hierlemann A, Mitrovics J, Schwweizer M, Weimar U, Göpel W (1996) Different strategies for the identification of gas sensing systems. Sensors Actuators B: Chem 34(1):213–223

    Article  CAS  Google Scholar 

  42. Marco S, Samitier J, Morante JR (1995) A novel time-domain method to analyze multicomponent exponential transients. Meas Sci Technol 6(2):135

    Article  CAS  Google Scholar 

  43. Samitier J, Lopez-Villegas JM, Marco S, Camara L, Pardo A, Ruiz O, Morante JR (1994) A new method to analyze signal transients in chemical sensors. Sensors Actuators B: Chem 18(1):308–312

    Article  CAS  Google Scholar 

  44. Gutierrez-Osuna R, Nagle HT, Schiffman SS (1999) Transient response analysis of an electronic nose using multi-exponential models. Sensors Actuators B: Chem 61(1):170–182

    Article  CAS  Google Scholar 

  45. Vilanova X, Llobet E, Alcubilla R, Sueiras JE, Correig X (1996) Analysis of the conductance transient in thick-film tin oxide gas sensors. Sensors Actuators B: Chem 31(3):175–180

    Article  CAS  Google Scholar 

  46. Kearns M, Ron D (1999) Algorithmic stability and sanity-check bounds for leave-one-out cross-validation. Neural Comput 11:1427–1453

    Article  CAS  Google Scholar 

  47. Steyerberg E, Harrell F, Borsboom G, Eijkemans MJ, Vergouwe Y, Habbema JD (2001) Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol 54:774–781

    Article  CAS  Google Scholar 

  48. Lindgren F, Hansen B, Karcher W, Sjöström M, Eriksson L (1996) Model validation by permutation tests: applications to variable selection. J Chemom 10:521–532

    Article  CAS  Google Scholar 

  49. Ojala M, Garriga GC (2010) Permutation tests for studying classifier performance. J Mach Learn Res 11:1833–1863

    Google Scholar 

  50. Westerhuis JA, Hoefsloot HC, Smit S, Vis DJ, Smilde AK, van Velzen EJ, van Duijnhoven JPM, van Dorsten FA (2008) Assessment of PLSDA cross validation. Metabolomics 4(1):81–89

    Article  CAS  Google Scholar 

  51. Rubingh CM, Bijlsma S, Derks EP, Bobeldijk I, Verheij ER, Kochhar S, Smilde AK (2006) Assessing the performance of statistical validation tools for megavariate metabolomics data. Metabolomics 2:53–61

    Article  CAS  Google Scholar 

  52. Fawcett T (2006) An introduction to ROC analysis. Pattern Recog Lett 27(8):861–874

    Article  Google Scholar 

  53. Lasko TA, Bhagwat JG, Zou KH, Ohno-Machado L (2005) The use of receiver operating characteristic curves in biomedical informatics. J Biomed Inf 38:4–415

    Article  Google Scholar 

  54. Gardner M, Altman D (1986) Confidence intervals rather than p-values: estimation rather than hypothesis testing. Br Med J 292:746–750

    Article  CAS  Google Scholar 

  55. Goodman S (1999) Toward evidence-based medical statistics. 1. The p-value fallacy. Ann. Intern Med 130:995–1004

    CAS  Google Scholar 

  56. Goodman S (1999) Toward evidence-based medical statistics. 2. The Bayes factor. Ann Int Med 130:1005–1013

    Article  CAS  Google Scholar 

  57. Kass R, Raftery A (1995) Bayes factors. J Am Stat Assoc 90:773–795

    Article  Google Scholar 

  58. Broadhurst DI, Kell DB (2006) Statistical strategies for avoiding false discoveries in metabolomics and related experiments. Metabolomics 2:171–196

    Article  CAS  Google Scholar 

  59. Kenny LC, Dunn WB, Ellis DI, Myers J, Baker PN, Kell DB (2005) Novel biomarkers for pre-eclampsia detected using metabolomics and machine learning. Metabolomics 1:227–234

    Article  Google Scholar 

  60. Ransohoff DF (2005) Lessons from controversy: ovarian cancer screening and serum proteomics. J Natl Cancer Inst 97:315–319

    Article  CAS  Google Scholar 

  61. Petricoin EF, Ardekani AM, Hitt BA, Levine PJ, Fusaro VA, Steinberg SM, Mills GB, Simone C, Fishman DA, Kohn EC, Liotta LA (2002) Use of proteomic patterns in serum to identify ovarian cancer. Lancet 359(9306):572–577

    Article  CAS  Google Scholar 

  62. West-Nørager M, Bro R, Marini F, Høgdall EV, Høgdall CK, Nedergaard L, Heegaard NH (2009) Feasibility of serodiagnosis of ovarian cancer by mass spectrometry. Anal Chem 81:1907–1913

    Article  Google Scholar 

  63. Vander Heyden Y, Nijhuis A, Smeyers-Verbeke J, Vandeginste BG, Massart DL (2001) Guidance for robustness/ruggedness tests in method validation. J Pharm Biomed Anal 24:723–753

    Article  CAS  Google Scholar 

  64. Zeaiter M, Roger J-M, Bellon-Maurel V, Rutledge DN (2004) Robustness of models developed by multivariate calibration. Part I. TrAC Trends Anal Chem 23:157–170

    Article  CAS  Google Scholar 

  65. Zeaiter M, Roger J-M, Bellon-Maurel V (2005) Robustness of models developed by multivariate calibration. Part II. The influence of pre-processing methods. TrAC Trends Anal Chem 24:437–445

    Article  CAS  Google Scholar 

  66. Lavine BK (2009) Validation of classifiers in comprehensive chemometrics 3:587–598

  67. Esbensen KH, Geladi P (2010) Principles of proper validation: use and abuse of resampling for validation. J Chemom 34:168–187

    Article  Google Scholar 

  68. Filzmoser P, Liebmann B, Varmuza K (2009) Repeated double cross-validation. J Chemom 23:160–171

    Article  CAS  Google Scholar 

  69. Rousseeuw J, Debruyne M, Engelen S, Hubert M (2006) Robustness and outlier detection in chemometrics. Crit Rev Anal Chem 36:221–242

    Article  CAS  Google Scholar 

Download references

Acknowledgments

This work was funded by the Spanish Ministerio de Economía y Competitividad under the project TEC2011-26143. Santiago Marco is member of the consolidated research group SGR2009-0753 by the Generalitat de Catalunya.

Author information

Authors and Affiliations

  1. Signal and Information Processing for Sensing Systems, Department of Biomedical Signals and Instrumentation, Institute for Bioengineering of Catalonia, 08028, Barcelona, Spain

    Santiago Marco

  2. Intelligent Signal Processing, Department of Electronics, Faculty of Physics, Universitat de Barcelona, 08028, Barcelona, Spain

    Santiago Marco

Authors
  1. Santiago Marco
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Santiago Marco.

Additional information

Published in the topical collection Chemosensors and Chemoreception with guest editors Jong-Heun Lee and Hyung-Gi Byun.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Marco, S. The need for external validation in machine olfaction: emphasis on health-related applications. Anal Bioanal Chem 406, 3941–3956 (2014). https://doi.org/10.1007/s00216-014-7807-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00216-014-7807-7

Keywords

Search

Navigation