Why Ecological Momentary Assessment Surveys Go Incomplete: When It Happens and How It Impacts Data

 

 Buy Article Permissions and Reprints

Abstract

Background Ecological momentary assessment (EMA) often requires respondents to complete surveys in the moment to report real-time experiences. Because EMA may seem disruptive or intrusive, respondents may not complete surveys as directed in certain circumstances.

Purpose This article aims to determine the effect of environmental characteristics on the likelihood of instances where respondents do not complete EMA surveys (referred to as survey incompletion), and to estimate the impact of survey incompletion on EMA self-report data.

Research Design An observational study.

Study Sample Ten adults hearing aid (HA) users.

Data Collection and Analysis Experienced, bilateral HA users were recruited and fit with study HAs. The study HAs were equipped with real-time data loggers, an algorithm that logged the data generated by HAs (e.g., overall sound level, environment classification, and feature status including microphone mode and amount of gain reduction). The study HAs were also connected via Bluetooth to a smartphone app, which collected the real-time data logging data as well as presented the participants with EMA surveys about their listening environments and experiences. The participants were sent out to wear the HAs and complete surveys for 1 week. Real-time data logging was triggered when participants completed surveys and when participants ignored or snoozed surveys. Data logging data were used to estimate the effect of environmental characteristics on the likelihood of survey incompletion, and to predict participants' responses to survey questions in the instances of survey incompletion.

Results Across the 10 participants, 715 surveys were completed and survey incompletion occurred 228 times. Mixed effects logistic regression models indicated that survey incompletion was more likely to happen in the environments that were less quiet and contained more speech, noise, and machine sounds, and in the environments wherein directional microphones and noise reduction algorithms were enabled. The results of survey response prediction further indicated that the participants could have reported more challenging environments and more listening difficulty in the instances of survey incompletion. However, the difference in the distribution of survey responses between the observed responses and the combined observed and predicted responses was small.

Conclusion The present study indicates that EMA survey incompletion occurs systematically. Although survey incompletion could bias EMA self-report data, the impact is likely to be small.

Publication History

Received: 21 April 2020

Accepted: 01 June 2020

Article published online:
15 December 2020

© 2020. American Academy of Audiology. This article is published by Thieme.

Thieme Medical Publishers, Inc.
333 Seventh Avenue, 18th Floor, New York, NY 10001, USA

• References

• 1 Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Annu Rev Clin Psychol 2008; 4: 1-32
  CrossrefPubMedGoogle Scholar
• 2 Timmer BHB, Hickson L, Launer S. Ecological momentary assessment: feasibility, construct validity, and future applications. Am J Audiol 2017; 26 (3S): 436-442
  CrossrefPubMedGoogle Scholar
• 3 Wu YH, Stangl E, Zhang X, Bentler RA. Construct validity of the ecological momentary assessment in audiology research. J Am Acad Audiol 2015; 26 (10) 872-884
  Article in Thieme ConnectPubMedGoogle Scholar
• 4 Wu YH, Stangl E, Chipara O, Zhang X. Test-retest reliability of ecological momentary assessment in audiology research. J Am Acad Audiol, In press
  Google Scholar
• 5 Wu YH, Stangl E, Chipara O, Gudjonsdottir A, Oleson J, Bentler RA. Comparison of in-situ and retrospective self-reports on assessing hearing aid outcomes. J Am Acad Audiol, In press
  Google Scholar
• 6 Preminger JE, Cunningham DR. Case-study analysis of various field study measures. J Am Acad Audiol 2003; 14 (01) 39-55
  Article in Thieme ConnectPubMedGoogle Scholar
• 7 Walden BE, Surr RK, Cord MT, Dyrlund O. Predicting hearing aid microphone preference in everyday listening. J Am Acad Audiol 2004; 15 (05) 365-396
  Article in Thieme ConnectPubMedGoogle Scholar
• 8 Wu YH, Bentler RA. Impact of visual cues on directional benefit and preference: part II--field tests. Ear Hear 2010; 31 (01) 35-46
  CrossrefPubMedGoogle Scholar
• 9 Wu YH, Bentler RA. Do older adults have social lifestyles that place fewer demands on hearing?. J Am Acad Audiol 2012; 23 (09) 697-711
  Article in Thieme ConnectPubMedGoogle Scholar
• 10 Bentler R, Wu YH, Kettel J, Hurtig R. Digital noise reduction: outcomes from laboratory and field studies. Int J Audiol 2008; 47 (08) 447-460
  CrossrefPubMedGoogle Scholar
• 11 Galvez G, Turbin MB, Thielman EJ, Istvan JA, Andrews JA, Henry JA. Feasibility of ecological momentary assessment of hearing difficulties encountered by hearing aid users. Ear Hear 2012; 33 (04) 497-507
  CrossrefPubMedGoogle Scholar
• 12 Timmer BHB, Hickson L, Launer S. Do hearing aids address real-world hearing difficulties for adults with mild hearing impairment? Results from a pilot study using ecological momentary assessment. Trends Hear 2018; 22: 2331216518783608
  Google Scholar
• 13 Wu YH, Stangl E, Chipara O, Hasan SS, DeVries S, Oleson J. Efficacy and effectiveness of advanced hearing aid directional and noise reduction technologies for older adults with mild to moderate hearing loss. Ear Hear 2019; 40 (04) 805-822
  CrossrefPubMedGoogle Scholar
• 14 Henry JA, Galvez G, Turbin MB, Thielman EJ, McMillan GP, Istvan JA. Pilot study to evaluate ecological momentary assessment of tinnitus. Ear Hear 2012; 33 (02) 179-290
  CrossrefPubMedGoogle Scholar
• 15 Pentony S, Xu J, Wu YH, Galster J. Using ecological momentary assessment in audiology research: the participants' perspective. Paper presented at the American Auditory Society, Scottsdale, AZ; 2018. Available at: https://haar.lab.uiowa.edu/sites/haar.lab.uiowa.edu/files/wysiwyg_uploads/pentony_et_al_aas_2018_2010.pdf
  Google Scholar
• 16 Shiffman S. Designing protocols for ecological momentary assessment. In: Stone A, Shiffman S, Atienza A, Nebeling L. eds. The Science of Real-Time Data Capture: Self-Reports in Health Research. New York, NY: Oxford University Press; 2007: 27-53
  Google Scholar
• 17 Ricketts TA, Bentler RA, Mueller HG. Pre-fitting tests using frequency-specific measures. In: Essentials of Modern Hearing Aids: Selection, Fitting, and Verification. San Diego, CA: Plural Publishing; 2018: 101-134
  Google Scholar
• 18 Humes LE, Rogers SE, Main AK, Kinney DL. The acoustic environments in which older adults wear their hearing aids: insights from datalogging sound environment classification. Am J Audiol 2018; 27 (04) 594-603
  CrossrefPubMedGoogle Scholar