The AVATAR-approach: how real-life listening affects speech intelligibility and listening effort

Having a successful conversation does not only depend on good auditory processing abilities. During complex real-life listening situations, cognition, multitasking skills and the processing of visual (speech) cues are well known to be of particular importance. Nevertheless, current speech-in-noise (SPIN) tests take these aspects only partially into account since they are developed from a static, one-dimensional auditory perspective. This results in a large variation in outcome measurements of persons with hearing impairment and hearing aid-users.

To close the gap between well-controlled SPIN measurements and self-report approaches, we developed the AVATAR-paradigm: a comprehensive method and test set-up for the real-life assessment of auditory functioning. A unimodal SPIN-measurement is extended to a multitasking test that incorporates both auditory and visual cues as well as cognition. Realistic listening scenarios are presented on a large screen, including computer animated avatars speaking to the test person.

The present project employed the AVATAR-approach to investigate the effect of multitasking on speech intelligibility (SI) and listening effort (LE). Multiple tasks were combined into listening scenarios of increasing complexity. In the most difficult situation the test person had to execute a SPIN-task together with three secondary tasks: an auditory localization test, detection of direction of sounds passing by and short-term memory storage of visual stimuli, imposing an extra cognitive load. Results showed that, for normal hearing adults of 18-30 years old, SI was robust to the amount of tasks that had to be performed simultaneously. On the contrary, LE during speech-processing significantly increased when the listening situation became more complex. Furthermore, the impact of multitasking on LE was more prominent for older and/or hearing impaired adults. Implications of these results will be discussed, together with data on the feasibility and sensitivity of the AVATAR-approach.