Abstract
The cochlear implant is one of the most successful rehabilitation prostheses, allowing deaf and severely hearing-impaired persons to hear again through electrical stimulation of the auditory nerve. In order to properly understand speech with a cochlear implant, a trained audiologist needs to adjust the sound processing and stimulation settings, which are highly subject-specific. Furthermore, this fitting procedure is time consuming, occurs only during infrequent visits to the clinic, and relies on behavioral feedback from the subject, which makes it challenging to do properly in young children and persons with cognitive impairment. Integrating a brain-computer interface (BCI) can alleviate the issues with the current fitting paradigms. If the implant can measure neural responses to speech, it can objectively assess how well the user understands speech and automatically adapt its sound processing settings if needed. This neuro-monitoring can happen continuously in the user’s everyday listening environment and does not rely on behavioral input. We present an overview of our ongoing research towards such neuro-steered hearing implants.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
S.J. Aiken, T.W. Picton, Human cortical responses to the speech envelope. Ear Hear. (2008). https://doi.org/10.1097/AUD.0b013e31816453dc
A. Aroudi, S. Doclo, Cognitive-driven binaural beamforming using eeg-based auditory attention decoding. IEEE/ACM Trans. Audio Speech Lang. Process. (2020). https://doi.org/10.1109/taslp.2020.2969779
J. Brown, P.J. Abbas, Electrically evoked whole-nerve action potentials: Data from human cochlear implant users. J. Acoust. Soc. Am. 88(September 1990), 1385–1391 (1990)
N. Das, A. Bertrand, T. Francart, EEG-based auditory attention detection: Boundary conditions for background noise and speaker positions. J. Neural Eng. (2018). https://doi.org/10.1088/1741-2552/aae0a6
N. Das, J. Vanthornhout, T. Francart, A. Bertrand, Stimulus-aware spatial filtering for single-trial neural response and temporal response function estimation in high-density EEG with applications in auditory research. NeuroImage (2020). https://doi.org/10.1016/j.neuroimage.2019.116211
L. Decruy, N. Das, E. Verschueren, T. Francart, The self-assessed békesy procedure: validation of a method to measure intelligibility of connected discourse. Trends Hear. (2018). https://doi.org/10.1177/2331216518802702
L. Decruy, D. Lesenfants, J. Vanthornhout, T. Francart, Top-down modulation of neural envelope tracking: the interplay with behavioral, self-report and neural measures of listening effort. BioRxiv (2019a). https://doi.org/10.1101/815365
L. Decruy, J. Vanthornhout, T. Francart, Evidence for enhanced neural tracking of the speech envelope underlying age-related speech-in-noise difficulties. J. Neurophysiol. (2019b). https://doi.org/10.1152/jn.00687.2018
L. Decruy, J. Vanthornhout, T. Francart, Hearing impairment is associated with enhanced neural tracking of the speech envelope. BioRxiv (2020). https://doi.org/10.1101/815530
H. Deprez, M. Hofmann, A. Van Wieringen, J. Wouters, M. Moonen, Cochlear implant artifact rejection in electrically evoked auditory steady state responses. 22nd European Signal Processing Conference (EUSIPCO), 1995–1999 (2014). https://doi.org/10.1017/CBO9781107415324.004
G.M. Di Liberto, J.A. O’Sullivan, E.C. Lalor, Low-frequency cortical entrainment to speech reflects phoneme-level processing. Curr. Biol. 25(19), 2457–2465 (2015). https://doi.org/10.1016/j.cub.2015.08.030
G.M. Di Liberto, E.C. Lalor, Isolating neural indices of continuous speech processing at the phonetic level. Adv. Exp. Med. Biol. (2016). https://doi.org/10.1007/978-3-319-25474-6_35
N. Ding, J.Z. Simon, Neural coding of continuous speech in auditory cortex during monaural and dichotic listening. J. Neurophysiol. 107(1), 78–89 (2012). https://doi.org/10.1152/jn.00297.2011
N. Ding, J.Z. Simon, Adaptive temporal encoding leads to a background-insensitive cortical representation of speech. J. Neurosci. 33(13), 5728–5735 (2013). https://doi.org/10.1523/JNEUROSCI.5297-12.2013
M.F. Dorman, B.S.B.S. Wilson, The design and function of cochlear implants. Am. Sci. 92(6), 436–445 (2004). https://doi.org/10.1511/2004.49.942
E.A. Durant, G.H. Wakefield, D.J. Van Tasell, M.E. Rickert, Efficient perceptual tuning of hearing aids with genetic algorithms. IEEE Trans. Speech Audio Process. (2004). https://doi.org/10.1109/TSA.2003.822640
M. Finke, M. Billinger, A. Büchner, Toward automated cochlear implant fitting procedures based on event-related potentials. Ear Hear. 38(2), e118–e127 (2017). https://doi.org/10.1097/AUD.0000000000000377
T. Francart, A. Van Wieringen, J. Wouters, Comparison of fluctuating maskers for speech recognition tests. Int. J. Audiol. (2010). https://doi.org/10.3109/14992027.2010.505582
S. Geirnaert, T. Francart, A. Bertrand, An interpretable performance metric for auditory attention decoding algorithms in a context of neuro-steered gain control. IEEE Trans. Neural Syst. Rehabil. Eng. (2020). https://doi.org/10.1109/TNSRE.2019.2952724
R. Gransier, H. Deprez, M. Hofmann, M. Moonen, A.Van Wieringen, J. Wouters, Auditory steady-state responses in cochlear implant users: effect of modulation frequency and stimulation artifacts. Hear. Res. 335, 1–43 (2016). https://doi.org/10.1016/j.heares.2016.03.006
S. Haumann, G. Bauernfeind, M.J. Teschner, I. Schierholz, M.G. Bleichner, A. Büchner, T. Lenarz, Epidural recordings in cochlear implant users. J. Neural Eng. (2019). https://doi.org/10.1088/1741-2552/ab1e80
M. Hofmann, J. Wouters, Electrically evoked auditory steady state responses in cochlear implant users. J. Assoc. Res. Otolaryngol.: JARO 11(2) (2010). https://doi.org/10.1007/s10162-009-0201-z
M. Hofmann, J. Wouters, Improved electrically evoked auditory steady-state response thresholds in humans. J. Assoc. Res. Otolaryngol. 13(4), 573–589 (2012)
D. Lesenfants, T. Francart, The interplay of top-down focal attention and the cortical tracking of speech. BioRxiv (2019). https://doi.org/10.1101/813147
D. Lesenfants, J. Vanthornhout, E. Verschueren, L. Decruy, T. Francart, Predicting individual speech intelligibility from the cortical tracking of acoustic- and phonetic-level speech representations. Hear. Res. (2019). https://doi.org/10.1016/j.heares.2019.05.006
M. McLaughlin, A. Lopez Valdes, R.B. Reilly, F.-G. Zeng, Cochlear implant artifact attenuation in late auditory evoked potentials: a single channel approach. Hear. Res. 302, 84–95 (2013). https://doi.org/10.1016/j.heares.2013.05.006
M. McLaughlin, T. Lu, A. Dimitrijevic, F.G. Zeng, Towards a closed-loop cochlear implant system: Application of embedded monitoring of peripheral and central neural activity. IEEE Trans. Neural Syst. Rehabil. Eng. 20(4), 443–454 (2012). https://doi.org/10.1109/TNSRE.2012.2186982
K.V. Nourski, C.P. Etler, J.F. Brugge, H. Oya, H. Kawasaki, R.A. Reale, P.J. Abbas, C.J. Brown, M.A. Howard, Direct recordings from the auditory cortex in a cochlear implant user. JARO: J. Assoc. Res. Otolaryngol. (2013). https://doi.org/10.1007/s10162-013-0382-3
J.A. O’Sullivan, A.J. Power, N. Mesgarani, S. Rajaram, J.J. Foxe, B.G. Shinn-Cunningham, M. Slaney, S.A. Shamma, E.C. Lalor, Attentional selection in a Cocktail Party Environment can be decoded from single-trial EEG. Cereb. Cortex 25(7), 1697–1706 (2015). https://doi.org/10.1093/cercor/bht355
J.E. Peelle, M.H. Davis, Neural oscillations carry speech rhythm through to comprehension. Front. Psychol. (2012). https://doi.org/10.3389/fpsyg.2012.00320
A. Presacco, J.Z. Simon, S. Anderson, Evidence of degraded representation of speech in noise, in the aging midbrain and cortex. J. Neurophysiol. (2016). https://doi.org/10.1152/jn.00372.2016
P. Ríos-López, N. Molinaro, M. Bourguignon, M. Lallier, Development of neural oscillatory activity in response to speech in children from four to six years old. Dev. Sci. (2020). https://doi.org/10.1111/desc.12947
R.V. Shannon, F.G. Zeng, V. Kamath, J. Wygonski, M. Ekelid, Speech recognition with primarily temporal cues. Science (1995). https://doi.org/10.1126/science.270.5234.303
B.G. Shinn-Cunningham, Object-based auditory and visual attention. Trends Cogn. Sci. (2008). https://doi.org/10.1016/j.tics.2008.02.003
B. Somers, T. Francart, A. Bertrand, A generic EEG artifact removal algorithm based on the multi-channel Wiener filter. J. Neural Eng. (2018). https://doi.org/10.1088/1741-2552/aaac92
B. Somers, C.J. Long, Francart, EEG-based diagnostics of the auditory system using cochlear implant electrodes as sensors. BioRxiv (2020)
B. Somers, E. Verschueren, T. Francart, Neural tracking of the speech envelope in cochlear implant users. J. Neural Eng. (2019). https://doi.org/10.1088/1741-2552/aae6b9
V.D. Tejani, P.J. Abbas, C.J. Brown, J. Woo, An improved method of obtaining electrocochleography recordings from Nucleus Hybrid cochlear implant users. Hear. Res. (2019). https://doi.org/10.1016/j.heares.2019.01.002
S. Van Eyndhoven, T. Francart, A. Bertrand, EEG-informed attended speaker extraction from recorded speech mixtures with application in neuro-steered hearing prostheses. IEEE Trans. Biomed. Eng. (2017). https://doi.org/10.1109/TBME.2016.2587382
M. Vander Ghinst, M. Bourguignon, M. Niesen, V. Wens, S. Hassid, G. Choufani, V. Jousmäki, R. Hari, S. Goldman, X. De Tiège, Cortical tracking of speech-in-noise develops from childhood to adulthood. J. Neurosci. (2019). https://doi.org/10.1523/JNEUROSCI.1732-18.2019
J. Vanthornhout, L. Decruy, T. Francart, Effect of task and attention on neural tracking of speech. Front. Neurosci. (2019). https://doi.org/10.3389/fnins.2019.00977
J. Vanthornhout, L. Decruy, J. Wouters, J.Z. Simon, T. Francart, Speech intelligibility predicted from neural entrainment of the speech envelope. JARO: J Assoc. Res. Otolaryngol. (2018). https://doi.org/10.1007/s10162-018-0654-z
E. Verschueren, B. Somers, T. Francart, Neural envelope tracking as a measure of speech understanding in cochlear implant users. Hear. Res. (2019a). https://doi.org/10.1016/j.heares.2018.12.004
E. Verschueren, J. Vanthornhout, T. Francart, The effect of stimulus choice on an EEG-based objective measure of speech intelligibility. BioRxiv (2019b). https://doi.org/10.1101/421727
A.S. Visram, H. Innes-Brown, W. El-Deredy, C.M. McKay, Cortical auditory evoked potentials as an objective measure of behavioral thresholds in cochlear implant users. Hear. Res. (2015). https://doi.org/10.1016/j.heares.2015.04.012
G.H. Wakefield, C. Van Den Honert, W. Parkinson, S. Lineaweaver, Genetic algorithms for adaptive psychophysical procedures: Recipient-directed design of speech-processor MAPs. Ear Hear. (2005). https://doi.org/10.1097/00003446-200508001-00008
J. Wouters, H.J. Mcdermott, T. Francart, Sound coding in cochlear implants: from electric pulses to hearing. IEEE Signal Process. Mag. 32(2), 67–80 (2015). https://doi.org/10.1109/MSP.2014.2371671
F.G. Zeng, S. Rebscher, W. Harrison, X. Sun, H. Feng, Cochlear implants: System design, integration, and evaluation. IEEE Rev. Biomed. Eng. (2008). https://doi.org/10.1109/RBME.2008.2008250
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Somers, B., Lesenfants, D., Vanthornhout, J., Decruy, L., Verschueren, E., Francart, T. (2021). Interfacing Hearing Implants with the Brain: Closing the Loop with Intracochlear Brain Recordings. In: Guger, C., Allison, B.Z., Tangermann, M. (eds) Brain-Computer Interface Research. SpringerBriefs in Electrical and Computer Engineering. Springer, Cham. https://doi.org/10.1007/978-3-030-60460-8_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-60460-8_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60459-2
Online ISBN: 978-3-030-60460-8
eBook Packages: EngineeringEngineering (R0)