Abstract
Cochlear implant (CI) listeners typically perform poorly on tasks involving the pitch of complex tones. This limitation in performance is thought to be mainly due to the restricted number of active channels and the broad current spread that leads to channel interactions and subsequent loss of precise spectral information, with temporal information limited primarily to temporal-envelope cues. Little is known about the degree of spectral resolution required to perceive combinations of multiple pitches, or a single pitch in the presence of other interfering tones in the same spectral region. This study used noise-excited envelope vocoders that simulate the limited resolution of CIs to explore the perception of multiple pitches presented simultaneously. The results show that the resolution required for perceiving multiple complex pitches is comparable to that found in a previous study using single complex tones. Although relatively high performance can be achieved with 48 channels, performance remained near chance when even limited spectral spread (with filter slopes as steep as 144 dB/octave) was introduced to the simulations. Overall, these tight constraints suggest that current CI technology will not be able to convey the pitches of combinations of spectrally overlapping complex tones.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Allen EJ, Oxenham AJ (2014) Symmetric interactions and interference between pitch and timbre. J Acoust Soc Am 135:1371–1379. https://doi.org/10.1121/1.4863269
Bates D, Mächler M, Bolker B, Walker S (2014) Fitting linear mixed-effects models using lme4
Bernstein JG, Oxenham AJ (2003) Pitch discrimination of diotic and dichotic tone complexes: harmonic resolvability or harmonic number? J Acoust Soc Am 113:3323–3334. https://doi.org/10.1121/1.1572146
Bernstein JGW, Oxenham AJ (2006) The relationship between frequency selectivity and pitch discrimination: sensorineural hearing loss. J Acoust Soc Am 120:3929–3945. https://doi.org/10.1121/1.2372452
Bierer JA, Middlebrooks JC (2002) Auditory cortical images of cochlear-implant stimuli: dependence on electrode configuration. J Neurophysiol 87:478–492. https://doi.org/10.1152/jn.00212.2001
Bolker BM, Brooks ME, Clark CJ, Geange SW, Poulsen JR, Stevens MH, White JS (2009) Generalized linear mixed models: a practical guide for ecology and evolution. Trends Ecol Evol 24:127–135. https://doi.org/10.1016/J.TREE.2008.10.008
Carlyon RP (1996) Encoding the fundamental frequency of a complex tone in the presence of a spectrally overlapping masker. J Acoust Soc Am 99:517–524. https://doi.org/10.1121/1.414510
Carlyon RP, Deeks JM, McKay CM (2010) The upper limit of temporal pitch for cochlear-implant listeners: stimulus duration, conditioner pulses, and the number of electrodes stimulated. J Acoust Soc Am 127:1469–1478. https://doi.org/10.1121/1.3291981
Chatterjee M, Peng S-C (2008) Processing F0 with cochlear implants: modulation frequency discrimination and speech intonation recognition. Hear Res 235:143–156. https://doi.org/10.1016/j.heares.2007.11.004
Cosentino S, Carlyon RP, Deeks JM, Parkinson W, Bierer JA (2016) Rate discrimination, gap detection and ranking of temporal pitch in cochlear implant users. J Assoc Res Otolaryngol 17:371–382. https://doi.org/10.1007/s10162-016-0569-5
Deroche MLD, Kulkarni AM, Christensen JA et al (2016) Deficits in the sensitivity to pitch sweeps by school-aged children wearing cochlear implants. Front Neurosci:10. https://doi.org/10.3389/fnins.2016.00073
Ewert SD (2013) AFC - a modular framework for running psychoacoustic experiments and computational perception models. Proc Conf Acoust AIA-DAGA 2013:1326–1329
Fielden CA, Kluk K, Boyle PJ, McKay CM (2015) The perception of complex pitch in cochlear implants: a comparison of monopolar and tripolar stimulation. J Acoust Soc Am 138:2524–2536. https://doi.org/10.1121/1.4931910
Fu Q-J, Chinchilla S, Galvin JJ (2004) The role of spectral and temporal cues in voice gender discrimination by normal-hearing listeners and cochlear implant users. J Assoc Res Otolaryngol JARO 5:253–260. https://doi.org/10.1007/s10162-004-4046-1
Galvin JJ, Fu Q-J, Nogaki G (2007) Melodic contour identification by cochlear implant listeners. Ear Hear 28:302–319. https://doi.org/10.1097/01.aud.0000261689.35445.20
Gaudrain E, Başkent D (2018) Discrimination of voice pitch and vocal-tract length in cochlear implant users. Ear Hear 39:226–237. https://doi.org/10.1097/AUD.0000000000000480
Glasberg BR, Moore BCJ (1986) Auditory filter shapes in subjects with unilateral and bilateral cochlear impairments. J Acoust Soc Am 79:1020–1033. https://doi.org/10.1121/1.393374
Glasberg BR, Moore BCJ (1990) Derivation of auditory filter shapes from notched-noise data. Hear Res 47:103–138. https://doi.org/10.1016/0378-5955(90)90170-T
Graves JE, Oxenham AJ (2019) Pitch discrimination with mixtures of three concurrent harmonic complexes. J Acoust Soc Am 145:2072–2083. https://doi.org/10.1121/1.5096639
Houtgast T (1972) Psychophysical evidence for lateral inhibition in hearing. J Acoust Soc Am 51:1885–1894. https://doi.org/10.1121/1.1913048
Houtsma AJM, Smurzynski J (1990) Pitch identification and discrimination for complex tones with many harmonics. J Acoust Soc Am 87:304–310. https://doi.org/10.1121/1.399297
Jaeger TF (2008) Categorical data analysis: away from ANOVAs (transformation or not) and towards logit mixed models. J Mem Lang 59:434–446. https://doi.org/10.1016/J.JML.2007.11.007
Johnson LA, Santina CCD, Wang X (2016) Selective neuronal activation by cochlear implant stimulation in auditory cortex of awake primate. J Neurosci 36:12468–12484. https://doi.org/10.1523/JNEUROSCI.1699-16.2016
Johnson LA, Santina CCD, Wang X (2017) Representations of time-varying cochlear implant stimulation in auditory cortex of awake marmosets (Callithrix jacchus). J Neurosci 37:7008–7022. https://doi.org/10.1523/JNEUROSCI.0093-17.2017
Kong Y-Y, Carlyon RP (2010) Temporal pitch perception at high rates in cochlear implants. J Acoust Soc Am 127:3114–3123. https://doi.org/10.1121/1.3372713
Kong Y-Y, Cruz R, Jones AJ, Zeng F-G (2004) Music perception with temporal cues in acoustic and electric hearing. Ear Hear 25(2):173–185
Kong Y-Y, Mullangi A, Marozeau J, Epstein M (2011) Temporal and spectral cues for musical timbre perception in electric hearing. J Speech Lang Hear Res JSLHR 54:981–994. https://doi.org/10.1044/1092-4388
Kreft HA, Nelson DA, Oxenham AJ (2013) Modulation frequency discrimination with modulated and unmodulated interference in normal hearing and in cochlear-implant users. J Assoc Res Otolaryngol 14:591–601. https://doi.org/10.1007/s10162-013-0391-2
Lau BK, Mehta AH, Oxenham AJ (2017) Super-optimal perceptual integration suggests a place-based representation of pitch at high frequencies. J Neurosci:1507–1517. https://doi.org/10.1523/JNEUROSCI.1507-17.2017
Lenth RV (2016) Least-squares means: the R package lsmeans. J Stat Softw 69. https://doi.org/10.18637/jss.v069.i01
Looi V, McDermott H, McKay C, Hickson L (2008) Music perception of cochlear implant users compared with that of hearing aid users. Ear Hear 29:421–434. https://doi.org/10.1097/AUD.0b013e31816a0d0b
Looi V, McDermott H, McKay C, Hickson L (2007) Comparisons of quality ratings for music by cochlear implant and hearing aid users. Ear Hear 28:59S–61S. https://doi.org/10.1097/AUD.0b013e31803150cb
Luke SG (2017) Evaluating significance in linear mixed-effects models in R. Behav Res Methods 49:1494–1502. https://doi.org/10.3758/s13428-016-0809-y
Macherey O, Deeks JM, Carlyon RP (2011) Extending the limits of place and temporal pitch perception in cochlear implant users. J Assoc Res Otolaryngol 12:233–251. https://doi.org/10.1007/s10162-010-0248-x
McDermott HJ (2004) Music perception with cochlear implants: a review. Trends Amplif 8:49–82. https://doi.org/10.1177/108471380400800203
Mehta AH, Oxenham AJ (2017) Vocoder simulations explain complex pitch perception limitations experienced by cochlear implant users. J Assoc Res Otolaryngol 1–14. https://doi.org/10.1007/s10162-017-0632-x
Mehta AH, Oxenham AJ (2018) Fundamental-frequency discrimination based on temporal-envelope cues: effects of bandwidth and interference. J Acoust Soc Am 144:EL423–EL428. https://doi.org/10.1121/1.5079569
Micheyl C, Hunter C, Oxenham AJ (2010a) Auditory stream segregation and the perception of across-frequency synchrony. J Exp Psychol Hum Percept Perform 36:1029–1039. https://doi.org/10.1037/a0017601
Micheyl C, Keebler MV, Oxenham AJ (2010b) Pitch perception for mixtures of spectrally overlapping harmonic complex tones. J Acoust Soc Am 128:257–269. https://doi.org/10.1121/1.3372751
Middlebrooks JC, Snyder RL (2007) Auditory prosthesis with a penetrating nerve array. J Assoc Res Otolaryngol 8:258–279. https://doi.org/10.1007/s10162-007-0070-2
Middlebrooks JC, Snyder RL (2010) Selective electrical stimulation of the auditory nerve activates a pathway specialized for high temporal acuity. J Neurosci 30:1937–1946. https://doi.org/10.1523/JNEUROSCI.4949-09.2010
Moore BCJ (2019) The roles of temporal envelope and fine structure information in auditory perception. Acoust Sci Technol 40:61–83. https://doi.org/10.1250/ast.40.61
Moore BCJ (1978) Psychophysical tuning curves measured in simultaneous and forward masking. J Acoust Soc Am 63:524–532. https://doi.org/10.1121/1.381752
Moore BCJ, Huss M, Vickers DA, Glasberg BR, Alcántara JI (2000) A test for the diagnosis of dead regions in the cochlea. Br J Audiol 34:205–224. https://doi.org/10.3109/03005364000000131
Nemer JS, Kohlberg GD, Mancuso DM, Griffin BM, Certo MV, Chen SY, Chun MB, Spitzer JB, Lalwani AK (2017) Reduction of the harmonic series influences musical enjoyment with cochlear implants. Otol Neurotol 38:31–37. https://doi.org/10.1097/MAO.0000000000001250
O’Neill ER, Kreft HA, Oxenham AJ (2019) Speech perception with spectrally non-overlapping maskers as measure of spectral resolution in cochlear implant users. J Assoc Res Otolaryngol 20:151–167. https://doi.org/10.1007/s10162-018-00702-2
Oxenham AJ (2012) Pitch Perception. J Neurosci 32:13335–13338. https://doi.org/10.1523/JNEUROSCI.3815-12.2012
Oxenham AJ (2018) How we hear: the perception and neural coding of sound. Annu Rev Psychol 69:27–50. https://doi.org/10.1146/annurev-psych-122216-011635
Oxenham AJ, Bernstein JGW, Penagos H (2004) Correct tonotopic representation is necessary for complex pitch perception. Proc Natl Acad Sci U S A 101:1421–1425. https://doi.org/10.1073/pnas.0306958101
Oxenham AJ, Kreft HA (2014) Speech perception in tones and noise via cochlear implants reveals influence of spectral resolution on temporal processing. Trends Hear 18:2331216514553783. https://doi.org/10.1177/2331216514553783
Oxenham AJ, Micheyl C, Keebler MV, Loper A, Santurette S (2011) Pitch perception beyond the traditional existence region of pitch. Proc Natl Acad Sci 108:7629–7634. https://doi.org/10.1073/pnas.1015291108
Oxenham AJ, Shera CA (2003) Estimates of human cochlear tuning at low levels using forward and simultaneous masking. J Assoc Res Otolaryngol 4:541–554. https://doi.org/10.1007/s10162-002-3058-y
Parncutt R, Reisinger D, Fuchs A, Kaiser F (2019) Consonance and prevalence of sonorities in Western polyphony: roughness, harmonicity, familiarity, evenness, diatonicity. J New Music Res 48:1–20. https://doi.org/10.1080/09298215.2018.1477804
Pinyon JL, Tadros SF, Froud KE et al (2014) Close-field electroporation gene delivery using the cochlear implant electrode array enhances the bionic ear. Sci Transl Med 6:233ra54-233ra54. https://doi.org/10.1126/scitranslmed.3008177
Plomp R (1967) Pitch of complex tones. J Acoust Soc Am 41:1526–1533. https://doi.org/10.1121/1.1910515
Qin MK, Oxenham AJ (2005) Effects of envelope-vocoder processing on f0 discrimination and concurrent-vowel identification. Ear Hear 26:451–460
Raggio MW, Schreiner CE (1999) Neuronal responses in cat primary auditory cortex to electrical cochlear stimulation. III. Activation patterns in short- and long-term deafness. J Neurophysiol 82:3506–3526. https://doi.org/10.1152/jn.1999.82.6.3506
R Core Team (2018) R: a language and environment for statistical computing. Vienna Austria: R Foundation for Statistical Computing. Retrieved from https://www.R-project.org/
Ritsma RJ (1967) Frequencies dominant in the perception of the pitch of complex sounds. J Acoust Soc Am 42:191–198. https://doi.org/10.1121/1.1910550
Rosen S, Souza P, Ekelund C, Majeed AA (2013) Listening to speech in a background of other talkers: effects of talker number and noise vocoding. J Acoust Soc Am 133:2431–2443. https://doi.org/10.1121/1.4794379
Shera CA, Guinan JJ, Oxenham AJ (2002) Revised estimates of human cochlear tuning from otoacoustic and behavioral measurements. Proc Natl Acad Sci 99:3318–3323. https://doi.org/10.1073/pnas.032675099
Shofner WP, Chaney M (2013) Processing pitch in a nonhuman mammal (Chinchilla laniger). J Comp Psychol 127:142–153. https://doi.org/10.1037/a0029734
Singmann et al. (2018) Analysis of factorial experiments, package ‘afex’. R Package version 0.13—145, 1–44. Comprehensive R Archive Network (CRAN)
Stickney GS, Zeng F-G, Litovsky R, Assmann PF (2004) Cochlear implant speech recognition with speech maskers. J Acoust Soc Am 116:1081–1091. https://doi.org/10.1121/1.1772399
Sumner CJ, Wells TT, Bergevin C, Sollini J, Kreft HA, Palmer AR, Oxenham AJ, Shera CA (2018) Mammalian behavior and physiology converge to confirm sharper cochlear tuning in humans. Proc Natl Acad Sci 115:11322–11326. https://doi.org/10.1073/pnas.1810766115
Walker KM, Gonzalez R, Kang JZ et al (2019) Across-species differences in pitch perception are consistent with differences in cochlear filtering. ELife 8:e41626. https://doi.org/10.7554/eLife.41626
Whitmal NA, Poissant SF, Freyman RL, Helfer KS (2007) Speech intelligibility in cochlear implant simulations: effects of carrier type, interfering noise, and subject experience. J Acoust Soc Am 122:2376–2388. https://doi.org/10.1121/1.2773993
Wise AK, Tan J, Wang Y, Caruso F, Shepherd RK (2016) Improved auditory nerve survival with nanoengineered supraparticles for neurotrophin delivery into the deafened cochlea. PLOS ONE 11:e0164867. https://doi.org/10.1371/journal.pone.0164867
Wright et al. (1987) Hair cell distributions in the normal human cochlea. Acta Otolaryngol (Stockh) 104:4–48. https://doi.org/10.3109/00016488709098971
Zeng F-G, Tang Q, Lu T (2014) Abnormal pitch perception produced by cochlear implant stimulation. PLoS ONE 9:e88662. https://doi.org/10.1371/journal.pone.0088662
Funding
This work is supported by NIH grant R01DC005216 (AJO).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
All participants provided written informed consent and were compensated for their time. The experiments were conducted at the University of Minnesota, Twin Cities. The University of Minnesota Institutional Review Board approved all protocols.
Conflict of Interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Mehta, A.H., Lu, H. & Oxenham, A.J. The Perception of Multiple Simultaneous Pitches as a Function of Number of Spectral Channels and Spectral Spread in a Noise-Excited Envelope Vocoder. JARO 21, 61–72 (2020). https://doi.org/10.1007/s10162-019-00738-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10162-019-00738-y