Abstract
Pitch plays a crucial role in speech and music, but is highly degraded for people with cochlear implants, leading to severe communication challenges in noisy environments. Pitch is determined primarily by the first few spectrally resolved harmonics of a tone. In implants, access to this pitch is limited by poor spectral resolution, due to the limited number of channels and interactions between adjacent channels. Here we used noise-vocoder simulations to explore how many channels, and how little channel interaction, are required to elicit pitch. Results suggest that two to four times the number of channels are needed, along with interactions reduced by an order of magnitude, than available in current devices. These new constraints not only provide insights into the basic mechanisms of pitch coding in normal hearing but also suggest that spectrally based complex pitch is unlikely to be generated in implant users without significant changes in the method or site of stimulation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abberton E, Fourcin AJ (1978) Intonation and speaker identification. Lang Speech 21:305–318. doi:10.1177/002383097802100405
Allen EJ, Oxenham AJ (2014) Symmetric interactions and interference between pitch and timbre. J Acoust Soc Am 135:1371–1379. doi:10.1121/1.4863269
Arehart KH (1994) Effects of harmonic content on complex-tone fundamental-frequency discrimination in hearing-impaired listeners. J Acoust Soc Am 95:3574–3585. doi:10.1121/1.409975
Bernstein JGW, Oxenham AJ (2006) The relationship between frequency selectivity and pitch discrimination: sensorineural hearing loss. J Acoust Soc Am 120:3929–3945. doi:10.1121/1.2372452
Bingabr M, Espinoza-Varas B, Loizou PC (2008) Simulating the effect of spread of excitation in cochlear implants. Hear Res 241:73–79. doi:10.1016/j.heares.2008.04.012
Burns EM, Viemeister NF (1976) Nonspectral pitch. J Acoust Soc Am 60:863–869. doi:10.1121/1.381166
Caclin A, McAdams S, Smith BK, Winsberg S (2005) Acoustic correlates of timbre space dimensions: a confirmatory study using synthetic tones. J Acoust Soc Am 118:471–482. doi:10.1121/1.1929229
Cariani PA, Delgutte B (1996) Neural correlates of the pitch of complex tones. I. Pitch and pitch salience. J Neurophysiol 76:1698–1716
Carlyon RP (1996) Encoding the fundamental frequency of a complex tone in the presence of a spectrally overlapping masker. J Acoust Soc Am 99:517–524. doi:10.1121/1.414510
Carlyon RP, Deeks JM, McKay CM (2010) The upper limit of temporal pitch for cochlear-implant listeners: stimulus duration, conditioner pulses, and the number of electrodes stimulated. J Acoust Soc Am 127:1469–1478. doi:10.1121/1.3291981
Cosentino S, Carlyon RP, Deeks JM et al (2016) Rate discrimination, gap detection and ranking of temporal pitch in cochlear implant users. J Assoc Res Otolaryngol 17:371–382. doi:10.1007/s10162-016-0569-5
Crew JD, Galvin JJ, Fu Q-J (2012) Channel interaction limits melodic pitch perception in simulated cochlear implants. J Acoust Soc Am 132:EL429–EL435. doi:10.1121/1.4758770
Dorman M, Loizou PC, Fitzke J, Tu Z (1998) The recognition of sentences in noise by normal-hearing listeners using simulations of cochlear-implant signal processors with 6–20 channels. J Acoust Soc Am 104:3583–3585. doi:10.1121/1.423940
Dudley H (1939) Remaking Speech. J Acoust Soc Am 11:169–177. doi:10.1121/1.1916020
Ewert SD (2013) AFC—a modular framework for running psychoacoustic experiments and computational perception models. Proc Conf Acoust AIA-DAGA 2013:1326–1329
Fastl H, Zwicker E (2007) Psychoacoustics—facts and models. Springer, Berlin
Faulkner A, Rosen S, Wilkinson L (2001) Effects of the number of channels and speech-to-noise ratio on rate of connected discourse tracking through a simulated cochlear implant speech processor. Ear Hear 22:431–438
Fielden CA, Kluk K, Boyle PJ, McKay CM (2015) The perception of complex pitch in cochlear implants: a comparison of monopolar and tripolar stimulation. J Acoust Soc Am 138:2524–2536. doi:10.1121/1.4931910
Friesen LM, Shannon RV, Baskent D, Wang X (2001) Speech recognition in noise as a function of the number of spectral channels: comparison of acoustic hearing and cochlear implants. J Acoust Soc Am 110:1150–1163. doi:10.1121/1.1381538
Fu Q-J, Nogaki G (2005) Noise susceptibility of cochlear implant users: the role of spectral resolution and smearing. J Assoc Res Otolaryngol 6:19–27. doi:10.1007/s10162-004-5024-3
Galvin JJ, Fu Q-J, Nogaki G (2007) Melodic contour identification by cochlear implant listeners. Ear Hear 28:302–319. doi:10.1097/01.aud.0000261689.35445.20
Glasberg BR, Moore BCJ (1986) Auditory filter shapes in subjects with unilateral and bilateral cochlear impairments. J Acoust Soc Am 79:1020–1033. doi:10.1121/1.393374
Glasberg BR, Moore BCJ (1990) Derivation of auditory filter shapes from notched-noise data. Hear Res 47:103–138. doi:10.1016/0378-5955(90)90170-T
Grantham DW, Wightman FL (1978) Detectability of varying interaural temporal differences. J Acoust Soc Am 63:511–523. doi:10.1121/1.381751
Greenwood DD (1990) A cochlear frequency-position function for several species—29 years later. J Acoust Soc Am 87:2592–2605. doi:10.1121/1.399052
Highnam C, Morris V (1987) Linguistic stress judgments of language learning disabled students. J Commun Disord 20:93–103. doi:10.1016/0021-9924(87)90001-3
Houtsma AJM, Smurzynski J (1990) Pitch identification and discrimination for complex tones with many harmonics. J Acoust Soc Am 87:304–310. doi:10.1121/1.399297
Kohlrausch A, Houtsma AJM (1992) Pitch related to spectral edges of broadband signals [and discussion]. Philos Trans R Soc Lond Ser B Biol Sci 336:375–382. doi:10.1098/rstb.1992.0071
Kong Y-Y, Cruz R, Jones AJ, Zeng F-G (2004) Music perception with temporal cues in acoustic and electric hearing. Ear Hear 25(2):173–185
Kong Y-Y, Deeks JM, Axon PR, Carlyon RP (2009) Limits of temporal pitch in cochlear implants. J Acoust Soc Am 125:1649–1657. doi:10.1121/1.3068457
Lieberman P, Michaels SB (1962) Some aspects of fundamental frequency and envelope amplitude as related to the emotional content of speech. J Acoust Soc Am 34:922–927. doi:10.1121/1.1918222
Loeb GE, White MW, Merzenich MM (1983) Spatial cross-correlation. Biol Cybern 47:149–163. doi:10.1007/BF00337005
Loizou PC, Dorman M, Tu Z (1999) On the number of channels needed to understand speech. J Acoust Soc Am 106:2097–2103. doi:10.1121/1.427954
Luo H, Wang Y, Poeppel D, Simon JZ (2007) Concurrent encoding of frequency and amplitude modulation in human auditory cortex: encoding transition. J Neurophysiol 98:3473–3485. doi:10.1152/jn.00342.2007
Macherey O, Deeks JM, Carlyon RP (2011) Extending the limits of place and temporal pitch perception in cochlear implant users. J Assoc Res Otolaryngol 12:233–251. doi:10.1007/s10162-010-0248-x
Macmillan NA, Creelman CD (2004) Detection theory: a user’s guide. Psychology Press
McDermott HJ (2004) Music perception with cochlear implants: a review. Trends Amplif 8:49–82. doi:10.1177/108471380400800203
McDermott JH, Lehr AJ, Oxenham AJ (2008) Is relative pitch specific to pitch? Psychol Sci 19:1263–1271. doi:10.1111/j.1467-9280.2008.02235.x
Micheyl C, Keebler MV , Oxenham AJ (2010) Pitch perception for mixtures of spectrally overlapping harmonic complex tones. J Acoust Soc Am 128(1):257-269. doi:10.1121/1.3372751
Middlebrooks JC, Snyder RL (2007) Auditory prosthesis with a penetrating nerve array. J Assoc Res Otolaryngol 8:258–279. doi:10.1007/s10162-007-0070-2
Middlebrooks JC, Snyder RL (2010) Selective electrical stimulation of the auditory nerve activates a pathway specialized for high temporal acuity. J Neurosci 30:1937–1946. doi:10.1523/JNEUROSCI.4949-09.2010
Moore BCJ (1973) Frequency difference limens for short-duration tones. J Acoust Soc Am 54:610–619. doi:10.1121/1.1913640
Moore BCJ, Ernst SMA (2012) Frequency difference limens at high frequencies: evidence for a transition from a temporal to a place code. J Acoust Soc Am 132:1542–1547. doi:10.1121/1.4739444
Moore BCJ, Gockel HE (2011) Resolvability of components in complex tones and implications for theories of pitch perception. Hear Res 276:88–97. doi:10.1016/j.heares.2011.01.003
Moore BCJ, Huss M, Vickers DA et al (2000) A test for the diagnosis of dead regions in the cochlea. Br J Audiol 34:205–224. doi:10.3109/03005364000000131
Oxenham AJ (2012) Pitch perception. J Neurosci 32:13335–13338. doi:10.1523/JNEUROSCI.3815-12.2012
Oxenham AJ, Kreft HA (2014) Speech perception in tones and noise via cochlear implants reveals influence of spectral resolution on temporal processing. Trends Hear 18:2331216514553783. doi:10.1177/2331216514553783
Oxenham AJ, Micheyl C, Keebler MV et al (2011) Pitch perception beyond the traditional existence region of pitch. Proc Natl Acad Sci U S A 108:7629–7634. doi:10.1073/pnas.1015291108
Palmer AR, Russell IJ (1986) Phase-locking in the cochlear nerve of the guinea-pig and its relation to the receptor potential of inner hair-cells. Hear Res 24:1–15. doi:10.1016/0378-5955(86)90002-X
Patel AD, Peretz I, Tramo M, Labreque R (1998) Processing prosodic and musical patterns: a neuropsychological investigation. Brain Lang 61:123–144. doi:10.1006/brln.1997.1862
Pinyon JL, Tadros SF, Froud KE et al (2014) Close-field electroporation gene delivery using the cochlear implant electrode array enhances the bionic ear. Sci Transl Med 6:233ra54–233ra54. doi:10.1126/scitranslmed.3008177
Plack CJ, Oxenham AJ, Fay RR, Popper AN (2006) Pitch: neural coding and perception. Springer Science & Business Media
Plomp R (1967) Pitch of complex tones. J Acoust Soc Am 41:1526–1533. doi:10.1121/1.1910515
Pressnitzer D, Patterson RD, Krumbholz K (2001) The lower limit of melodic pitch. J Acoust Soc Am 109:2074–2084. doi:10.1121/1.1359797
Qin MK, Oxenham AJ (2005) Effects of envelope-vocoder processing on F0 discrimination and concurrent-vowel identification. Ear Hear 26:451–460
Ritsma RJ (1967) Frequencies dominant in the perception of the pitch of complex sounds. J Acoust Soc Am 42:191–198. doi:10.1121/1.1910550
Rosen S, Faulkner A, Wilkinson L (1999) Adaptation by normal listeners to upward spectral shifts of speech: implications for cochlear implants. J Acoust Soc Am 106:3629–3636. doi:10.1121/1.428215
Rosen S, Souza P, Ekelund C, Majeed AA (2013) Listening to speech in a background of other talkers: effects of talker number and noise vocoding. J Acoust Soc Am 133:2431–2443. doi:10.1121/1.4794379
Samson S, Zatorre RJ, Ramsay JO (1997) Multidimensional scaling of synthetic musical timbre: perception of spectral and temporal characteristics. Can J Exp Psychol Can Psychol Exp 51:307–315. doi:10.1037/1196-1961.51.4.307
Sayles M, Winter IM (2008) Reverberation challenges the temporal representation of the pitch of complex sounds. Neuron 58:789–801. doi:10.1016/j.neuron.2008.03.029
Shackleton TM, Carlyon RP (1994) The role of resolved and unresolved harmonics in pitch perception and frequency modulation discrimination. J Acoust Soc Am 95:3529–3540. doi:10.1121/1.409970
Shamma S, Klein D (2000) The case of the missing pitch templates: how harmonic templates emerge in the early auditory system. J Acoust Soc Am 107:2631–2644. doi:10.1121/1.428649
Shannon RV, Zeng F-G, Kamath V et al (1995) Speech recognition with primarily temporal cues. Science 270:303
Shofner WP (2014) Perception of degraded speech sounds differs in chinchilla and human listeners. J Acoust Soc Am 135:2065–2077. doi:10.1121/1.4867362
Shofner WP, Campbell J (2012) Pitch strength of noise-vocoded harmonic tone complexes in normal-hearing listeners. J Acoust Soc Am 132:EL398–EL404. doi:10.1121/1.4757697
Shofner WP, Chaney M (2013) Processing pitch in a nonhuman mammal (Chinchilla laniger). J Comp Psychol 127:142–153. doi:10.1037/a0029734
Smith ZM, Delgutte B, Oxenham AJ (2002) Chimaeric sounds reveal dichotomies in auditory perception. Nature 416:87–90. doi:10.1038/416087a
Stickney GS, Zeng F-G, Litovsky R, Assmann PF (2004) Cochlear implant speech recognition with speech maskers. J Acoust Soc Am 116:1081–1091. doi:10.1121/1.1772399
Titze IR (1989) Physiologic and acoustic differences between male and female voices. J Acoust Soc Am 85:1699–1707. doi:10.1121/1.397959
Wise AK, Tan J, Wang Y et al (2016) Improved auditory nerve survival with nanoengineered supraparticles for neurotrophin delivery into the deafened cochlea. PLoS One 11:e0164867. doi:10.1371/journal.pone.0164867
Zeng F-G, Tang Q, Lu T (2014) Abnormal pitch perception produced by cochlear implant stimulation. PLoS One 9:e88662. doi:10.1371/journal.pone.0088662
Zwicker E (1970) Masking and psychological excitation as consequences of the ear’s frequency analysis. Freq Anal Period Detect Hear:376–394
Acknowledgements
This work was supported by the NIH grant R01DC005216 (A.J.O).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
The University of Minnesota Institutional Review Board gave approval for this experiment.
Conflict of Interest
The authors declare that they have no conflict of interest.
Rights and permissions
About this article
Cite this article
Mehta, A.H., Oxenham, A.J. Vocoder Simulations Explain Complex Pitch Perception Limitations Experienced by Cochlear Implant Users. JARO 18, 789–802 (2017). https://doi.org/10.1007/s10162-017-0632-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10162-017-0632-x