What breaks a melody: Perceiving F0 and intensity sequences with a cochlear implant

Abstract

Pitch perception has been extensively studied using discrimination tasks on pairs of single sounds. When comparing pitch discrimination performance for normal-hearing (NH) and cochlear implant (CI) listeners, it usually appears that CI users have relatively poor pitch discrimination. Tasks involving pitch sequences, such as melody perception or auditory scene analysis, are also usually difficult for CI users. However, it is unclear whether the issue with pitch sequences is a consequence of sound discriminability, or if an impairment exists for sequence processing per se. Here, we compared sequence processing abilities across stimulus dimensions (fundamental frequency and intensity) and listener groups (NH, CI, and NH listeners presented with noise-vocoded sequences). The sequence elements were firstly matched in discriminability, for each listener and dimension. Participants were then presented with pairs of sequences, constituted by up to four elements varying on a single dimension, and they performed a same/different task. In agreement with a previous study (Cousineau et al., 2009) fundamental frequency sequences were processed more accurately than intensity sequences by NH listeners. However, this was not the case for CI listeners, nor for NH listeners presented with noise-vocoded sequences. Intensity sequence processing was, nonetheless, equally accurate in the three groups. These results show that the reduced pitch cues received by CI listeners do not only elevate thresholds, as previously documented, but also affect pitch sequence processing above threshold. We suggest that efficient sequence processing for pitch requires the resolution of individual harmonics in the auditory periphery, which is not achieved with the current generation of implants.

Introduction

The cochlear implant, a surgically-implanted device that bypasses cochlear processing to directly stimulate the auditory nerve, has been used to restore auditory function in many individuals with profound deafness. The original aim of the implant design was, understandably, to enable speech intelligibility. But whereas speech intelligibility in quiet can be achieved with a coarse representation of acoustic information (Shannon et al., 1995), other auditory abilities may require acoustic cues that are not currently available to cochlear implant (CI) users. In particular, pitch perception seems to be impaired when using a cochlear implant (for reviews, see McDermott, 2004, Moore and Carlyon, 2005, Drennan and Rubinstein, 2008). This is problematic, as accurate processing of pitch patterns is essential for speech perception (intonation, tonal languages), music perception (melodies), or auditory scene analysis (streaming, speech in noise). A better understanding of which aspects of pitch patterns’ perception are impaired in CI users is therefore of fundamental importance.

Pitch perception in CI users has often been assessed using pitch-ranking tasks with acoustically presented stimuli (Geurts and Wouters, 2001, Laneau et al., 2004, McDermott, 2004, Pressnitzer et al., 2005, Vandali et al., 2005, Sucher and McDermott, 2007, Looi et al., 2008a, Looi et al., 2008b). A similar procedure was used in all of these studies: two complex sounds (often sung vowels) that differed in fundamental frequency (F0) were presented. Listeners had to rank them on the pitch dimension by indicating which sound was higher in pitch, and threshold was estimated as the smallest F0 difference producing a consistent ranking. Results showed average thresholds much poorer than those observed for normal-hearing (NH) listeners (e.g. around 10% of F0 for the best performers in the CI groups, compared to less than 0.5% for NH, Geurts and Wouters, 2001, Pressnitzer et al., 2005). Moreover, a prominent feature of all results is a large inter-subject variability: for a 50% difference in F0 (7 semitones), performance of CI users may range from chance to near perfect (McDermott, 2004).

Another type of measure has focused on the processing of pitch sequences that extend over time, because of their immediate relevance to music perception (Cooper et al., 2008, Fujita and Ito, 1999, Galvin et al., 2007, Gfeller et al., 2002, Kong et al., 2004, Looi et al., 2008a, Looi et al., 2008b, Pressnitzer et al., 2005, Singh et al., 2009). In some of these studies, familiar melodies were to be recognized from a closed set (e.g. Fujita and Ito, 1999) whereas in others, simple melodic contours were to be discriminated (e.g. Galvin et al., 2007). Results generally indicate that melody perception is poor in CI users, with, again, a large variability between subjects. Galvin et al. (2007), for instance, obtained results ranging from chance to near-NH performance in a 5-note melodic contour identification task.

There are many potential sources for the variability reported across these studies, from different processing strategies to various nerve survival rates in individual CI listeners (Moore and Carlyon, 2005). It is unclear, moreover, if the variability observed in the pitch sequence tasks is simply a reflection of the diverse pitch-ranking abilities of individual CI users. Obviously, a large pitch-ranking threshold should induce poor pitch sequence representation. Looi et al. (2004) found that subjects’ ability to rank pitches was correlated with their ability to recognize melodies. However, it is also possible that, in addition, pitch sequence processing per se is impaired in CI users. For NH listeners, McFarland and Cacace (1992) suggested that sequences of pitch were more accurately processed than sequences of loudness, even though the discriminability between single elements of the sequences was approximately equated on each dimension. Cousineau et al. (2009), using a method that took into account the exact discriminability thresholds of individual listeners on each dimension, confirmed the advantage for pitch sequence processing for NH listeners. Pitch sequence discrimination performance was found to be superior to loudness sequence discrimination performance, presumably because of contour-encoding mechanisms available only for pitch (Demany and Ramos, 2005, Demany et al., 2009). In addition, it was found that the pitch sequence advantage was restricted to sounds made of resolved harmonics; pitch sequences made up of complex tones without any resolved harmonic were processed no more accurately than loudness sequences.

The latter finding leads to the prediction that CI users, being generally unable to resolve the individual harmonics of complex tones (Laneau et al., 2004), may suffer from a specific impairment in pitch sequence processing, independent of their pitch-ranking abilities. The following experiments were designed to test this hypothesis. We used the psychophysical method of Cousineau et al. (2009) to test the perception of sequences varying in either F0 or intensity in three groups of listeners: CI users, NH listeners, and NH listeners presented with noise-vocoded sequences (NH-voc). Importantly, the method aimed to uncouple sequence processing performance from any difference in terms of pitch discriminability that was expected between (and within) the different groups of listeners.