Abstract
The present study examined cross-modal selective attention using a task-switching paradigm. In a series of experiments, we presented lateralized visual and auditory stimuli simultaneously and asked participants to make a spatial decision according to either the visual or the auditory stimulus. We observed consistent cross-modal interference in the form of a spatial congruence effect. This effect was asymmetrical, with higher costs when responding to auditory than to visual stimuli. Furthermore, we found stimulus-modality-shift costs, indicating a persisting attentional bias towards the attended stimulus modality. We discuss our findings with respect to visual dominance, directed-attention accounts, and the modality-appropriateness hypothesis.
Similar content being viewed by others
References
Allport, D. A., Styles, E. A., & Hsieh, S. (1994). Shifting intentional set: Exploring the dynamic control of tasks. In C. Umiltà & M. Moscovitch (Eds.), Attention and performance XV: Conscious and nonconscious information processing (pp. 421–452). Cambridge, MA: MIT Press.
Aschersleben, G., & Bertelson, P. (2003). Temporal ventriloquism: crossmodal interaction on the time dimension. International Journal of Psychophysiology, 50, 157–163. doi:10.1016/S0167-8760(03)00131-4.
Bertelson, P., & Radeau, M. (1981). Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Perception & Psychophysics, 29, 578–584.
Bryck, R., & Mayr, U. (2008). Task selection cost asymmetry without task switching. Psychonomic Bulletin & Review, 15, 128–134. doi:10.3758/PBR.15.1.128.
Choe, C. S., Welch, R. B., Gilford, R. M., & Juola, J. F. (1975). The “ventriloquist effect”: visual dominance or response bias? Perception & Psychophysics, 18, 55–60.
Cohen, R., & Rist, F. (1992). The modality shift effect. Further explorations at the crossroads. In D. Friedman & G. E. Bruder (Eds.), Annals of the New York Academy of Sciences. Psychophysiology and experimental psychopathology: A tribute to Samuel Zutton (Vol. 658, pp. 163–181). New York: New York Academy of Sciences.
Colavita, F. B. (1974). Human sensory dominance. Perception & Psychophysics, 16, 409–412.
Egeth, H. E., & Sager, L. C. (1977). On the locus if visual dominance. Perception & Psychophysics, 22, 77–86.
Egner, T., Delano, M., & Hirsch, J. (2007). Separate conflict-specific cognitive control mechanisms in the human brain. NeuroImage, 35, 940–948. doi:10.1016/j.neuroimage.2006.11.061.
Gibson, J. J. (1933). Adaption, after-effect, and contrast in the perception of curved lines. Journal of Experimental Psychology, 16, 1–31. doi:10.1037/h0074626.
Harvey, N. (1980). Non-informative effects of stimuli functioning as cues. The Quarterly Journal of Experimental Psychology, 32, 413–425. doi:10.1080/14640748008401835.
Hazeltine, E., Ruthruff, E., & Remington, R. W. (2006). The role of input and output modality pairings in dual-task performance: evidence for content-dependent central interference. Cognitive Psychology, 52, 291–345. doi:10.1016/j.cogpsych.2005.11.001.
Hohnsbein, J., Falkenstein, M., Hoormann, J., & Blanke, L. (1991). Effects of crossmodal divided attention on late ERP components. I. Simple and choice reaction tasks. Electroencephalography and Clinical Neurophysiology, 78, 438–446. doi:10.1016/0013-4694(91)90061-8.
Hommel, B. (1997). Interactions between stimulus–stimulus congruence and stimulus–response compatibility. Psychological Research, 59, 248–260. doi:10.1007/BF00439302.
Kornblum, S. (1992). Dimensional overlap and dimensional relevance in stimulus–response and stimulus–stimulus compatibility. In G. E. Stelmach (Ed.), Advances in psychology: Tutorials in motor behavior II (pp. 743–777). Amsterdam: North-Holland.
Kornblum, S., Hasbroucq, T., & Osman, A. (1990). Dimensional overlap. Cognitive basis for stimulus–response compatibility—a model and taxonomy. Psychological Review, 97, 253–270. doi:10.1037/0033-295X.97.2.253.
Lu, C. H., & Proctor, R. W. (1995). The influence of irrelevant location information on performance: a review of the Simon and spatial Stroop effects. Psychonomic Bulletin & Review, 2, 174–207.
Meiran, N. (2008). The dual implication of dual affordance. Stimulus-task binding and attentional focus changing during task preparation. Experimental Psychology, 55, 252–260. doi:10.1027/1618-3169.55.4.251.
Miller, E. K., & Cohen, J. D. (2001). An integrative theory of prefrontal cortex function. Annual Review of Neuroscience, 24, 167–202. doi:10.1146/annurev.neuro.24.1.167.
Monsell, S. (2003). Task switching. Trends in Cognitive Sciences, 7, 134–140. doi:10.1016/S1364-6613(03)00028-7.
Notebaert, W., & Soetens, E. (2003). Irrelevant auditory attention shifts prime corresponding responses. Psychological Research, 67, 253–260. doi:10.1007/s00426-002-0126-1.
Pick, H. L., Warren, D. H., & Hay, J. C. (1969). Sensory conflict in judgments of spatial direction. Perception & Psychophysics, 6, 203–205.
Posner, M. I. (1982). Cumulative development and attentional theory. The American Psychologist, 37, 168–179. doi:10.1037/0003-066X.37.2.168.
Posner, M. I., Nissen, M. J., & Klein, R. M. (1976). Visual dominance: An information-processing account of its origins and significance. Psychological Review, 83, 157–171. doi:10.1037/0033-295X.83.2.157.
Proctor, R. W., & Pick, D. F. (1998). Lateralized warning tones produce typical irrelevant-location effects on choice reactions. Psychonomic Bulletin & Review, 5, 124–129.
Proctor, R. W., & Pick, D. F. (2003). Display-control arrangement correspondence and logical recoding in the Hedge and Marsh reversal of the Simon effect. Acta Psychologica, 112, 259–278. doi:10.1016/S0001-6918(02)00125-7.
Proctor, R. W., & Vu, K.-P. L. (2006). Stimulus–response compatibility principles: Data, theory, and application. Boca Raton, FL: CRC Press.
Quinlan, P. T., & Hill, N. I. (1999). Sequential effects in rudimentary auditory and visual tasks. Perception & Psychophysics, 61, 375–384.
Ragot, R., Cave, C., & Fano, M. (1988). Reciprocal effects of visual and auditory stimuli in a spatial compatibility situation. Bulletin of the Psychonomic Society, 26, 350–352.
Rogers, R. D., & Monsell, S. (1995). Costs of a predictable switch between simple cognitive tasks. Journal of Experimental Psychology. General, 124, 207–231. doi:10.1037/0096-3445.124.2.207.
Shams, L., Kamitani, Y., & Shimojo, S. (2000). What you see is what you hear. Nature, 408, 788. doi:10.1038/35048669.
Simon, J. R., & Craft, J. L. (1970). Effects of an irrelevant auditory stimulus on visual choice-reaction time. Journal of Experimental Psychology, 86, 272–274. doi:10.1037/h0029961.
Sinnett, S., Spence, C., & Soto-Faraco, S. (2007). Visual dominance and attention: the Colavita effect revisited. Perception & Psychophysics, 69, 673–686.
Spence, C., & Driver, J. (1997). On measuring selective attention to an expected sensory modality. Perception & Psychophysics, 59, 389–403.
Stelzel, C., Schumacher, E. H., Schubert, T., & D’Esposito, M. (2006). The neural effect of stimulus–response modality congruence on dual-task performance: an fMRI study. Psychological Research, 70, 403–561. doi:10.1007/s00426-005-0013-7.
Sutton, S., & Zubin, J. (1965). Effect of sequence on reaction time in schizophrenia. In A. T. Welford & J. E. Birren (Eds.), Behavior, aging, and the nervous system (pp. 562–579). Springfield, Ill.: Charles C. Thomas.
Tagliabue, M., Zorzi, M., & Umiltá, C. (2002). Cross-modal re-mapping influences the Simon effect. Memory & Cognition, 30, 18–23.
Van der Lubbe, R. H. J., & Van der Helden, J. (2006). Failure of the extended contingent attentional capture account in multimodal settings. Advances in Cognitive Psychology, 2, 255–267. doi:10.2478/v10053-008-0060-x.
Wallace, R. J. (1971). S–R compatibility and the idea of a response-code. Journal of Experimental Psychology, 88, 354–360. doi:10.1037/h0030892.
Ward, L. M. (1994). Supramodal and modality-specific mechanisms for stimulus-driven shifts of auditory and visual attention. Canadian Journal of Experimental Psychology, 48, 242–259. doi:10.1037/1196-1961.48.2.242.
Ward, L. M., McDonald, J. J., & Lin, D. (2000). On asymmetries in cross-modal spatial attention orienting. Perception & Psychophysics, 62, 1258–1264.
Welch, R. B., & Warren, D. H. (1980). Immediate perceptual response to intersensory discrepancy. Psychological Bulletin, 88, 638–667. doi:10.1037/0033-2909.88.3.638.
Welch, R. B., & Warren, D. H. (1986). Intersensory interactions. In K. R. Boff, L. Kaufman, & J. P. Thomas (Eds.), Handbook of perception and human performance: Vol. 1. Sensory processes and perception (pp. 25.1–25.36). New York: Wiley.
Wendt, M., & Kiesel, A. (2008). The impact of stimulus–specific practice and task instructions on response congruency effects between tasks. Psychological Research, 72, 425–432. doi:10.1007/s00426-007-0117-3.
Wühr, P., Biebl, R., Umiltà, C., & Müsseler, J. (2009). Perceptual and attentional factors in encoding irrelevant spatial information. Psychological Research, 73, 350–363.
Yeung, N., & Monsell, S. (2003). Switching between tasks of unequal familiarity: the role of stimulus-attribute and response-set selection. Journal of Experimental Psychology. Human Perception and Performance, 29, 455–469. doi:10.1037/0096-1523.29.2.455.
Acknowledgments
Sarah Lukas, Andrea M. Philipp, and Iring Koch, Institute of Psychology, RWTH Aachen University, Aachen, Germany. This research was funded by the Deutsche Forschungsgemeinschaft (DFG) Grant KO 2045/04-03 to Iring Koch and Andrea M. Philipp in the context of DFG SPP 1107 (Executive Functions). The authors would like to thank Bernhard Hommel, Peter Wühr, and Thomas Kleinsorge for their comments on an earlier version of the paper. We are also grateful to Marion Marksteiner, Dunja Kosanke, and Sandra Beecks for conducting the experiments.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lukas, S., Philipp, A.M. & Koch, I. Switching attention between modalities: further evidence for visual dominance. Psychological Research 74, 255–267 (2010). https://doi.org/10.1007/s00426-009-0246-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00426-009-0246-y