Abstract
The account of signs, signification and meaning set out by the philosopher Charles Sanders Peirce around the beginning of the twentieth century is a foundation stone of modern semiotics. In Peirce’s conception, semiotics concerned the process of signification at its most general and was intrinsically multimodal. It is then logical that contemporary multimodal research should engage with the Peircean view of semiotics. This in turn makes it particularly important that better understandings of the potential interconnections between Peircean semiotics and multimodality be achieved. Until now, however, such interconnections have been limited not only with respect to what has been drawn from Peirce’s account but also with respect to the areas of multimodality concerned. To deepen and broaden communication among approaches to multimodality and between those approaches and Peircean semiotics, this article sets out a reading of Peirce’s ‘mature’ 1903 position on semiotics aimed specifically at improving the utility of the account for multimodal analysis. The article argues that a lack of methodological precision within semiotics has led to relatively impoverished and empirically unreliable analyses. A new synthesis between Peirce’s account and multimodality is proposed as a stronger metatheoretical framework both for addressing multimodality and for drawing more from Peirce.
Acknowledgements
Several people have provided feedback on previous versions of this article or material presented here, including George Rossolatos, David Machin, Søren Vigild Poulsen, participants in the “Pictures and Texts–Pictures as Texts: Iconicity and Indexicality in Graphic Communication” conference of the Topoi Excellence Cluster of the Humboldt-University Berlin and the 3rd Bremen International Conference on Multimodality “Multimodality – Towards a new discipline”, as well as in courses held on Peirce at Bremen University. Thanks also go to particularly helpful comments from two anonymous reviewers. Any errors and misrepresentations remain the responsibility of the author.
References
Asher, Nicholas, and Lascarides, Alex. (2003). Logics of Conversation. Cambridge: Cambridge University Press.Search in Google Scholar
Atã, Pedro, and Quieroz, João. (2014). Icon and abduction: Situatedness in Peircean cognitive semiotics. In: Model-Based Reasoning in Science and Technology (Studies in Applied Philosophy, Epistemology and Rational Ethics 8), L. Magani (Ed.), 301–313. Springer.10.1007/978-3-642-37428-9_17Search in Google Scholar
Atkin, Albert. (2016). Peirce. Abingdon and New York: Routledge.Search in Google Scholar
Bal, Mieke, and Bryson, Norman. (1991). Semiotics and art history. Art Bulletin, 73(2):174–208.10.2307/3045790Search in Google Scholar
Barsalou, Lawrence W., Breazeal, Cynthia, and Smith, Linda B. (2007). Cognition as coordinated non-cognition. Cognitive Processes, 8:79–91.10.1007/s10339-007-0163-1Search in Google Scholar
Barthes, Roland. (1964). Elements of Semiology. London: Jonathan Cape. Translated by Annette Lavers and Colin Smith.Search in Google Scholar
Barthes, Roland. (1977 [1964]). The rhetoric of the image. In: Image–Music–Text, Stephen Heath (Ed.), 32–51. London: Fontana.Search in Google Scholar
Bateman, John A. (2011). The decomposability of semiotic modes. In: MultiModal Studies: Multiple Approaches and Domains Routledge Studies in Multimodality, Kay L. O’Halloran and Bradley A. Smith (Eds.), 17–38. London: Routledge.Search in Google Scholar
Bateman, John A. (2014). The constitutive role of semiotic modes for the theory and practice of multimodal analysis. In: Language Arts in Asia 2: English and Chinese through Literature, Drama and Popular Culture, Christina DeCoursey (Ed.), 8–33. Newcastle, UK: Cambridge Scholars Publishing.Search in Google Scholar
Bateman, John A. (2016). Methodological and theoretical issues for the empirical investigation of multimodality. In: Sprache im Multimodalen Kontext / Language and Multimodality (Handbooks of Linguistics and Communication Science (HSK) 7), Nina-Maria Klug and Hartmut Stöckl (Eds.), 36–74. Berlin: de Gruyter Mouton.Search in Google Scholar
Bateman, John A., and Wildfeuer, Janina. (2014). A multimodal discourse theory of visual narrative. Journal of Pragmatics, 74:180–218. doi:10.1016/j.pragma.2014.10.001.Search in Google Scholar
Bateman, John A., Wildfeuer, Janina, and Hiippala, Tuomo. (2017). Multimodality – Foundations, Research and Analysis. A Problem-Oriented Introduction. Berlin: Mouton de Gruyter.Search in Google Scholar
Bergman, Mats. (2009). Peirce’s Philosophy of Communication. London and New York: Contninuum.Search in Google Scholar
Bezemer, Jeff, and Kress, Gunther. (2016). Multimodality, Learning and Communication: A Social Semiotic Frame. New York and London: Routledge.Search in Google Scholar
Chandler, Daniel. (2007). Semiotics: The Basics. 2nd edn. London and New York: Routledge.10.4324/9780203014936Search in Google Scholar
Cienki, Alan. (1998a). Metaphoric gestures and some of their relations to verbal metaphorical expressions. In: Discourse and Cognition: Bridging the Gap, Jean-Pierre Koenig (Ed.), 189–204. Stanford: Center for the Study of Language and Information.Search in Google Scholar
Cienki, Alan. (1998b). STRAIGHT: An image schema and its metaphorical extensions. Cognitive Linguistics, 9:107–149.10.1515/cogl.1998.9.2.107Search in Google Scholar
Clark, Andy, and Chalmers, David. (1998). The extended mind. Analysis, 58(1):7–19.10.1093/analys/58.1.7Search in Google Scholar
Colapietro, Vincent. (2011). Image, diagram and metaphor. Unmined resource and unresolved questions. In: Semblance and Signification, Christina Ljungberg, Olga Fischer, and Pascal Michelucci (Eds.), 157–171. Amsterdam: John Benjamins.10.1075/ill.10.09colSearch in Google Scholar
Danaher, David. (1998). Peirce’s semiotic and cognitive metaphor theory. Semiotica, 119(1/2):171–207.Search in Google Scholar
De Waal, Cornelus. (2013). Peirce: A Guide for the Perplexed. London: Bloomsbury.Search in Google Scholar
Deledalle, Gérard. (1979). Théorie et pratique du signe: Introduction à la sémiotique de Charles S. Peirce. Paris: Payot.Search in Google Scholar
Deledalle, Gérard. (2000). Charles S. Peirce’s Philosophy of Signs: Essays in Comparative Semiotics chap. Semeiotic and Semiology: Peirce and Saussure, 100–114. Bloomington and Indianopolis, IN: Indiana University Press.Search in Google Scholar
Deppermann, Arnulf. (2013). Multimodal interaction from a conversation analytic perspective. Journal of Pragmatics, 46(1):1–7.10.1016/j.pragma.2012.11.014Search in Google Scholar
Eco, Umberto. (1976). A Theory of Semiotics. Bloomington: Indiana University Press.10.1007/978-1-349-15849-2Search in Google Scholar
Elleström, Lars. (2013). Spatiotemporal aspects of iconicity. In: Iconic investigations, Lars Elleström, Olga Fischer, and Christina Ljungberg (Eds.), Amsterdam: John Benjamins.10.1075/ill.12.09ellSearch in Google Scholar
Freadman, Anne. (1986). Structuralist uses of Peirce: Jakobson, Metz et al. In: Semiotics – language – ideology (Sydney Studies in Society and Culture 3), T. Threadgold, E. A. Grosz, Gunther Kress, and Michael A. K. Halliday (Eds.), 93–124. Sydney: Sydney Association for Studies in Society and Culture.Search in Google Scholar
Fricke, Ellen. (2007). Origo, Geste und Raum: Lokaldeixis im Deutschen. Berlin: de Gruyter.10.1515/9783110897746Search in Google Scholar
Fricke, Ellen. (2012). Grammatik multimodal: Wie Wörter und Gesten zusammenwirken. Berlin and New York: Mouton de Gruyter.10.1515/9783110218893Search in Google Scholar
Fricke, Ellen. (2013). Between reference and meaning: Object-related and interpretant-related gestures in face-to-face interaction. In: Body–Language–Communication. An International Handbook on Multmodality in Human Interaction (Handbücher zur Sprach- und Kommunikationswissenschaft / Handbooks of Linguistics and Communication Science (HSK) 38/1), Cornelia Müller, Alan Cienki, Ellen Fricke, Silva Ladewig, David McNeill, and Sedinha Tessendorf (Eds.), 1788–1802. Berlin and New York: Mouton de Gruyter.10.1515/9783110302028.1788Search in Google Scholar
Halliday, Michael A. K. (1978). Language as Social Semiotic. London: Edward Arnold.Search in Google Scholar
Hjelmslev, Louis. (1961 [1943]). Prolegomena to a Theory of Language. Madison, Wisconsin: University of Wisconsin Press. Originally published 1943; Translated by F.J. Whitfield.Search in Google Scholar
Hobbs, Jerry R. (1990). Literature and Cognition. Stanford, California: CSLI. CSLI Lecture Notes.Search in Google Scholar
Hoven, Paul J. van den. (2010). The intriguing iconic sign. Interstudia (Review of Interdisciplinary Centre for Studies of Contemporary Discursive Forms) 6. 24–37. Issue on: Cultural Spaces and Identities in (inter)action.Search in Google Scholar
Iversen, Margaret. (1986). Saussure v. Peirce: Models for a semiotics of visual art. In: The New Art History, A.L. Rees and Frances Borzello (Eds.), 82–94. London: Camden Press.Search in Google Scholar
Jakobson, Roman. (1965). Quest for the essence of language. Diogenes, 13:21–37.10.1177/039219216501305103Search in Google Scholar
Jakobson, Roman. (1990 [1956]). The two aspects of language and two types of aphasic disturbances. In: On Language, Linda R. Waugh and Monique Monville-Burston (Eds.), 115–133. Cambridge, MA and London: Harvard University Press.Search in Google Scholar
Jappy, Tony. (2013). Introduction to Peircean Visual Semiotics Bloomsbury Advances in Semiotics. London and New York: Bloomsbury.Search in Google Scholar
Kappelhoff, Hermann, and Müller, Cornelia. (2011). Embodied meaning construction. Multimodal metaphor and expressive movement in speech, gesture, and in feature film. Metaphor and the Social World, 1(2):121–153.10.1075/msw.1.2.02kapSearch in Google Scholar
Kendon, Alan. (2014). Semiotic diversity in utterance production and the concept of ‘language’. Philosophical Transactions of the Royal Society, B369(1651):20130293.10.1098/rstb.2013.0293Search in Google Scholar
Kress, Gunther. (2003). Literacy in the New Media Age. London: Routledge.Search in Google Scholar
Kress, Gunther. (2010). Multimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge.Search in Google Scholar
Kress, Gunther, and van Leeuwen, Theo. (2001). Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold.Search in Google Scholar
Kress, Gunther, and van Leeuwen, Theo. (2006). Reading Images: The Grammar of Visual Design. 2nd Edition. London and New York: Routledge.10.4324/9780203619728Search in Google Scholar
Lakoff, George, and Johnson, Mark. (1980). Metaphors We Live by. Chicago, Il: Chicago University Press.Search in Google Scholar
Lakoff, George, and Johnson, Mark. (1999). Philosophy in the Flesh: The Embodied Mind and Its Challenge to Western Thought. New York: Basic Books.Search in Google Scholar
Lefebvre, Martin. (2007). The Art of Pointing. On Peirce, Indexicality, and Photographic Images. In: Photography theory, James Elkins (Ed.), 220–244. London: Routledge. The art seminar, Volume II.Search in Google Scholar
Magnani, Lorenzo. (2001). Abduction, Reason, and Science. Processes of Discovery and Explanation. New York: Kluwer Academic/Plenum Publishers.10.1007/978-1-4419-8562-0Search in Google Scholar
Manghani, Sunil. (2013). Image Studies: Theory and Practice. London: Routledge.Search in Google Scholar
Mitchell, W.J.T. (1994). Picture Theory: Essays on Verbal and Visual Representation. Chicago: University of Chicago Press.Search in Google Scholar
Mitchell, W.J.T. (2005). There are no visual media. Journal of Visual Culture, 4(2):257–266. doi:10.1177/1470412905054673.Search in Google Scholar
Mittelberg, Irene. (2006). Metaphor and Metonymy in Language and Gesture: Discourse Evidence for Multimodal Models of Grammar. Ann Arbor, MI: UMI. Cornell University Dissertation.Search in Google Scholar
Mittelberg, Irene. (2008). Peircean semiotics meets conceptual metaphor. iconic modes in gestural representations of grammar. In: Metaphor and Gesture, Alan Cienki and Cornelia Müller (Eds.), 115–154. Amsterdam and Philadelphia: John Benjamins Publishing Company.Search in Google Scholar
Mittelberg, Irene, and Waugh, Linda R. (2009). Metonymy first, metaphor second: A cognitive-semiotic approach to multimodal figures of thought in co-speech gesture. In: Multimodal Metaphor, Charles J. Forceville, and Eduardo Urios-Aparisi (Eds.), 329–356. Berlin/New York: Mouton de Gruyter.Search in Google Scholar
Mittelberg, Irene, and Waugh, Linda R. (2014). Gestures and metonymy. In: Body – Language – Communication. An International Handbook on Multimodality in Human Interaction, Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeil, and Jana Bressem (Eds.), vol. 2, 1747–1766. Berlin/Boston: Mouton.Search in Google Scholar
Müller, C., and Cienki, Alan. (2009). Words, gestures, and beyond: Forms of multimodal metaphor in the use of spoken language. In: Multimodal Metaphor, Charles J. Forceville and Eduardo Urios-Aparisi (Eds.), 297–328. Berlin/New York: Mouton de Gruyter.Search in Google Scholar
Müller, Cornelia. (1998). Redebegleitende Gesten: Kulturgeschichte – Theorie – Sprachvergleich. Berlin: Berliner Wissenschafts-Verlag.Search in Google Scholar
Nöth, Winfried. (2010). The criterion of habit in Peirce’s definitions of the symbol. Transactions of the Charles S. Peirce Society, 46(1):82–93.10.2979/tra.2010.46.1.82Search in Google Scholar
Nöth, Winfried. (2011). Visual semiotics: Key features and an application to picture ads. In: The SAGE Handbook of Visual Research Methods, Eric Margolis and Luc Pauwels (Eds.), 298–316. London.10.4135/9781446268278.n16Search in Google Scholar
Nöth, Winfried. (2014). The growth of signs. Sign Systems Studies, 42(2/3):172–192.10.12697/SSS.2014.42.2-3.02Search in Google Scholar
Nöth, Winfried, and Jungk, Isabel. (2015). Peircean visual semiotics: Potentials to be explored. Semiotica, 207:657–673.10.1515/sem-2015-0058Search in Google Scholar
Parret, Herman. (1984). Peirce and Hjelmslev: The two semiotics. Language Sciences, 6(2):217–227.10.1016/S0388-0001(84)80017-0Search in Google Scholar
Peirce, Charles Sanders. (1931–1958). Collected Papers of Charles Sanders Peirce. Cambridge, MA: Harvard University Press. Vols. 1–6, 1931–1935, edited by Charles Hartshorne and Paul Weiss; Vols. 7–8, 1958, edited by Arthur W. Burks.Search in Google Scholar
Peirce, Charles Sanders. (1998 [1893–1913]). The Essential Peirce – Volume 2. Selected Philosophical Writings (1893–1913). Bloomington: Indiana University Press.Search in Google Scholar
Queiroz, João, and Merrell, Floyd. (2009). On Peirce’s pragmatic notion of semiosis a contribution for the design of meaning machines. Minds & Machines 19:129–143.10.1007/s11023-008-9129-zSearch in Google Scholar
Scott, David. (1997). Indexical/iconic tensions: The semiotics of the postage stamp. In: Semiotics of the Media. State of the Art, Projects, and Perspectives (Approaches to Semiotics 127), Winfried Nöth (Ed.), 191–201. Berlin/New York: Mouton de Gruyter.10.1515/9783110803617-014Search in Google Scholar
Short, Thomas L. (1982). Life among the legisigns. Transactions of the Charles S. Peirce Society, 18(4):285–310.Search in Google Scholar
Short, Thomas L. (1996). Interpreting Peirce’s Interpretant: A response to Lalor, Liszka, and Meyers. Transactions of the Charles S. Peirce Society, 32(4):488–541.Search in Google Scholar
Short, Thomas L. (2007). Peirce’s Theory of Signs. Cambridge: Cambridge University Press.10.1017/CBO9780511498350Search in Google Scholar
Skagested, Peter. (2004). Peirce’s semeiotic model of the mind. In: The Cambridge Companion to Peirce Cambridge Companions to Philosophy, Cheryl Misak (Ed.), 241–256. Cambridge: Cambridge University Press.10.1017/CCOL0521570069.010Search in Google Scholar
Stjernfelt, Frederik. (2000). Diagrams as centerpiece of a Peircean epistemology. Transactions of the Charles S. Peirce Society, 36(3): 357–384.Search in Google Scholar
Stjernfelt, Frederik. (2014). Natural Propositions: The Actuality of Peirce’s Doctrine of Dicisigns. Boston, MA: The Docent Press.Search in Google Scholar
Stöckl, Hartmut. (2004). In between modes: language and image in printed media. In: Perspectives on Multimodality, Eija Ventola, Cassily Charles, and Martin Kaltenbacher (Eds.), 9–30. Amsterdam: John Benjamins.10.1075/ddcs.6.03stoSearch in Google Scholar
Varela, Francisco J., Thompson, Evan, and Rosch, Eleanor. (1991). The Embodied Mind: Cognitive Science and Human Experience. Cambridge, MA: MIT Press.10.7551/mitpress/6730.001.0001Search in Google Scholar
Wharton, Tim. (2009). Pragmatics and Non-Verbal Communication. Cambridge: Cambridge University Press.10.1017/CBO9780511635649Search in Google Scholar
© 2018 Walter de Gruyter GmbH, Berlin/Boston
