Abstract
The visual narratives of comics involve complex multimodal interactions between written language and the visual language of images, where one or the other may guide the meaning and/or narrative structure. We investigated this interaction in a corpus analysis across eight decades of American superhero comics (1940–2010s). No change across publication date was found for multimodal interactions that weighted meaning towards text or across both text and images, where narrative structures were present across images. However, we found an increase over time of narrative sequences with meaning weighted to the visuals, and an increase of sequences without text at all. These changes coincided with an overall reduction in the number of words per panel, a shift towards panel framing with single characters and close-ups rather than whole scenes, and an increase in shifts between temporal states between panels. These findings suggest that storytelling has shifted towards investing more information in the images, along with an increasing complexity and maturity of the visual narrative structures. This has shifted American comics from being textual stories with illustrations to being visual narratives that use text.
Acknowledgement
Gerardo Soto-Becerra is thanked for additional assistance in data gathering.
Appendix. Works analyzed
Our corpus analysis used the following books, listed chronologically by publication date.
Berold, B. and Eisner, W. (1940). The Flame. 3: 1–20. Fox Comics.
Kirby, J. and Wellman, M. W. (1941). Captain Marvel Adventures. 1: 1–31. Fawcett Comics.
Binder, J. (1942). Captain Midnight. 1: 1–34. Fawcett Comics.
Vagoda, B. and Weisbecker, C. (1943). Black Hood. 9: 1–23. MLJ Magazine
Nordling, K. (1949). Lady Luck. 86: 1–32. Quality Comics.
Quackenbush, B and Eisner, W. (1950). Doll Man. 30: 1–32.Comic Favorites, Inc.
Anderson, M and Siegel, J. (1951). Lars of Mars. 11: 1–16 & 28–31. Approved Comics.
Ferstadt, L, Fago, A., & Fox, V. (1955). Blue Beetle. 18: 1–25. Charlton Comics.
Cole, J. and Woolfolk, B. (1956). Plastic Man. 64: 1–31. Comic Magazines.
Plastino, A. and Bernstein, R. (1959). Action Comics. 252: 1–27. DC Comics.
Springer, F. and Kastle, H. (1962). Brain Boy. 2: 1–28. Dell Comics/Western Publishing.
Wood, W. and Lee, S. (1964). Daredevil. 5: 1–20. Marvel Comics.
Ditko, S. and Gill, J. (1965). Captain Atom. 78: 1–20. Charlton Comics.
Ditko, S. and Glanzman, D.C. (1967). The Blue Beetle. 3: 1–20. Charlton Comics.
Steranko, J. (1968). Nick Fury Agent of S.H.I. E.L.D. 1: 1–20. Marvel Comics.
Adams, N. and O’Neil, D. (1970). Green Lantern. 76: 1–23. DC Comics.
Novick, I. and Bates, C. (1973). The Flash. 211: 1–16. DC Comics.
Dillion, D. and Pasko, M. (1975). Justice League of America. 122: 1–18. DC Comics.
Tuska, G. and Mantlo, B. (1977). The Invincible Iron Man. 100: 1–19. Marvel Comics.
Rogers, M. and Englehart, S. (1978). Detective Comics. 475: 1–17. DC Comics.
Colan, G. and Mantlo, B. (1981). The Avengers. 210: 1–22. Marvel Comics.
Colan, G. and Thomas, R. (1982). Wonder Woman. 289: 1–26. DC Comics.
Byrne, J. (1984). Fantastic Four. 269: 1–22. DC Comics.
Bolland, B. and Moore, A. (1988). Batman: The Killing Joke. 1: 1–20. DC Comics.
Dwyer, K. and Gruenwald, M. (1989). Captain America. 358: 1–22. Marvel Comics.
McFarlane, T. and Michelinie, D. (1990). The Amazing Spider-Man. 328: 1–23. Marvel Comics.
Lyle, T. and Potts, C. (1993). Venom: Funeral Pyre. 2: 1–22. Marvel Comics.
Kubert, A. and David, P. (1997). The Incredible Hulk. 454: 1–25. Marvel Comics.
Yu, L. F. and Ellis, W. (1998). Wolverine. 121: 1–21. Marvel Comics.
Quesada, J. and Smith, K. (1999). Daredevil. 8: 1–22. Marvel Comics.
Dillon, S. and Ennis, G. (2001). The Punisher. 6: 1–22. Marvel Comics.
Sciver, E. V. and Johns, G. (2005). Green Lantern: Rebirth. 5: 1–21. DC Comics.
Garney, R. and Straczynski, J. M. (2007). Amazing Spider-Man. 539: 1–23. Marvel Comics.
Romita Jr., J. and Hudlin, R. (2008). Black Panther. 35: 1–22. Marvel Comics.
Medina, P. and Way, D. (2009). Deadpool. 11: 1–22. Marvel Comics.
Gleason, P. and Tomasi, P. (2010). Green Lantern Corps. 42: 1–24. DC Comics.
Finch, D. and Jenkins, P. (2011). Batman The Dark Knight. 2: 1–20. DC Comics.
Pacheco, C., Diaz, P., and Gillen, K. (2012). Uncanny X-men. 10: 1–20. Marvel Comics.
Guedes, R., Fawkes, R. & Lemire, J. (2013). Constantine. 1: 1–21. DC Comics.
Gerads, M. and Edmondson, N. (2014). The Punisher. 5: 1–20. Marvel Comics.
References
Abbott, M., and Forceville, C. (2011). Visual representation of emotion in manga: loss of control is loss of hands in Azumanga Daioh Volume 4. Language and Literature, 20:91–112.10.1177/0963947011402182Search in Google Scholar
Bateman, J. A. (2014). Text and Image: A Critical Introduction to the Visual/Verbal Divide. New York: Routledge.10.4324/9781315773971Search in Google Scholar
Bateman, J. A., and Wildfeuer, J. (2014a). Defining units of analysis for the systematic analysis of comics: a discourse-based approach. Studies in Comics, 5:373–403.10.1386/stic.5.2.373_1Search in Google Scholar
Bateman, J. A., and Wildfeuer, J. (2014b). A multimodal discourse theory of visual narrative. Journal of Pragmatics, 74:180–208.10.1016/j.pragma.2014.10.001Search in Google Scholar
Brienza, C. (2016). Manga in America: transnational book publishing and the domestication of Japanese comics. London: Bloomsbury Academic.Search in Google Scholar
Burgas, G. (2007). Compressed storytelling versus decompressed storytelling: pros and cons [Online]. Comic Book Resources. Available: http://www.cbr.com/compressed-storytelling-versus-decompressed-storytelling-pros-and-cons/ [Accessed 2/27/2017].Search in Google Scholar
Cicci, M. (2015). Turning the page: Fandoms, multimodality, the transformation of the “comic book” superhero. Doctoral Dissertation, Wayne State University.Search in Google Scholar
Cohn, N. (2011). A different kind of cultural frame: an analysis of panels in American comics and Japanese manga. Image [&] Narrative, 12:120–134.Search in Google Scholar
Cohn, N. (2013a). The Visual Language of Comics: Introduction to the Structure and Cognition of Sequential Images. London, UK: Bloomsbury.Search in Google Scholar
Cohn, N. (2013b). Visual narrative structure. Cognitive Science, 37:413–452.10.1111/cogs.12016Search in Google Scholar
Cohn, N. (2014). The architecture of visual narrative comprehension: the interaction of narrative structure and page layout in understanding comics. Frontiers in Psychology, 5:1–9.10.3389/fpsyg.2014.00680Search in Google Scholar
Cohn, N. (2015). Narrative conjunction’s junction function: the interface of narrative grammar and semantics in sequential images. Journal of Pragmatics, 88:105–132.10.1016/j.pragma.2015.09.001Search in Google Scholar
Cohn, N. (2016). A multimodal parallel architecture: A cognitive framework for multimodal interactions. Cognition, 146:304–323.10.1016/j.cognition.2015.10.007Search in Google Scholar
Cohn, N., and Bender, P. (2017). Drawing the line between constituent structure and coherence relations in visual narratives. Journal of Experimental Psychology: Learning, Memory, & Cognition, 43:289–301.10.1037/xlm0000290Search in Google Scholar
Cohn, N., and Ehly, S. (2016). The vocabulary of manga: visual morphology in dialects of Japanese visual language. Journal of Pragmatics, 92:17–29.10.1016/j.pragma.2015.11.008Search in Google Scholar
Cohn, N., Jackendoff, R., Holcomb, P. J., and Kuperberg, G. R. (2014). The grammar of visual narrative: neural evidence for constituent structure in sequential image comprehension. Neuropsychologia, 64:63–70.10.1016/j.neuropsychologia.2014.09.018Search in Google Scholar
Cohn, N., and Kutas, M. (2015). Getting a cue before getting a clue: event-related potentials to inference in visual narrative comprehension. Neuropsychologia, 77:267–278.10.1016/j.neuropsychologia.2015.08.026Search in Google Scholar
Cohn, N., and Kutas, M. (Under Review). What’s your neural function, visual narrative conjunction? Grammar, meaning, and fluency in sequential image processing.Search in Google Scholar
Cohn, N., Paczynski, M., Jackendoff, R., Holcomb, P. J., and Kuperberg, G. R. (2012a). (Pea)nuts and bolts of visual narrative: structure and meaning in sequential image comprehension. Cognitive Psychology, 65:1–38.10.1016/j.cogpsych.2012.01.003Search in Google Scholar
Cohn, N., Taylor-Weiner, A., and Grossman, S. (2012b). Framing attention in Japanese and American comics: cross-cultural differences in attentional structure. Frontiers in Psychology – Cultural Psychology, 3:1–12.10.3389/fpsyg.2012.00349Search in Google Scholar
Coogan, P. (2006). Superhero: The Secret Origin of a Genre. Austin, TX: MonkeyBrain.Search in Google Scholar
Cutting, J. E. (2015). The framing of characters in popular movies. Art & Perception, 3:191–212.10.1163/22134913-00002031Search in Google Scholar
Cutting, J. E., Delong, J. E., and Nothelfer, C. E. (2010). Attention and the evolution of hollywood film. Psychological Science, 21:432–439.10.1177/0956797610361679Search in Google Scholar
Duncan, R., and Smith, M. J. (2009). The Power of Comics. New York: Continuum Books.Search in Google Scholar
Duncan, R., Smith, M. J., and Levitz, P. (2015). The Power of Comics. New York: Continuum Books.Search in Google Scholar
Forceville, C. (2011). Pictorial runes in Tintin and the Picaros. Journal of Pragmatics, 43:875–890.10.1016/j.pragma.2010.07.014Search in Google Scholar
Forceville, C. (2016). Conceptual metaphor theory, blending theory, and other cognitivist perspectives on comics . In: The Visual Narrative Reader, N. Cohn (Ed.). Bloomsbury: London.Search in Google Scholar
Forceville, C., and Urios-Aparisi, E. (2009). Multimodal Metaphor. New York: Mouton De Gruyter.10.1515/9783110215366Search in Google Scholar
Forceville, C., Veale, T., and Feyaerts, K. (2010). Balloonics: The visuals of balloons in comics. In: The Rise and Reason of Comics and Graphic Literature: Critical Essays on the Form, J. Goggin and D. Hassler-Forest (Eds.). Jefferson: McFarland & Company, Inc.Search in Google Scholar
Foulsham, T., Wybrow, D., and Cohn, N. (2016). Reading without words: eye movements in the comprehension of comic strips. Applied Cognitive Psychology, 30:566–579.10.1002/acp.3229Search in Google Scholar
Gernsbacher, M. A., Varner, K. R., and Faust, M. (1990). Investigating differences in general comprehension skill. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16:430–445.10.1037/0278-7393.16.3.430Search in Google Scholar
Goldberg, W. (2010). The manga phenomenon in America. In: Manga: An Anthology of Global and Cultural Perspectives, T. Johnson-Woods (Ed.). New York: Continuum Books.Search in Google Scholar
Goldin-Meadow, S. (2006). Talking and thinking with our hands. Current Directions in Psychological Science, 15:34–39.10.1111/j.0963-7214.2006.00402.xSearch in Google Scholar
Goldin-Meadow, S., Mcneill, D., and Singleton, J. (1996). Silence is liberating: removing the handcuffs on grammatical expression in the manual modality. Psychological Review, 103:34–55.10.1037/0033-295X.103.1.34Search in Google Scholar
Green, J. (2014). Drawn from the Ground: Sound, Sign and Inscription in Central Australian Sand Stories. Cambridge, UK: Cambridge University Press.10.1017/CBO9781139237109Search in Google Scholar
Guynes, S. A. (2014). Four-color sound: a peircean semiotics of comic book onomatopoeia. The Public Journal of Semiotics, 6:58–72.10.37693/pjos.2014.6.11916Search in Google Scholar
Hatfield, C., Heer, J., and Worcester, K. (2013). The Superhero Reader. Jackson: University Press of Mississippi.10.14325/mississippi/9781617038068.001.0001Search in Google Scholar
Kaan, E. (2007). Event-related potentials and language processing: a brief overview. Language and Linguistics Compass, 1:571–591.10.1111/j.1749-818X.2007.00037.xSearch in Google Scholar
Lacassin, F. (1972). The comic strip and film language. Film Quarterly, 26:11–23.10.2307/1211407Search in Google Scholar
Magliano, J. P., Larson, A. M., Higgs, K., and Loschky, L. C. (2015). The relative roles of visuospatial and linguistic working memory systems in generating inferences during visual narrative comprehension. Memory & Cognition 44(2):207–219.10.3758/s13421-015-0558-7Search in Google Scholar
Magliano, J. P., Miller, J., and Zwaan, R. A. (2001). Indexing space and time in film understanding. Applied Cognitive Psychology, 15:533–545.10.1002/acp.724Search in Google Scholar
Magliano, J. P., and Zacks, J. M. (2011). The impact of continuity editing in narrative film on event segmentation. Cognitive Science, 35:1489–1517.10.1111/j.1551-6709.2011.01202.xSearch in Google Scholar
Martinec, R., and Salway, A. (2005). A system for image–text relations in new (and old) media. Visual Communication, 4:337–371.10.1177/1470357205055928Search in Google Scholar
Mazur, D., and Danner, A. (2014). Comics: A Global History, 1968 to the Present. London: Thames & Hudson.Search in Google Scholar
McCloud, S. (1993). Understanding Comics: The Invisible Art. New York, NY: Harper Collins.Search in Google Scholar
McCloud, S. 1996. Understanding manga. Wizard Magazine, 56, 44–48.Search in Google Scholar
Moore, S. (2003). In the old days, it woulda been eight pages. In: A Thousand Flowers: Comics, Pop Culture and the World Outside, Brady, M. (Ed.).Search in Google Scholar
Neff, W. A. (1977). The Pictorial and Linguistic Features of Comic Book Formulas. Doctoral Doctoral Dissertation, University of Denver.Search in Google Scholar
Osaka, M., Yaoi, K., Minamoto, T., and Osaka, N. (2014). Serial changes of humor comprehension for four-frame comic Manga: An fMRI study. Scientific Reports:4.Search in Google Scholar
Painter, C., Martin, J. R., and Unsworth, L. (2012). Reading Visual Narratives: Image Analysis of Children’s Picture Books. London: Equi-nox.Search in Google Scholar
Pederson, K., and Cohn, N. (2016). The changing pages of comics: Page layouts across eight decades of American superhero comics. Studies in Comics, 7:7–28.10.1386/stic.7.1.7_1Search in Google Scholar
Petersen, R. S. (2011). Comics, Manga, and Graphic Novels: A History of Graphic Narratives. Santa Barbara, CA: ABC-CLIO.Search in Google Scholar
Pratha, N. K., Avunjian, N., and Cohn, N. (2016). Pow, punch, pika, and chu: The structure of sound effects in genres of American comics and Japanese manga. Multimodal Communication, 5:93–109.10.1515/mc-2016-0017Search in Google Scholar
Royce, T. D. (2007). Intersemiotic complementarity: a framework for multimodal discourse analysis. In: New Directions in the Analysis of Multimodal Discourse, T. D. Royce, and W. L. Bowcher (Eds.), 63–109. Mahwah, NJ: Lawrence Erlbaum Associates.Search in Google Scholar
Schwan, S., and Ildirar, S. (2010). Watching film for the first time: how adult viewers interpret perceptual discontinuities in film. Psychological Science, 21:970–976.10.1177/0956797610372632Search in Google Scholar
Stainbrook, E. J. (2016). A little cohesion between friends; or, we’re just exploring our textuality: reconciling cohesion in written language and visual language. In: The Visual Narrative Reader, N. Cohn (Ed.). London: Bloomsbury.Search in Google Scholar
Tasić, M., and Stamenković, D. (2015). The interplay of words and images in expressing multimodal metaphors in comics. Procedia – Social and Behavioral Sciences, 212:117–122.10.1016/j.sbspro.2015.11.308Search in Google Scholar
Verano, F. (2006). Spectacular consumption: visuality, production, and the consumption of the comics page. International Journal of Comic Art, 8:378–387.Search in Google Scholar
Wilkins, D. P. (1997/2016). Alternative representations of space: arrernte narratives in sand. In: The Visual Narrative Reader, N. Cohn (Ed.). London: Bloomsbury.10.5040/9781474283670.ch-010Search in Google Scholar
Zwaan, R. A., and Radvansky, G. A. (1998). Situation models in language comprehension and memory. Psychological Bulletin, 123:162–185.10.1037/0033-2909.123.2.162Search in Google Scholar
© 2017 Walter de Gruyter GmbH, Berlin/Boston
