Let’s just, for a minute, assume that comics have audio. Of course, they don’t. We can’t hear the comic. But the use of onomatopoeia, and other expressive graphical representations of sound, allows us to read what we should be hearing. So, let’s assume we can hear the comic, that the comic medium has audio-visuality.
According to Hugh Bredin, onomatopoeia is characterised by the “relation between signifier and signified in which the signifier is motivated, in part, by its sound” (1996, p. 562). Onomatopoeia, such as “boom” or “bang”, is written in the way the referent word sounds and, as such, provides the reader with an enactment of the sound in question (ibid. p. 568), while becoming a way to experience the world described in words through sensory modalities. While onomatopoeia commonly features in both literature and poetry, going as far back as Plato who discussed it in Kratylos (Gonda, 1975), comics is where it has been turned into an art form.
Even though comics have no audio per se, it can be argued that onomatopoeia provides them with a functional method that, through the direct involvement of the reader and the audio-visual process, works by “bringing the full power of our own experiences to bear on the world our eyes report” (McCloud, 1996, p. 136), or, as Winifred Nowottny claims “enact the senses” (1969, p. 116). One can therefore say that audio is present, the ears are appealed to, but only through the one sense of sight, engaged and animated by the written word.
Graphically enhanced onomatopoeia in Kyle Higgin’s C.O.W.L.(Image Comics, 2014).
But, onomatopoeia is not only a relation of sound to sound, of similarity or instantiation. According to Bredin, onomatopoeia is also a form of “sound symbolism” that describes the relation of sounds to “semantic fields such as brightness and darkness”, making it a relation of association (1996, p. 568). And so, complex and graphically stylized onomatopoeia can express and describe audio both in terms of natural sounds and sound effects, even language and dialect.
Written dialogue given dialect through colour and reversal of letter ‘N’, while the onomatopoeia of the gun is enhanced by utilising expressive graphical representations. From C.O.W.L.(Image Comics, 2014).
According to Scott McCloud, onomatopoeia in comics attempt to capture the “essence of sound” (1996, p. 134), and therefore implies a synaesthetic relationship between the components. Within speech research, it is established knowledge that slight, universal forms of synaesthesia contribute to speech structures, and that fully to understand the onomatopoeic structure means including cross-modal relationships (Ramachandran et. al., 2001, Assaneo et. al. 2011).
Nonsensical onomatopoeia as spot-sound/sound effect and as foreign language/music, in Brenden Fletcher and Karl Kerschl’s Isola(Image Comics, 2018).
Of course, in comics, there is no audio per se, but I would argue that the usage of complex and graphically stylized onomatopoeicon is a very good attempt at capturing the “essence of sound” (McCloud, p. 134). Add to that the direct involvement of the reader, and the audio-visual process works by: “bringing the full power of our own experiences to bear on the world our eyes report” (ibid, p. 136).
The following is an attempt at outlining a process of analysis that can be used to locate words, graphemes, clauses, and paragraphs of a script that imbue expressive audio-visual potential that is carried over, and into, a subsequent production process, or what Genette terms ‘intermodal transmodalization’: the shift from one mode to another. The purpose of such a historical, psychological and biological inquiry into form is to, according to I. A. Richards, find out: “Given these words, in this order, what gives them the powers they have?” (p. 11). And so, it is a formal investigation. Formalism tries to understand how a text is put together and, from that, how it ‘works’. This is also where my interests lie and so, the purpose of my method is also to discover the function of a text’s expressive modalities by making inquiries into the process of the construction and materials of said text.
Boris Eichenbaum said, the purpose of the formal method is to establish a distinction between the elements of a works construction, or what he termed shuzet, and the elements comprising the material being used for that construction, what he termed fabula (p. 18). Such an inquiry also shares affinities with David Bordwell’s poetics of cinema; a systematic inquiry into the: “finished work of any artistic medium as the result of a process of construction” (p. 12). The objective is to find what materials are being organised through the artistic form, including themes, and subjects, and how these are organised, in an attempt to know the artist’s: “[…] secrets, especially those they don’t know they know.” (Bordwell, p. 22). And so, this text-to-imagery analysis is an attempt at understanding the creative process through which a script is constructed with what Bruce Isaacs call ‘scripted words’, words and graphemes that give presence to expressive audio-visual potential in the literal text itself.
Another aspect of this analysis is that it tries to provide some understanding of the creative process, the dialogue that occurs between the co-authors of a transmodal text as it is created, or as the shift occurs from one mode to the next. As Rosamund Davies points out, large parts of such a process is: “internal or verbal, and goes unrecorded” but that there are physical artefacts available as a result of the creative process, artefacts that can be recovered and investigated to “recover traces of these dialogues” (2013, p. 163). For this analysis, I have chosen the artefacts relating to the creation of The Private Eye #1, a comic written by Brain K. Vaughn, and art created by Marcos Martínwith Munta Vincente. Also available was a ‘Making of’ book with script, art drafts and email dialogue between the creators. Having only these two sources available, my aim is not to provide a complete avant-texte of the script nor the comic, but to use the material available in order to see what traces of the creative process might be discerned within this structure.
The Private Eye is a 10-issue science-fiction detective comic set in a future where the digital cloud was hacked, and everyone’s most sensitive information was suddenly available to everyone else. They called it ‘The Great Flood’ As a result, computers and digital files are now considered dangerous, even illegal, and it is common for people to have multiple ‘personalities’ manifested in costumes of different complexity, that are worn in public to maintain anonymity. The narrative moment in issue #1 that caught my ear so to speak, provided a distinctly embodied reaction. The lead up to the moment is the main character, P.I., visiting his grandfather, Gramps, a drug-addled hipster survivor of the flood, still trying to connect his redundant iPhone to a non-existing network.
Let’s have a look at the page (24) which had such an impact on me.
Page 24 from Brian K. Vaughn, Marcos Martín, and Munta Vincente’s The Private Eye #1 (Panel Syndicate, 2013)
The page describes a moment between P.I. and Gramps, in which P.I. explains the ‘Great Flood’, yet again it seems as Gramps is conveniently senile, and Gramps, outwardly, refuses any responsibility for, even knowledge of, the flood. It consists of seven panels, distributed in a symmetrical lay-out, dominated by the storm cloud panel and P.I.’s dialogue/voice-over that explains the ‘great flood’. The page is laid out in such a way as to create a downward flow: your eyes start at panel one, top-left corner and flow down, then up and down again; then up and down again ending on panel seven bottom-right.
Panel one and sets up the conflict. As we move up to the next column, the flow takes us down the dominant centre panel of storm clouds. Trickling down P.I.’s three-bubble dialogue, we float down, ‘tap-tap-tap-tap’ as the storm clouds burst, and the first drops of rain come down.
Then, as we move up to panel five, top-right corner, the downward motion is broken. We cut from the close-up of rain drops and its onomatopoeia, and back onto Gramps just as, like drops of rain, slow realization dawns on him. But, the flash of recognition is quickly broken. Holding on Gramps for panel six, he regresses into bitter and stubborn denial.
Then, the cut to a wide-frame of the two characters is the moment that got me. As we pull out, the trickling raindrops of recognition have turned into a plunging downpour. In a perfectly synchronised moment of synaesthetic impact, the rain hammers down, and Gramps shoulders sink, his head drops, and slow realization sinks into/onto him.
Onomatopoeia, the downward panel flow movement and lines of plunging rain, the full mise-en-scène of the wide-frame combine to provide an element that not only works as a metaphor for the ‘flood’ thematics, but also generates an audio-visual synaesthetic moment of embodied rapture, flowing as it does from the previous panels, always emphasising the downwards movement, and, just to cap it off, functioning as a perfect audio bridge, carrying the narrative into its next ‘scene’, as it the downward pour continues.
Now that we have located the particular, the moment of synaesthetic synch in the audio-visuals of the comic page, we can trace it back to the written word, the text of the script. To disassemble this construction into its working parts, I will utilise a method that I will call Screen-to-Text, combining elements of Bordwell’s ‘Poetics of Cinema’ methodology, Chion’s ‘Audio-Visual Analysis’, and Eisenstein’s ‘Word to Image’ process. The application of this method allows for schematisation of the audio-vision, particularly the important points of synchronisation, and the functions and effects of the aural and visual devices. These schemas can then be used to trace the audio-vision backwards through to the screenplay text, enabling direct comparisons between the two texts, seeking out the written language used to formulate the on-page audio-vision.
Scanning the script page, we notice an immediate difference. Page 24 is divided into five panels, not seven as the total is in the finished page. The two panels that do not appear in the script is the storm cloud panel, and the onomatopoeic close-up of the rain drops, panels five and six respectively.
As a matter of fact, there is no mention of rain, or storm clouds anywhere in the preceding script pages. The script for page 22 actually mentions the sun setting.
And, the script for panel one on page 25 states that the sun has set.
Clearly, the storm and the rain has been added by artist Marcos Martín. A close read of the script however, reveals the audio-vision of the storm/flood thematics is already present. The script for panel four and five on page 23 mentions: “the cloud” and, more importantly: “one day that cloud burst”.
Next, for panel one on page 24, the ‘great flood’ theme is carried on with “burst” again, and “[…] for forty days and forty nights, the cloud poured down for the whole world to see”.
Not only is the rain/flood thematic continued, but the movement and flow that is such an important element of the page’s synaesthetic audio-visuality is also set up through the words “poured down”.
A close analysis of the available palimpsest of textual artefacts provides a taste for the creative process that brings words in a text to visuals on a comic-page, visuals imbued with the essence of sound through onomatopoeia and graphical expressions. Utilising the screen-to-text methodology allows us to locate the points of synchronisation in the final work, and trace these back to the literal text, the script, and reveal the words and graphemes in the literal text itself that give presence to expressive audio-visual potential that springs forth through Marcos Martín’s art.
Bibliography
Assaneo, M. F., Nichols, J. I., & Trevisan, M. A. (2001). The Anatomy of Onomatopoeia. PLoS One, 6 (12). Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3237459/
Bredin, H. (1996). Onomatopoeia as a Figure and a Linguistic Principle. New Literary History, 26(3), 555-569. Retrieved from https://muse.jhu.edu/article/24304.
Bordwell, D. (2008). Poetics of cinema. New York: Routledge.
Davies, R. (2013). Don’t Look Now: The screenwork as palimpsest. Journal of Screenwriting, 4(2), 163-177.
Eichenbaum, B. (1965). The Theory of the Formal Method. In L. Matejka & K. Pomorska (Ed.), Readings in Russian Poetics: Formalist and Structuralist Views. (pp. 3-37). Ann Arbor, MI: University of Michigan Press.
Eisenstein, S. M. (1968). Word and Image. In J. Leyda, (Trans. Ed.). The Film Sense. (pp.3-65). New York: Harcourt Brace Jovanovich.
Fletcher, B. (Writer) & Kerschl, K. (Writer/Artist). (2018). Isola #2 [Comic]. USA: Image Comics.
Genette, G. ([1982] 1997), Palimpsests (trans. C. Newman and C. Doubinsky), Lincoln and London: University of Nebraska Press.
Gonda, J. (1975). Some remarks on onomatopoeia, sound-symbolism and wordformation a propos of the theories of C. N. Maxwell. Selected Studies Vol. V: Indonesian Linguistics. (pp. 166-243). Leiden: Brill.
Higgins, K., Siegel, A. (Writers) & Reis, R. (Artist). (2014). C.O.W.L. #1 [Comic]. USA: Image Comics.
McCloud, S. (1994). Understanding Comics. New York: Harper Perennial.
–––– (2006). Making Comics: Storytelling secrets of comics, manga, and graphic novels. New York: Harper Perennial.
Ramachandran V., & Hubbard E. (2001). Synaesthesia: A window into Perception, Thought and Language. Journal of Consciousness Studies, 8, 3–34.
Richards, I. A. (1960). Poetic Process and Literary Analysis. In T. Sebeok (Ed.), Style in Language. (pp. 9-24). Cambridge, MA: M.I.T. Press.
Vaughn, B. K. (Writer) & Martín, M. (Artist). (2013). The Private Eye #1 [Comic]. USA: Panel Syndicate.