Fig 1: Maurice Sendak: Where the Wild Things are. (1963) Harper and Row
SIPE, L. (1998) How picture books work: A semiotically framed theory of text-picture relationships. Children’s literature in Education Vol 29 (2) p97-108 [online] . Available from: http://web.a.ebscohost.com.ezproxy.herts.ac.uk/ehost/pdfviewer/pdfviewer?sid=52fb9a64-da7b-46b2-8ea3-ac13be3fa4d4%40sessionmgr4010&vid=1&hid=4109(Accessed 13 December 2016)
Sipe has a PhD in Children’s literature and has a research interest is in the development of literary understanding. I’ve chosen this article as I’ve introduced text as an integral element in my illustrations and want to clarify how they are read and understood with each other. In this article, Sipe examines the image-text relationship within picture books, but it is relevant wherever image and text exist together.
The article begins by referencing alternative descriptions of the relationship using metaphors from music (antiphonal, duet), science (interference) and geology (plate techtonics). Sipe then briefly explains other approaches to defining the image text relationship, and includes the ideas of Barthes, Nodelman and Lewis, among others. Sipes theoretical approach is phenomenological. He identifies the relationship as a synergy, where the value together is stronger than the values individually.
Sipe then systematically refers to a range of literary theories that examine the reader response to a text, and their perception of meaning. He goes onto explain the reflective process of reading a book and how this creates a tension between wanting to read on with the words, but dwell on the images. He suggests that the way a picture book is read invites re-reading because the tension stops a full-speed-ahead read.
The second part of the article is about transmediation, and uses the semiotic theories of Pierce to explain how a reader moves between the dual sign systems of visual and verbal to create an understanding of the whole text. A double page spread from “Where the wild things are’, by Maurice Sendak is analysed through a Piercean semiotic triad (simpler than it sounds!). Plainly put, it’s where the eyes travel over the narrative space, and how the reader interprets and relates the visual and verbal signs.
This is a useful and concise dissection of how text and image work together. It has a thorough bibliography for further study, and an overview of the leading theories.
Particularly useful for my own project is the idea of hyperlinking between text, image and from page to page.