Saturday, April 11, 2015

Words and Images: Teaching English in a Visual Culture

For this week’s readings I’ll be focusing primarily on Gunther Kress’s “Gains and losses: New forms of texts, knowledge, and learning.” In exploring his piece, I’ll be circling a few general questions: What do Kress’s “gains,” or the enriching advantages of an increasingly visual culture, (as imagined by Kress) mean for English teachers and students? Relatedly, I wonder what bearing the infinite possibilities of visual meanings, posited by Kress, has on assessment of multimodal “texts” – a topic we’ve covered before but that I’d like to revisit in light of Davis & Yancey’s compelling discussion.

First, a brief synopsis of Kress:

 Kress begins by describing a cultural shift “from the centrality of writing to the increasing significance of image” (6). Kress uses semiotic theory as a means of examining the distinct possibilities and limitations of written/spoken language (though he is careful not to conflate these) and visual depictions. Through this examination he hopes to present a clear-headed assessment of what he calls “gains and losses,” untainted by the emotionally charged nostalgia/pessimism or “unwarranted optimism” often invoked in the discussion of changing cultural values regarding representation and communication. Kress compellingly implicates his discussion of the distinct merits/limitations of these separate modes in changing social and cultural attitudes which may begin to account for new kinds of “texts.” To illustrate this, Kress compares the Institute of Education’s prospectus from 1992 to the current (via 2005) version of their webpage. He notes the linearity and rigidity of the 1992 version, which has a single entry-point, seems to presume that “the structure of the institution and of its knowledge were identical with the needs of the life-worlds of the individuals who might come to it as its students” (9). Conversely, the latter webpage has 13 points of entry and follows the “image-based logic of contemporary pages” (9). [He’s describing it a decade ago, here’s the currentpage, still image centric.] For Kress, this stark contrast reveals changing attitudes about authority and authorship, which will become a central tenant of his discussion. In the linearity of the 1992 prospectus “the power of authorship was strictly governed,” as opposed to the later formulation in which, “the author(s) of this page clearly have in mind that visitors will come to this page from different cultural and social spaces […] not necessarily know to or knowable by the maker(s) of this page” (10).  Kress comes to associate this uniquely open image-based logic with greater reader/viewer agency and a kind of erosion of the binary between author and reader, which will have important implications for students.

I’d like to briefly outline the stark distinctions he makes between visual depictions and spoken/written language.

Speech/Writing: temporal, narrative
Image: Spatial, display

The question asked by speech: “What were the salient events and in what (temporal) order did they occur? (14)
The question asked by display is: “What were the salient entities in the visually encountered and recollected world, and in what order are they related?” (14)

These delineations lead Kress to make some evaluative observations about the possibilities of these categories. He posits that “because words rely on convention” they are general and vague, while depictions, on the other hand “are full of meaning; they are always specific” (15). He furthers this prizing of visual over word by suggesting that words are limited in their finiteness, we can’t express something we don’t have a word for while images are infinite; “the former tend to occur in […] fixed order […] the latter tend to occur in an open order fixed by the reader and/or viewer’s interest” (Kress 16).

So all of this, I think, begins to lead us back to the classroom. I found the most interesting part of Kress’s discussion to be the way in which he aligns these distinctions with compelling questions about teacher and student subjectivity and agency. Drawing on the “visual geography” image to the left he asks:

“What is the assumed subjectivity of the students to whom not just this aspect of the curriculum but nearly all of science is presented in this manner? And equally, what is the subjectivity of the science teacher who teaches science in this manner? [what are the] implied notions of convention, of competence, of knowledge, and of authority?” (19).
Earlier, Kress posits that when we “read” visual texts we compile information, but have the agency and control to create our own knowledge – but when we read linear, narrative texts, knowledge is presented ready-made (10). Do we agree? Do you think these kind of visual aids more readily allow students to fashion their own “knowledge?”

Kress obliquely answers his own questions by asking more. He ends his piece with a series of questions that read more like statements. In fact, I was a bit puzzled by the title of his piece because this doesn’t seem to boil down to an exploration of gains and losses of each category – but rather, a more straightforward gain loss hierarchy in which images occupy the former category, and words, the latter.

Kress asks, “Would the next generation of children actually be much more attuned to truth through the specificity of depiction rather than the vagueness of word?” (21). Though I think Kress’s framework and assertions about subjectivity are compelling, I have a hard time accepting the word/image binary that seems to be at work in this piece. Are these things really so distinct? And are images really less convention laden or finite than words? Drucker discusses graphic “ideologies” and I’m tempted to think that visual representation is just as socially constrained as language. Finally, if we accept the supremacy and infinite possibilities of the image, what does that mean for English teachers? Further, do we need to teach students how to “read” images, or are they inherently abounding with meaning? If a visually representative culture gives more agency to the viewer (and, also the designer) do we need to teach them how to use it? And if so, how?

 Further, the infinity of visual meanings (which are also somehow specific for Kress?) seem to add an interesting dimension to the recurring issue of assessment. I really appreciated Davis and Yancey’s thoughts on this subject, I wonder if others felt similarly?

They write: “assessment is about what dialogue one might have” and speculate that it could potentially be a dialogue “about meaning-making, about how we make meaning and what meaning we make out of that” (14). This focus on meaning making, and secondarily, interpreting that very meaning making, seems apt given Kress’s attitude, which seems to imply that the possibilities of multimodal texts are infinite. If there are infinite meanings, then the assessor has infinite choices leading one to believe that when assessing these things, we have to pay close attention to the choices we make – in other words, to interpret our own interpreting, if you will. For current teachers, what’s are some of your strategies? Did you find Davis and Yancey’s discussion of scrapbooks/e-portfolios helpful? 


  1. Thanks so much for this incisive interrogation of Kress! If you read my tweet on this topic, you know that I do not agree with Kress (at least in some respects). In fact, I included Drucker on the syllabus this year as a counter to Kress. Whereas Kress sees the visual as "open" and "designed by the reader" (10, 11) and wants to move "Beyond critique to design" (16ff), Drucker argues for the danger in a non-critical approach to the visual, one that allows charts and graphs "to pass as unquestioned representations of 'what is'" (125). Kress, like another social scientist Bruno Latour, sees the work of critique as finished, and wants us all to focus on building things, focusing on the future-oriented skills like design. By contrast, Drucker believes that visual depictions like charts and graphs have indeed become closed forms, subjected to the conventions that Kress seems to associate exclusively with language. She therefore argues at the end of her chapter that these "realist models" (hence Kress' focus on specificity) "need[] to be subjected to a radical critique to return the humanistic tenets of constructedness and interpretation to the fore" (125).

    As I think I've indicated, my sympathies lie with Drucker, mainly. That said, I think Kress is very much right that the visual is gaining a dominance that I don't think we should necessarily lament. To get to your pedagogical question, Theresa, I believe we do indeed need to teach our students both the skills of design AND critique. We don't need to oppose these two important creative acts.

  2. Theresa, excellent post! I think that given the subject of this post it’s especially rad that you included text, image, and hyperlink. Well done! Now, a couple of points that I’d like to address!

    First, I’m glad that you took the time to mention Kress’ comparison of the Institute of Education’s 1992 and 2005 prospectuses. Although it’s worth considering just how much changed in thirteen years, I can’t help but think about the following ten-year progression to 2015. It’s with a scoff of embarrassment, I imagine, that “the linearity and rigidity of 1992 version,” is described by Kress. However, what are we to make of today’s possibilities of representation? Can we appreciate them for what they are? Will we look back and sneer, having already moved onto something else? Or will we only know just what we’ve got well after we’ve passed the present.

    Perhaps it’s tangential, but I think of Warren Ellis’ HOW TO SEE THE FUTURE talk: “We are summoning it into the present. It’s here right now. It’s in the room with us. We live in the future. We live in the Science Fiction Condition, where we can see under atoms and across the world and across the methane lakes of Titan.”

    Secondly, I’m definitely interested in seeing what everyone makes of Kress’ idea that visual texts and linear/narrative texts produce drastically different responses. As an individual and a teacher (sorry – couldn’t think of another way to differentiate), I’m uncomfortable with this sort of binary opposition being set up. After all, visual aids and liner/text-based works are not inherently oppositional, and can actually be used in tandem. Anecdotally, I’ve seen students who’re prompted to create/explore when engaging with the visual, some who’re more inspired by narratives, and case-by-case bases which vary.

    Moreover, I’m especially intrigued by the combination of the two. In fact, in many ways isn’t this the very concept of comic books? There’re even comic artists who revel in the possibilities of this marriage. Check out Jonathan Hickman, who loves infographics:

    Anyways, I excited to see what everyone’s thinking!

  3. Theresa, great post. I’m glad you ninja chopped Kress’ piece and really brought it into the context of our class by considering it within our ongoing conversation.

    I’d like to first start with addressing some of your questions and try to add to Allen’s post. You ask: “Do we agree? Do you think these kind of visual aids more readily allow students to fashion their own ‘knowledge?’” I would disagree that the linear narrative only allows for a “ready-made” perception. I think that both the narrative and images allow for us to create meaning. I think what Kress is lacking to do is use the humanistic approach to both the image and the narrative. Ideally, we should be teaching students to engage in dialogue with both narratives and images, asking questions that allow meaningful connection – or, as I like to say, “Let’s make a memory.” While there may be some affordances of one mode and not the other, I think our approach should ask students to use them together and separately. For example, just think about how we would critically approach a comic book, as Allen has said. It would be impossible. I wonder how Kress would feel about our digital story projects?

    Theresa, as you say, it would be difficult to separate image and word. Just thinking about our the format of your blog post, how would we separate the image and your words in this context? I really like your line, “Drucker discusses graphic ‘ideologies’ and I’m tempted to think that visual representation is just as socially constrained as language.” I would have to agree. Even though there is an image, we are always approaching it as something to talk, write, or remix about. I think as we have been discussing in class, importance lies in student’s adaptability to make meaning of many forms and create a new product in whichever medium is suitable. I think in this notion, we can look towards Drucker, especially when she says, “Recognizing that such methods (constructivist) are anathema to the empirically minded makes even more clear that they are essential for the generation of graphical displays of interpretive and interpreted information” (135). This just shows how much is involved in the humanistic approach to creating and processing images. In this sense, Drucker is saying when we use graphics we must recognize what we are trying to show and who will see them.

    As for your questions about showing students how to look at the multimodal “choices we make” it is a good idea to have students look at the affordances and constraints allowed by many modes. The same questions like: “Would this help expressing a comparison?” or “What type of story will hook your reader?” could be applied to any form of expression. While we normally teach students in K-12 towards standardized assessments (dare I say), I think we see that these generally do not allow for students to express themselves as they might find normally outside of school. If the modes of assessment don’t change, kids will only get more and more apathetic towards their own education.

    I really enjoyed the Davis and Yancey article, especially how a tactile mode can be added to meaning making – i.e. the difference between the hard copy and digital scrapbook. I think more than anything, this article showed how important narrative is to us as human beings. Stories allow us to connect with one another and feel part of something bigger than ourselves. They say, “Letters and cards and newsletters and postcards create, narrate, document, and represent a material mosaic of experience for a group of people increasingly bound to each other, people who largely had never met” (26). I think this is something that is becoming harder to come by as we move towards data driven instruction, at least in K-12. While meaning can be made on an intellectual and factual level, it is sometimes beneficial to make meaning on the emotional and philosophical. I know some of our readings, especially Sherry Turkle’s piece, have suggested this too.

  4. Thanks for the great post, Theresa! You’ve done a great job analyzing Kress while integrating Davis and Yancey’s thoughts on the visual presentation and linear narrative.

    When I was reading this week’s materials, I could not help but wondering which side I should go. I find Kress’s setting up the binary relationship between image/writing quite convincing in a way, considering the presence/ambiguity of the author and the single/multiple reading path. However, emotionally I’m also quite unwilling to separate the two notions, and I’m happy to see that all of you are on the united front with me. And it also seems to me that while linear narrative can always be independent from graphics to make meaning, graphics sometimes need the help of words to articulate the meaning it wants to convey. I was also, during my reading, thinking about the example of comic books, which is kind of a remix of images and words. Here in Boston my roommate is an undergraduate who is taking some basic writing courses, and I’m surprised to find that many of her reading assignments are comic books! So I assume that teachers may start to be aware that the integration of images and narrative is a good way in arousing students’ interests as well as getting them to know how meaning can be made effectively through the interaction. And I mentioned in my twitter that the idea of scrapbook is quite fresh and fancy to me, but I may like to try it in my future class, as a method to navigate students’ journey of learning to make meaning.

    Like Brian, I also share the same feeling with your line that “Drucker discusses graphic ‘ideologies’ and I’m tempted to think that visual representation is just as socially constrained as language.” When describing the T-O maps (80), Drucker says “they were symbolic, and fulfilled an exception that the earth conform to a Christian plan of divine design”, which reminds me that for most of the time, images are just as ideology/culture-loaded as written/spoken words. It, to some extent, corresponds to the criticism Kress imposes on words, the readers of which “interpret[e] what sign the writer may have intended to make with this signifier” (7). That is to say, although in an image the author is ambiguous, but he still exists and kind of manipulates how we interpret his work. So yes, Kress seems to be too optimistic about the tendency towards the visualization of everything.

    In Drucker’s book, she gives an explicit account of the history of graphics. But I feel perplexed that as graphics came into being earlier than words, can we say the “revolution” of modes of representation, namely “from the centrality of writing to the increasing significance of image”, is the revival of images? Besides the media which carry the images turn from stones to screen, is there any essential distinction between the old images and the newly popular ones?

  5. Excellent post, Theresa! It seems to have sparked some really in depth discussion. Many of you mention comics, which is also where my head went while reading this. I can’t think of a more perfect example of writing uniting with image to produce a narrative. Like Allan, Kress pitting the visual and text against each other unsettles me. I am not even certain how to respond, other than to think of all my experiences where text and the visual have been in concert or separate and produced equally as rich responses.

    Kress points out that “writing is giving way, is being displaced by image in many instances of communication where previously it had held sway” (5) and at first reading it made me laugh. We cannot deny that image is becoming more important, as images and design is a key element in catching the attention of viewers/readers—but on the other hand, considering how long text and image have been used separately, we’re now in a period where image and text are combining to create meaning together.

    At the question of whether or not to teach students how to “read” images, it strikes me as something that comes closer to art and design rather than something that falls under the purview of English. This may have a basis in my own schooling, where we studied images and the composition of them in a visual art class. Depending on the image, it may have “inherent meaning” or it may not, someone unfamiliar with studying art may be able to take a leap at an interpretation of Munch’s “Scream,” while abstract art would be something that does not easily lend itself to interpretation. An infographic gains meaning through the text combined with the image, comic books can go with or without text and still have equally as much meaning—the panel layout is particularly useful in leading the reader along a predetermined path.

    Like Brian and Jiuqing point out, images are at the mercy of our socially constructed ideologies and cultures, just as much as text is. Kress idea that images are somehow freer to interpretation than text falls flat for me, because like the others observed, he does not seem to take into account the fact that we as people, student, individuals, and teachers, are products of our environment and are inherently influenced by it as a result. If anything, limitation in interpretation lies within the viewer/reader more dominantly than within the images or texts themselves.

  6. Great post, Teresa! I have to agree with you that I am not so sure about Kress’s implication of visual images taking a supremacy over words. I feel that there are a lot of things images can not provide for the reader (like context), and without some sort of textual accompaniment, a reader has no idea how to “make meaning” from the images. Even though Yancey points out that, “the ways we “read” photos are likely different than the way we read their captions,” (14)- I still think photos on their own need some sort of explanation. Images and videos do enhance texts, but on their own they cannot possibly substitute for a full-length narrative. Narratives offer so much that images are unable to, like teaching students conventions of English grammar, vocabulary, and pragmatic use of the language.

    As for Yancey’s discussion of scrapbooking, I have done this as an assignment in the past. For our eighth grade trip to Washington D.C. our English teachers assigned us to make a scrapbook of pictures and then answer writing prompts/reflections about places we visited while on the trip. We were to compile the pictures and responses into a large binder and turn it in for an end of term grade. At the time I couldn’t understand the reasoning behind the assignment at all. But as Yancey says, the fact that you can touch the scrapbook and interact with it makes it have more of a meaning for the reader (15-16). Until reading the article, I would never have considered this, but of all my writing assignments from grades six until now, the scrapbook is one of the only writing assignments I have kept, simply because the pictures (multi-modal text) do give it much more meaning than any other paper that I have written.

    Also, for some reason Yancey’s discussion of scrapbooks really makes me think of Drucker’s presentation of “trees of knowledge” (95). There are so many similarities between the two, mainly being that, “trees of knowledge are graphical forms whose structure is static and fixed, but whose spatial relations carry meaning,” (95). The creator/author of a scrapbook usually creates their scrapbook in such a way to convey meaning to the reader. The arrangement of the photos and text mean something, and its up for the reader to “make meaning” not only from what is being directly presented, but also through the arrangement of the presentation. I think it is important for teachers to educate students on how images can be read, however they should also include textual explanations with their alternate modalities if they are to substitute a longer writing project.

  7. Thanks, Theresa! This is a thorough analysis of a pretty dense piece, and I found it really helpful when I re-read Kress. I was starting to think that I had missed something, because the title is a bit misleading. I agree with you that this isn’t really an examination of the gains and losses but rather “a more straightforward gain loss hierarchy in which images occupy the former category, and words, the latter.” Nevertheless, Kress details some interesting and valid points about our shift from words to images as the preferred form of representation. His argument that “it is the viewer’s action that orders the simultaneously present elements in relation to his or her interest” (13) is true in a very literal sense. Yes, the arrangements of images is different and not as sequential as those of words in a novel, and perhaps provides the reader with more responsibility and authorship to construct his or her own knowledge, but I would argue that the same can be said of words on a page. Although the physical layout is different, Kress seems to be arguing that words cannot allow readers to “fashion their own knowledge” or that words can’t be “selected by individuals to be transformed by them into knowledge to solve a problem in their life-world” (10). We all know that two people can read a poem and arrive at two entirely separate interpretations of it, so while I get what he’s staying, I can only agree with him to an extent.

    While reading this article, I started to think of The Garden of Forking Paths that we read a few weeks ago. I don’t think a piece of writing has to necessarily follow that format in order to allow readers the authority to construct their own knowledge, but that’s another example of a text that provides much of the “gains” I think Kress is pointing out about images. Are images really “always specific” as he claims? People have argued for centuries whether or not all language is argumentative, and isn’t that a similar question?

  8. Thanks for this, Theresa! So the first place my mind went to after reading your post was actually a discussion I had back in my Theory and Film class. In it, we talked about how some of the most blockbustery-blockbusters are designed specific to the foreign market. These movies (think Transformers as the perfect example, but The Avengers also clearly works) are designed to be a visual spectacle, crafted so that the dialogue of the film can be ignored in favor of the impressive action and set pieces. I viewed this concept pessimistically when we discussed it, and although I still have large reservations about potential applications, I think that Kress points out a potentially beneficial use of this type of thinking. I think he’s onto something with the idea of image/visual based spaces increasing accessibility, and (as with the example of the website) has the potential to be utilized for a beneficial end.
    But I also think I agree with you on your reservations about a paradigm that bequeaths supremacy on any particular mode. I’m thinking here actually of what Scott McCloud says in his book, Understanding Comics. I don’t have it on me right now so I can’t quote from it directly, but in one chapter he makes unpacks the relationship between words and images, revealing them to be both be opposite ends (when taken in their extremes) of the same spectrum. After all, words (or, more specifically, the letter that made them up) started out as images that depicted a one-for-one sign/signified. Over time, though, these images increased in their abstract nature until they had separated far enough apart so as to appear, if not entirely recognizable, then at least dynamically altered. And this development happens with any memetic images; McCloud gives examples about how, in Japanese comics, nosebleeds have come to symbolize sexual arousal in comics, even when a nosebleed would not make immediate sense otherwise. Another example might be the presence of flies representing stink in a particular area.
    If these two methods are not oppositional, but rather two extremes on the spectrum of representation and interpretation, then it seems silly to champion one side predominantly at the expense of the other. It just means that there must be much consideration into how you want to display something – satisfying questions concerning accessibility certainly being one of them – when crafting the piece of media.

  9. Thanks for the post. I think you are right to look at the idea of Davis and Yancey about meaning-making. There certainly is value in making things like images more prominent in education, as Kress suggests, especially for the upcoming generations. I think that your question, “if we accept the supremacy and infinite possibilities of the image, what does that mean for English teachers?” is the right one to ask. Certainly text can be vague or misleading, but so too can images. In this regard, the role of English teachers must remain, to a large extent, what it always has. It is the duty of the instructor to help students to be able to recognize context. For a text, this could come in the form of what the author’s background or motivation is, while for an image, this could take the form of asking the same questions about the composer. Like texts, images must be considered for what there perspective is, what images they’ve included and what they exclude, and how accurately/artistically is the subject being represented.

    In the classroom, I have tried presenting ideas to students in mediums other than printed text. I have found great success in bringing in Youtube clips, and excerpts from radio shows (Radiolab, The Moth, This American Life). I have used these as ways of introducing topics, but also, as a way of offering examples of a concept that has been explored in on of the texts. The students seem to engage with the texts in ways that they may not with the texts. And, in this way, the students are able to connect the idea from to an example that is not a personal experience, which they be unwilling or unable to do. I’ve yet to require that they include analysis of these stories in their essays, but inevitably examples from these programs make their way into the students’ writing. Another way that I have used this media, is by playing a segment from Radiolab, and having the students analyze the structure of the segment. I had them try to find when the subject shifted, where the thesis was, and how they used examples to support their claims. This seemed to show the students that writing does not have to be (or should not be) academic writing.

  10. Theresa, great post! I like how you included text, image, and a hyperlink in it. It really connects to the “visual culture” Kress talked about.

    I do think that the visual aspects of learning are becoming more prevalent, but I don’t think that is necessarily a bad thing. Throughout all of our schooling, we were always encouraged to use visual aids when we did presentations. Why shouldn’t we continue to do it as we teach? I’m not a teacher right now, but I would like to think that when I start teaching I would try to blend visual and textual reading in my classroom. For one thing, I think that visual aids help include student who may have specific learning disabilities in reading. And for another it just makes the text more fun/interesting for all of the students. We, as teachers, would have to teach them how to “read” a visual aid. If we used a painting that is connected to a work of literature, we’d have to teach the students how to look at the background, the facial expressions, the surrounding, etc. I feel that it would help them use other parts of their brain and it may help them remember the information they learned in class better than text alone would have.

    Keep in mind, I don’t think that we should eliminate text completely. That would be crazy. But I think finding a good balance between visual reading and textual reading would help the students immensely, especially if they may be having trouble with the reading.

  11. Thank you for your thought provoking post, Theresa. While reading Kress, Drucker, et al., I was stricken to think of our upcoming remix assignment in reference to their opposed arguments as visual/pictorial arguments as open and spatial versus closed and specific. I believe relegating images as open an interpretative process as Kress does is misguided. If anything, images become more specific views of interpretation in many cases than words are. I think my fellow classmates bringing up comic books as a very specific example of this. Representation in movies I think is another example that seems to deflate Kress's argument. In relating this back to our remix project, while I think it definitely is wide open for the writer/user of images to use them in different ways to make a point, I think the meaning behind those definitions become fairly concrete and finite once placed into the context they are used in.

    To more specifically address some of the very interesting question Theresa poses in her post, I would say images representation is only non-conventional in that we are only know realizing the potentials for using images as a representational force with the advent of more sophisticated technologies that make it more easy to allow them represent what we want them to represent. I think then it becomes incumbent on us as teachers to focus on the context images are presented in and help students to recognize how important context is (perhaps even more so in the course of both writing with and reading images than with words/speech) in "reading" images.

  12. Thank you for your engaging post Theresa! I found your introduction of Kress helpful.

    One of the problems I had, in addition to those already mentioned above, is the assumption that reading is always "right to left," without considering other forms of reading that do not follow this trend. I am thinking of examples like Manga, art work (sculpture, abstract art, graffiti, etc.) and other medias and mediums that are not read in the "traditionally Western" way. Which made me wonder how the act of reading varies among the various forms of reading and how the act of reading impacts what we "learn" from the source. Perhaps I am reading too much into this, but I believe reading is not merely limited to the page or what is perceived as a traditional text (whether online, tablet, book, etc.).


MOOCS: A Problematic Solution to the Disinvestment in Public Higher Education

I agree wholeheartedly with Bady’s cynicism of the speed, inevitability, and necessity of the MOOC movement. From 2011-2015 I direct...