Intro to Multimodality
Behind the concept of multimodality is the want to communicate a meaning. The study of multimodality aims to achieve the communication of a meaning as effectively as possible (Kress, 2012). Communicating can be done can be done through different modes like text, image, sound and music etc (Jewitt & Kress, 2003). A mode is essentially “a system of visual and verbal entities created within or across various cultures to represent and express meanings” (Serafini, 2014, p. 12). According to Frank Serafini (2014), all modes contain a different ability to convey a certain type of meaning. Yet, with multimodality, more than one mode is used, because this might help communicate the meaning more effectively. In Reading the Visual: An Introduction to Teaching Multimodal Literacy, Serafini (2014) refers to something that is multimodal as a multimodal ensemble. He argues that the term multimodal text, as it is often called in academics, lays to much emphasis on text that is often automatically associated with print. Yet, a multimodal ensemble and a multimodal text inherently are the same thing (p. 12). Serafini (2014) ultimately defines multimodal ensemble as “a complex, multimodal entity that occurs in both print and digital environments, utilizing a variety of cultural and semiotic resources to articulate, render, represent, and communicate an array of concepts and information.” (p. 13).
So based on what we are trying to communicate, we can pick and choose modes. Nonetheless, now it is important how to actually use modes effectively. According to Kress (2012), in an interview called “What is Multimodality”, we pick certain modes because they will allow us to articulate the meaning we want to convey the best. Here we combine a sense of what a mode will do and a sense of what the meaning is we want to communicate. Therefore modes are arranged by the means of the interest of the person communicating a meaning. Yet the audience has to also be in mind when choosing which modes to use. Here we have to consider for example; what would be most interesting and unforgettable for the audience?
Kress (2012) suggests that in the different functions of modes is a conventional notion embedded in our society. These conventions in our society essentially indicate to us what the best mode is to use for a specific kind of meaning. Kathy Mills and Len Unsworth (2018) therefore call modes “culturally shaped resources” (p. 5). For example the notion that for academic purposes, writing is the best mode to use. This conventional notion is embedded in our society because of the history of writing, where writing carried most of the academic information available to us (Kress, 2012). This is explained in more detail in the blog post about the history of multimodality.
However then why is it so important to have knowledge or even consider multimodality as an approach? Kress (2012) in “Why adopt a multimodal approach?” explains that without considering every mode, an analysis of anything that is trying to convey meaning would be incomplete. Indicating that if you would exclude one mode from the other modes that also contribute to the meaning that is being conveyed, your analysis would not be able to identify a part of, or possibly even all of the meaning.
In order to illustrate the concept of multimodality further, and in the hope to make it even clearer, I will now illustrate a few examples that can be recognized in day-to-day life. If you rather prefer to listen to examples of multimodality through a more interactive explanation of the concept, you could also refer to our podcast on multimodality: A Living Room Talk: Discovering Multimodality.
The first example of multimodality that I will be discussing that everyone has most likely dealt with in their life is instruction manuals. Take for example the instruction sheet to assemble a closet from Ikea. Most of us have handled such an instruction manual before, but have likely never considered the different modes that are embedded in it. Within the instruction manuals from Ikea there is a use of text, images and symbols. This is thus a combination of multiple modes in order to convey a meaning. In this case, the meaning this instruction manual is trying to convey is how to correctly assemble this closet. Text is used, as seen in the images, to convey a warning, to make sure certain actions are prevented. Text is additionally used to convey the order of the steps that one has to take. These meanings are most clearly communicated through the mode of text. Images on the other hand, are used to convey what instruments should be used and what parts should be put in which place. Having a visual of the specific part or screw you need to have makes it way easier to identify this at home than a written description of the article would. Additionally, in the example below, symbols like a cross is used to indicate which screws not to use. The use of symbols communicates this meaning very clearly and quickly. This is a perfect example of multiple modes used to achieve the most effective way of delivering this meaning to the reader.
Another great example of multimodality is a storyboard. Storyboards are most frequently used in filmmaking, though can also be conveniently applied when mapping out any sort of project. With the use of a storyboard, someone can chronologically map out a structure with the use of for example text, symbols and drawings. I have included the storyboard we made to map out the podcast about multimodality. As this is just a rough draft, it still examples how it could be used. Making a storyboard can make your ideas more visible and clear, and additionally help you be more creative.
Additionally, multiple great everyday examples of multimodality are to be found on social media platforms. Though there is a lot to delve into considering this example. If you want to get to know more about multimodality in social media, you can go to this blog post. In one of our other blog posts, the use of multimodality on the Maastricht University website is researched in greater detail. If you are interested in this, click here. Even this project about multimodality is actually multimodal! This project that you are currently reading one blog post of, consists of numerous blog posts and a podcast. Meaning this project makes us of modes like text, images and sound.
Multimodality is so inherently embedded in our digital society, which makes it even more relevant as a concept of academic research. Being aware of the modes that are being used and using them effectively is a great skill to obtain. This blog post provided the needed understanding of multimodality in order to see its relevance to make sure you, as a reader, are able to use it to your benefit in the future!
Disclaimer: This blog post is part of a series of different elements in regards to the concept of multimodality. This contains multiple blog posts discussing different aspects of this concept as well as a podcast tying all of this together in verbal form. To access the other elements of the series, take a look at the links below!
Bezemer, J. (2012, March 15). “What is Multimodality” [Video]. Youtube. https://www.youtube.com/watch?v=nt5wPIhhDDU
Bezemer, J. (2012, March 16). “Why Adopt a Multimodal Approach?” [Video]. Youtube. https://www.youtube.com/watch?v=rZ4rMVCWkQs
Serafini, F. (2014). Reading the Visual: An Introduction to Teaching Multimodal Literacy. New York, pp. 11-19.
Kress, G., & Jewitt, C. (2003). Multimodal literacy. New York: Peter Lang. P. 1–18.
Mills, K. & Unsworth, L. (2018). Multimodal literacy. Oxford Research Encyclopedia of Education.