The History of Multimodality：How people communicate and gain knowledge

“Multimodality is a theory which looks at the many different modes that people use to communicate with each other and to express themselves.”——Kress.G.

Introduction

Following the podcast : Multimodality, knowledge to go! Laila, Leah, Nina , and I recorded, I want to elaborate on one of our key topics, history of Multimodality.

If you are watching vedios on Youtube, posting your lives on Instagram or reading this blog right now, you are consuming and producing “multimodal” content. Multimodality refers to the diversity of communicative symbols in oral or written communication (Dai , 2013). Multimodality theory aims to study how people communicate and interact with each other in different modes（Jewitt & Kress ，2003). Modes, in this case, refer to the means of communication. Visual, spatial, aural, gestures, and linguistics are the five modes of communication (Jewitt, Bezemer, & O’Halloran, 2016). In other words, we can express meaning or communicate with each other not only by language, but also by other modes such as paintings, sculptures, music, photography, text, and so on (Serafini, 2014).

In this article, I will introduce the history of multimodality by studying cases which based on how people communicate and express meaning. In addition, I will explore the influence of multimodality’s development. Finally, I will draw a conclusion in the form of a personal reflection.

The history of multimodality

It is very easy to divide human history based on the sensibilities of the media that they had around them. Media is crucial to multimodality because it allows its users to use it along language for more effective communication. Notably, there have been several distinct eras that can be used to understand the progression of multimodality. The oral language era was the first (Boston University, 2017). According to Li(2020) oral language appeared about 300,000 years ago. At that time, people used simple language words with gestures or body language to communicate. It can be consider as multimodal ways. Oral expression was the dominant mode of communication. It was the immediate communication mode save for the people from the western culture who, between 1850 to 1980 C.E, thought that writing was the absolute communication means( Clivaz.C, Sanker.M ,2016). They had been convinced by the term literacy. Literacy was a term based on mass scholarization in the middle of the 19th century. This was a time where writing was hyped, and many people were seeking education, thus the mass scholarization. Literacy, therefore,  refers to skills that individuals develop to function in society, the primary ones being the ability to read and write(Serafini, 2014, p12). In this situation, writing was a trend. It was considered superior and powerful. It is important to mention the Print Era if you want to find the development of multimodality of this period. Just as Serafini (2014, p.11) argued that no one could imagined back then how easy it would become to produce anything multimodal in print era.

The Print Era ——15th century

In history, the print era is also known as the written meaning highlighted by Johannes Gutenberg’s invention of a printing press (Rees, 2005). The 15th-century invention by Gutenberg’s of the printing press enabled mass production of print media.It involves conveying meaning or communicating through a written language that is handwritten or on print pages. The use of words and phrases is formed in a sentence that organizes in linguistic grammar to create meaning. In this case, communication is dependent on personal interpretation. During the print period, oral communication, visuals, and gesturing were still available (Rees, 2005). However, they wanted a way to conserve the information. This facilitated the Print era.

In 1423, the first Woodcut printing on paper happened. It is the medium that was used to print religious images. It operated like a wooden stamp, from my understanding. Imagine a technique where images and texts are cut out of a wooden block; another block is level, the carved woodblock is inked and pressed against the other block (Rees, 2005). Incredible, right? Bear in mind, the ink was made of soot from oil lamps mixed with boiled linseed oil. Whoever came up with this idea, clearly necessity is the mother of inventions. In 1424, writing and printing are still laborious and reserved for the scribes. The parchment that was used for writing books was so expensive that books were never thrown away. Reuse and recycle was the motto. They would scrape off the text and write afresh. In 1430, the first block books were produced in Holland and Germany. In 1436, Gutenberg started working on a printing press that he completed in 1440 and set up a printing shop in 1448. He starts printing a first mass-printed Bible in 1455; 180 copies of the 42 line bible written in gothic text (Rees, 2005). Fast forward, several milestones in the print era occurred; color painting starts, books are printed with illustrations, roman type font is implemented, and by 1499, there were several print shops, and smaller books could be printed.

Another invention in 1810 by Friedrick Koenig to use the steam engine in printing further advanced the print media. Through these inventions, about 3000 pages would be printed per hour by the 1930s. As stated in the case study, different forms of paralanguage including sound, font size, and font layout could complement and reinforce the transmission of language. Print media illustrates a form of paralanguage that allowed the dissemination of information.

Multimodality in the 1960s

Even though it had not been named still, multimodality gained momentum in the 1960s. This was because of the academic and scientific attention to the subject.  This was in a bid to help account for the different ways communication could be expressed to create meaning. In this aspect, multimodality is defined as making meaning through the use of multiple means. During this period, the 1960s and 1970s, writers were engrossed in film, photography, and audiotape recordings seeking new composing ideas. Ideas for them to express themselves outside of oral communication as was the norm before. This gave way to expressionism, sensory illustrations of emotional experiences as described by Murray. Donald Murray, a journalist and English professor is linked to expressionist as his teaching methods to writers emphasized the importance of focusing on how we feel, what we see. all our sensory experiences, and expressing them in writing (Newkirk, 2008). By doing this, the writing was defined as a multisensory experience. Expressionists needed to compose this experience using the different modes to capture the experience.

The Digital Period ——1980s

It is argued that the printing age led to many revolutions that changed the world to what it is that we recognize today. However, one of the most dramatic eras in multimodality is the digital age . It was characterized by people engaging various forms of media like pictures, videos, audios, and text to supplement the spoken language (Boston University, 2017). The digital age, also known as the information age, started in the 1980s. Computers started penetrating the economy in the 1960s and 1970s. The introduction of personal computers in the 1980s and the internet in the 1990s drastically changed communication (Lohani, 2019, pp. 118-130). This led to a change in literacy as the generation could now communicate informally across multiple mediums with color, images, movement, and sound. People had cell phones and could text each other in shortened texts. This was a fundamental shift in multimodality as it represents the change of focus from print media to screen-based text. Digitalization resulted in the evolution of literacy. Students could now access information online as text, videos, or graphics to get informed .Although digitization has been accused of causing cultural transformation that has facilitated print media’s fall (Lohani, 2019, pp. 118-130), people exchange and share information on social networks at a fast and unprecedented way that could not have been anticipated in the printing age. Nowadays, people only need to present visual images, sound effects,  video clips and written text in the digital environment through the same basic code (digital bytes).

I will take dariah teach as an example in this part. Dariah is an useful tool in our MA study which is a multimodal website. It uses video, text, audio,images , quiz and other modes to educate. This multimodal education tool allows students to better master knowledge and can diversify their choice of learning methods.

Conclusion

Multimodality literacy can be viewed from historical ages to the modern period, where its application and implementation have transformed communication and education among other sectors. The technology tools can be applied in the multimodality era for multiliteracy systems. Multimodality creates an opportunity for easy expression and communication due to the availability of all the modes. This was especially important during the Covid-19 pandemic, where processes were moved online, including education and businesses. The ability to attend virtual classes, have audio, images and still communicate through text is through multimodality. However, one of the significant challenges of implementing multimodal writing is ensuring that students have interpretation skills for the concepts portrayed. While reading and writing can be taught, interpretation is based on one’s perception. In education, it would be essential if new media could be used to enhance alphabetic literacy. Multimodality should be treated as a way of stretching writing and reading to addressing communication needs.

References

Boston University. (2017). Printing Press, Digital Age, and Social Movements. College of Communication. https://sites.bu.edu/cmcs/2017/11/16/printing- press-digital-age-and- social-movements/

Breaking News: Renaissance Journalism and the Birth of the Newspaperhttps://folgerpedia.folger.edu/Breaking_News:_Renaissance_Journalism_and_the_Birth_of_the_Newspaper

Claire Clivaz and Martial Sankar (2016). Multimodal Literacies. DARIAH Teach. [Training module]. https://teach.dariah.eu/course/view.php?id=24§ion=0 

Dai Sl.(2013) The Origin and Development of Multimodal Discourse Analysis(Jiangsu University of Technology ，Changzhou 213001, China)

Jewitt, C., Bezemer, J., & O’Halloran, K. (2016). Introducing multimodality. Routledge.

Jewitt, C., Bezemer, J., O’Halloran, K. (2016). Introducing Multimodality. Routledge.

Kress, G., & Jewitt, C. (2003). Introduction. Multimodal literacy (pp. 1–18). New York

 Li Bin(2020) “Language Quest”. Journal of the School of Chinese Language and Literature of Nanjing Normal University (04), 189.doi:CNKI:SUN:NJSD.0.2020-04-026.

Lohani, S. (2019). The history of multimodal composition, its implementation, and challenges. The Criterion: An International Journal in English, 10(1), 118-130.

Newkirk, T. (2008). Donald Murray and the” Other Self”. Writing on the Edge, 19(1), 47-52.

Rees, F. (2005). Johannes Gutenberg: Inventor of the printing press. Capstone.

Serafini, F. (2014). Reading the visual: An introduction to teaching multimodal literacy. Teachers College Press.

More about Multimodality by my teammate

Multimodal Literacy: How we make meaning of the world

Multimodality: an introduction