camila valgas | media study + digital culture

Or Why we should advocate for multimodal journalism?

  Multimodality has been described as a field of research and practice that analyses the co-presence of different modes to pass a message. By doing that, multimodality helps people to enhance cognitive processes using diversified approaches. Thus, multimodality is the idea of making a composition with various modes (such as writing, video, photos, gifs, maps, and others) to communicate a message and express meaning. To do it, it is important to be aware that “signals from different modalities often carry complementary information about different aspects of an object, event, or activity of interest” (Liu et al., 2018). Thus, a particular mode expresses meaning in a distinctive way, as it produces a specific semiotic work. 

      Creating multimodal products has been considerably easier with the advancements of digital culture. These new literacies can be explored by diverse tools and platforms available for free on the web, which makes it possible for users and media professionals to create content and stimulate new compositions and layers of modalities. Technology is, indeed, generating layers and layers of information. This brings new challenges such as educating people to a plural notion of literacy that goes far beyond reading, writing, and calculating (Unesco, 2004). This plural notion of literacy, recognized by the United Nations as part of a global aspiration towards education for all, is connected with cultural identity, socio-economic progress, and human rights. 

Journalism and Conscientization   

Committed to this broader understanding of literacy, the pedagogue Paulo Freire has developed the concept of conscientization. This concept is described by him as a complex human process of consciousness about being a part of the world and about our role in it (Freire, 1998). Thus, engaging people to learn, think, and reflect creatively about the world would also encourage them to question the social reality they live in and to take action to improve it. Literacy is then a technical and social practice aiming to grasp and express the world’s reality in a creative language (Unesco, 2004; Freire, 1998).  Literacy wins a political dimension.

   Meditsch (1998) has then advocated for an application of Freire’s ideas into journalism. The scholar followed a course taught by Paulo Freire, at the time a visiting professor at the School of Communications and Arts at the University of Sao Paulo. He also had the opportunity to be received at Freire’s home to discuss his research about journalism as a field of knowledge production, one that “reveal aspects of reality that other modes of knowledge are not capable of revealing” (Meditsch, 1998, p. 21). To this extent, journalism would perform for the public the same functions that perception performs for individuals (Zelizer, 2005). By adopting Freire’s theoretical framework, Meditsch argues that journalism helps the individual to recover his awareness; which would be the first step towards conscientization. It would be a starting point headed for critical thinking and action aiming at social justice. 

Journalism and Multimodality 

     The recognition that a plural notion of literacy can play a life-changing role for individuals and communities, as well as the digital culture’s prominent position in our daily lives, has made journalism increasingly adopt multimodality in narratives. Several media organisations including the New York Times and the Guardian “have invested substantial time and financial resources in developing embedded multimedia stories, investment that is propelled by the belief that audiences crave higher value journalism and storytelling” (Fisher, 2015; Greenfield, 2012; Hernandez & Rue, 2015; Jula, 2014, as cited in Pincus, 2017, p. 749). To this extent, journalism is a central activity in democratic societies using multimodality to contribute to citizens’ understanding of the world surrounding them. 

       Journalists and media companies are adopting multimodality in new ways and in multiple spaces, such as printed newspapers, websites and social media platforms, which asks for knowledge about digital culture, design, photography, audio editing, filmmaking, and several other areas (Hiippala, 2017; Song, 2018). These efforts to improve the accessibility and the critical choice of literacies are also related to a theory called the wisdom of journalism (Stephens, 2014). It argues that the audience does not want only the news the faster as possible, they want to truly “grasp the meaning of what has happened” (Stephens, 2014, p. 83). Stephens (2014) argues that journalistic work needs to produce “pieces bold enough to do more than report the facts” (p. 88).

To understand a little more about these audience’ habits, I have conducted a survey research design with five questions about news consumption and social media. The survey was shared with students from the master’s degree Media Studies: Digital Cultures at Maastricht University, journalism’ students at University Federal of Santa Catarina; and graduate journalists in Brazil.

 

   This dataset has limitations due to its size, and also related to the choice of participants. As professionals engaged with digital culture or journalism, they might have different choices if compared with other niches of society. Nevertheless, it has permitted insights that corroborate with the conscientization‘ and the wisdom of journalism’ theories. It was possible to notice that more than a half (64%) of the participants check the news multiple times per day; 66.5% follow the news on social media, and only 1.8% totally disagree that to check the news on social media is not a good way to get informed. 

     In relation to multimodality and multimedia, an interesting finding was that 64% of the participants affirm to prefer to check the news in a mix of reading reports, watching videos and documentaries, listening to podcasts, and accessing posts on social media. All these different media formats carry uncountable different modes. Multimodality, as we can see, has a vital role in journalism. 

Tweets by voxdotcom

    Vox’s idea of producing digitally-mediated multimodal journalistic content was also adapted to a series called Explained in partnership with Netflix. Vox affirmed that this project considers that the traditional journalism based on the velocity of transmitting the news needs to be transformed into a format that sustains “deep reporting on the questions, forces, and ideas that rarely find themselves in the bright light of the daily news cycle” (Vox, 2018), what corroborates with the Wisdom of Journalism theory. They also declared to build their editorial process taking into consideration how to maintain the relevance of their content even after years they were launched. This is reflected in the choice of themes for the project. For instance, the two seasons available approach topics such as pandemics (launched before the COVID-19 spread), the gender wage gap, and the racial wage gap, showing long-term societal relevance. Watch the trailer:   

      Vox emphasizes the importance of visual journalism (such as computer graphics and moving images) in this project and in all others. This choice is motivated by the idea that these resources help to build new storytelling formats, and make it possible to transmit additional information, making ideas more understandable and interesting (Vox, 2018). It recognises that “the whole way that we communicate is changing. The previously segregated role of the different semiotic modes, of language and different visual elements such as colour, has changed” (Machin & Polzer, 2015, p. 16). These visual resources are utilised in all their video productions. The organisation also declared that they are used to publishing video stories before text stories, having their bigger audience base on YouTube with more than eight million subscribers. This goes against the idea that writing would be the most important literacy, showing that other modes such as audio, music, and images are also extremely relevant (Unesco, 2014). The use of different modes in their content helps to create literate environments, where people can absorb a new perception of reality and become aware of it. Journalism is then a practice of “communication to restore its collective transparency to reality” (Meditsch, 1998, p. 24). 

     According to the United Nations General Assembly, even though literate environments are not directly related to poverty reduction, gender equality, and other themes mentioned in the UN’s development goals, they are “mutually beneficial” (Unesco, 2004, p. 18). Thus, the use of literacies requires critical thinking. It is necessary to be aware that to deal with today’s digitally-mediated world asks for an iterative process of rethinking the accessibility and semiotic meaning of information, connecting it with social justice. According to Lage, “journalism descends from the oldest and simplest form of knowledge – only, now, projected on an industrial scale, organized in a system, using a fantastic technological apparatus” (as cited in MEdiTsch, 1992, p. 23). Thus, journalists need to face the use of multimodality as a political act. As Ratto (2011) defends, it is necessary to be aware that visualization is a metaphor that will emphasise what we choose to. 

     As this example above has shown, it is also essential to guarantee media literacy for all if we want to create enlightened societies. Citizens need to be trained to work with different modes, creating and distributing their own knowledge. It is necessary to give tools to people to understand the potential and the semiotic meaning behind each mode, creating a more critical audience for journalism products as well. The critical use of literacies and multimodalities by journalists need to be aligned to an educational process that teaches society about these multiple literacies and its semiotic meanings. In this way, we can construct a better use of multimodality as a political act, aiming for social justice for all.

---

      If you really found yourself in the concept of multimodality and would love to learn more about it, check the blogpost of my two colleagues, Rebecca and Eleni, which discuss respectively the use of multimodality on social media and in educational contexts. In addition, feel free to listen to our podcast episode about multimodality, giving emphasis to its historical background and possible applications.

References

Claire Clivaz and Martial Sankar (2016). Multimodal Literacies. DARIAH Teach. [Training module].

Fuchs, C. (2014). Social media and the public sphere. tripleC: Communication, Capitalism & Critique. Open Access Journal for a Global Sustainable Information Society.

Fuchs, C. (2017). Social media: A critical introduction. Sage.

Hiippala, T. (2017). The multimodality of digital longform journalism. Digital journalism.

Liu, K., Li, Y., Xu, N., & Natarajan, P. (2018). Learn to combine modalities in multimodal deep learning. arXiv preprint arXiv:1805.11730.

Machin, D., & Polzer, L. (2015). Visual journalism. Macmillan International Higher Education.

Meditsch, E. (1998). Jornalismo como forma de conhecimento. [Journalism as a way of knowledge]. Intercom-Revista Brasileira de Ciências da Comunicação.

MEdiTsch, E. (1992). O conhecimento do jornalismo.

Ratto, M. (2011). Critical making: Conceptual and material studies in technology and social life. The information society.

Song, Y. (2018). Multimedia news storytelling as digital literacies: An alternative paradigm for online journalism education. Journalism.

Sector, U. E. (2004). The plurality of literacy and its implications for policies and programs: Position paper. Paris: United National educational, scientific and cultural organization.

Stephens, M. (2014). Beyond news: The future of journalism. Columbia University Press.

Pincus, H., Wojcieszak, M., & Boomgarden, H. (2017). Do multimedia matter? Cognitive and affective effects of embedded multimedia journalism. Journalism & Mass Communication Quarterly.

Vox. (2018). Vox’s Netflix show “Explained,” explained. Vox Website.

Zelizer, B. (2005). Definitions of journalism. New York: Oxford University Press.

Source of cover photo.