-Multi? Modal? Modes? Multim..what? 

Don’t worry, Camila, Rebecca, and I have the answer for you. Just click on the podcast episode below, and I am sure you will get a better understanding of multimodality. We will not only provide a short definition but also describe its historical overview, as well as a couple of everyday examples.

Now that the term Multimodality is clear, let me show you how multimodality benefits educational processes, especially language learning, as well as how the use of  technology enhances the learning experience. Yes, I am talking about Duolingo!

As the world consists of “visual images, design elements as well as written language”, one should learn how to “read” between the lines as well as extract meaning out of images, in order to shape and improve their identity (Serafini, 2014, p. 3). Multimodality is the term that refers to “texts that utilise a variety of modes to communicate or represent concepts and information” (Serafini, 2014, p. 12). Hence, meaning may not only be conveyed through speaking, but also through different modes, i.e. sound, image, gestures, or music (Serafini, 2014, p. 13). A text consisting of various modes is referred to as a multimodal ensemble, or multimodal text (p. 12). The rise of technology in modern society and the powerful presence of contemporary mass media have paved the way to the distribution of multimodal texts in digital environments thus increasing the importance of understanding the role of texts, images, videos, and their combination (Serafini, 2014, p. 3).  

One of the fields in which multimodality is highly used and which closely involves the interplay between modes and meaning-making is education. An example of using multimodality in education is the process of foreign language learning, where it can easily become evident that the use of multiple modes assist in the acquisition of different language-related aspects (Gilakjani, Ismail, & Ahmadi, 2011, p. 1325). 

Taking into account the fast-changing digital society, one would argue that learning can be achieved through different ways of using technology, for example, platforms and applications to assist and enhance the learning experience since the digital advancements have enhanced the use of multimodality. There are plenty of language learning software tools available for users to experiment with and practice a new language, such as Duolingo, Rosetta Stone, etc. In this essay, I am going to present multimodality in education and guide you through the use of Duolingo as an example of language learning software, which makes use of different multimodal tools to engage learners and improve learning quality.

Multimodal Education

Educational materials have long been “dominated by the mode of written language” (Serafini, 204, p. 17). However, given the increased use of digital technologies in contemporary social life and modern communication, the heavily written-based texts in education might pose a problem for students’ ability to understand visual texts as they grow older. It might, therefore, be important to teach students how to read visual texts. Multiliteracy education has, thus, been linked to the interrelationship between written texts and images found mainly in children’s picture books, educational textbooks and Internet sites (Duncum, 2004, p. 260). 

Different modalities convey different information as a resource for students’ learning and meaning-making (Jewitt, Kress, Ogborn, & Tsatsarelis, 2001, p. 1). As learning “requires the interplay among multiple sensory modalities and representations” (Gilakjani et al., 2011, p. 1325),  gestures, use of space, images, and speech draw students’ attention and can be fully representational in conveying the meaning in the classroom (Jewitt et al., 2001, p. 6). Therefore, for learners to be part of a successful holistic learning experience, they need to connect the senses used in everyday life to the educational material, “to be situated” (Gilakjani et al., 2011, p. 1325). The learners’ use of more than one sense or modality is thus called “multimodal learning” (p. 1325). 

Technology-assisted learning and multimodality

So how exactly does technology enhance multimodal practices in language learning processes?

With modern digital advancements, multimodal texts shifted from print-based ensembles to more complex, digital-based ensembles, providing the opportunity to include a larger amount and variety of modes, including “sound effects, moving images and other digitally rendered elements” (Serafini, 2014, p. 13). Those features, together with the ubiquity of digital devices, allow users to enjoy a plethora of learning resources, create and study various multimodal ensembles, and achieve a more encompassed learning experience (Eisenlauer, 2014, p. 329). The portability and connectivity these devices offer, as well as the multi-touch interface are linked to “emergent multimodal literacy practices” (Eisenlauer, 2014, p. 329). This has led to the emergence of multimedia in learning, which offers, according to Jaskova, an “illustrative approach” (2014, p. 13). Moreover, users are provided with animations, sounds, videos, “charts”, “hypertext links to additional sources of information” and more elements that facilitate the learning process; thus the user is rendered an active rather than a passive learner and is exposed to rich multi-sensory learning resources (Jaskova, 2014, p. 13).

Foreign Language Learning

Studies in second-language learning have indicated that the use of non-verbal resources such as “verbal contexts and imagery” in combination with the target (foreign) language can positively contribute to the faster comprehension and acquisition (Sadoski, 2005, p. 234). 

The use of multimodality through mobile devices may even add to the language instruction methods. It is a fact that more and more people make use of their smartphones, tablets, and computers to learn a new language. According to Kalz, Bayyurt, and Sprecht (2014), mobile devices contribute to an enhanced image-based vocabulary learning, as they provide a range of opportunities to upload, connect, and generate photos that relate to certain “verbal data” or “the respective entries of the mobile vocabulary learning environment” (p. 337). Language is a “social phenomenon” (Gilakjani et al., 2011), therefore it seems important to construct contextual environments in which the acquisition and the practice can take place (p. 1325). This can be achieved with the combination of different modes such as “animation (visuals and movement) and narration” in a variety of contexts, by means of multimedia (Gilakjani et al., p. 1325).

Duolingo

A prominent example of multimodal learning in the mobile and e-learning era is Duolingo. This application combines a variety of modes in a single platform, in order to interact with the audience and facilitate knowledge acquisition.

Duolingo is one of the most popular language-learning platforms/apps worldwide. It was developed to help translate different websites, but it is now counting more than 300 million users who learn a foreign language through the platform (Duolingo, n.d., para. 2). Its aim is to make education feel like a game which is fun and free for all its users. It offers more than 30 languages that can be learned through quick, multimodal lessons that contain different kinds of tasks and allow the users to practice receptive and productive oral, aural, reading and writing skills (Duolingo, n.d.).

Unlike traditional learning methods, it promotes independent, individual learning and “encourages the users to review what they have learned as well as relearning what they have forgotten” (Ahmed, 2016, p. 257). The gamelike approach makes it fun and effective (Jaskova, 2014, p. 16); users can keep track of their process and gain points and rewards, according to their performance throughout the different levels.

Let us see which are some of the multimodal features of Duolingo:

Duo

Duo, the small Duolingo owl avatar is there to introduce learners to the platform and guide them through the app’s important functions. Duo can adopt the roles of different “personas in the real-world classroom” according to the context. In this way, it helps create a personalized learning process and evoke a sense of closeness for the learners to engage with the programme as Falk, Götz, Zeyer, Stuhlmann, and Jones state (2016, p. 245). For example, Duo appears as a sports coach when it “enables learners to achieve their self-set goals in the learning process”, as a “university graduate” by explaining a feature of the target language which may be important or hard to learn, or appears with a scarf of a language resembling a trophy “which cheers” when learners have passed a level (Falk et al., p. 246).

Finally, Duo sends users daily reminders to devote some time to practice via “push notifications or emails”, which according to Falk et al. (2016) help some of the users to continue learning (246).

Colours, impersonation, and roles are a combination of visual, aural, textual, and interaction modes used to keep the user motivated and enhance the learning experience. Duo, however, is only one of the various multimodal elements one could find while learning through Duolingo.

Multimodal Lessons

Throughout the learning process users are presented with tasks, which are collections of activities focused on different aspects of the language, in a structured way. Tasks involve text-image-sound combinations and are divided into thematic areas, for example, family, places, travel, etc. (Duolingo, 2020). While solving these tasks, every time the user hovers over an image or a word, they can hear the word pronounced. There are a number of activities in Duolingo:

• “word translation”: users are called to translate a word or phrase into a template form accompanied with a set of pictures or (part of) their meanings.
• “word matching” : The users have to match the word in their language to one of the four words in the foreign language, each of which is accompanied by a figure, illustrating its meaning.

Both tasks involve meaning related images aiming to “stimulate learning” as users get to create semantic connections which help “enhance the retention of new words” (Eisenlauer, 2014, p. 333). 

• “type what you hear”: Users listen to an audio of a sentence in the foreign language and they have to transcribe it. A button with a turtle repeats the sentence slowly as many times as needed.
• “speak this sentence”: Users have to listen to a word or phrase and then repeat it verbally. A set of options are also presented as possible translations.
• “how do you say…”: Here, users are presented with a word in their native language, which they have to translate by selecting the correct among three options given in the foreign language.

In this combination of aural, oral, and written elements, Duolingo manages to engage both auditory and visual learners and helps them acquire and memorize the dictation, the vocabulary, and the pronunciation of the words.

It is evident that Duolingo’s activities provide a range of audio, visual, and linguistic modes that appeal to most learning styles. The combination of aural cues with the words, the images and the production of texts highlight the multimodal nature of its activities and give an alternative sense of practicing a language, compared to the older text-based traditional ways. 

Finally, let us explore the game-like features in Duolingo that help create a fun, relaxing, and enhanced learning environment.

Game-like features

• First of all, users need “hearts” to start new lessons. A heart mimics a common feature in many games representing “life points”. In Duolingo, they correspond to the number of mistakes the user can make.
• Every time the users complete a task and reach their pre-set goal, they can gain a reward and the levels of each section become complete. The “Wildfire”, the “Sage” and the “Scholar” are only three of the rewards one can claim, earning in this way the so-called gems. 
• Gems can be used for various app features, such as a heart refill, outfits for Duo, and Bonus Skills (e.g. idioms).
• Feedback to the user is also given as part of the game-like process. A green colored message with a distinctive “beep” tone appears every time an answer is correct, whereas a red note with the correct answer is shown after a mistake is made. 

As can be seen, the features above together with color-coding convey information about the progress of learners, which otherwise could be conveyed in text form. All messages are communicated in an interactive manner and attempt to immerse learners in an environment of knowledge which is perceived as a simple game.

Conclusion

Multimodality proves a useful approach on language learning and the technological advancements are quite promising in helping enhance the quality of such a process. Duolingo is an effective multimodal tool that can appeal to different kinds of learners and encompass a multisensory learning experience. Of course, this online platform can not substitute teachers and the actual learning environment, but it can provide learners with the opportunity to practise a language, exercise listening and translation, as well as vocabulary and pronunciation.

The multimodal elements of Duolingo that aim at user engagement do not stop when exiting the app; Duolingo uses a variety of multimodal features to keep users on track through social media, such as Instagram. Voting, questions, polls and giveaways through Instagram stories and posts enhance the multimodal approach and interaction with the learners. 

Have you ever learned a language through Duolingo?

For further information about multimodality on Instagram as well as in Journalism, feel free to check out Rebecca’s and Camila’s blog posts below!

Tags: designthinkingduolingomultimodality