The ability to speak and comprehend English in today’s globalizing economy is a bankable skill, and the demand for rapid and efficient English learning is high. Research in learning methodologies and neurological processing has led to the generally accepted idea that the brain has evolved to learn optimally through multi-sensory input; that is, information is presented in such a way that the learners interact with it using more than one of their senses, most commonly through a combination of auditory and visual channels. Using computer technology to deliver learning input allows a learner-centered environment and can be adapted to many different learning strategies. Use of technology in language learning is very common, and it can be combined with multi-sensory learning for an optimized learning experience.

Multi-sensory Learning

Multi-sensory learning is the processing of new information via more than one sensory channel, thus stimulating multiple areas in the brain. When people learn new information, such as a second language, the brain must process it in a way that the information is optimally retained.

Research indicates that the brain has evolved to operate optimally in multi-sensory environments, since the natural environment itself contains many simultaneous stimuli that we must perceive, process, and react to (e.g. Ghazanfar and Schroeder 2006). If a classroom exercise stimulates more than one of the learner’s senses at any given time, multiple channels of the brain are working to process the same information, and the brain is devoting more cognitive resources to that information. Most the most common senses used in learning are sight and hearing, the inputs for which are processed through an auditory/verbal channel and a visual/pictorial channel (Mayer and Moreno 2003). In addition, the information is not simply processed separately through the different channels, but the brain creates some interactivity between the two modes, essentially storing the information cross-referenced between the channels (Paivio 2006).

Previous theories held that only one sensory channel could be used at a time for optimal learning, otherwise it would result in cognitive overload and the learner would not be able to process more information. Research shows, however, that there is minimal performance decline when audio and visual tasks are performed simultaneously (e.g. Clark, Nguyen, and Sweller 2011). Multi-sensory processing may also enhance brain plasticity (Shams and Seitz 2008, Shams et al., 2011).

 

Multi-sensory strategy in language learning

In the language-learning field, much research has been done on how learners with learning difficulties can benefit from multi-sensory instruction to acquire basic language skills but also second languages (e.g. Birsh 2011). Multimedia has been used in language learning classrooms for decades—the latest available technologies are readily adopted, evolving from audio to video to computer to internet delivery (Davies 2011), supplementing or even replacing an instructor.

One aspect of multi-sensory learning in second language acquisition that has been repeatedly studied is the effect of video subtitles. With video media presented in classrooms, instructors sometimes have the option to use subtitles. Studies have shown that using captions (both subtitles and audio are in the target language) enable listening comprehension significantly more than solely audio or audio with subtitles in the native language. One such study (Baltova 1999) examined late high-school aged students learning French, comparing English audio-French subtitles with French audio and French subtitles and French audio with no subtitles. This study found that both French-subtitled conditions had high listening comprehension rates compared to the condition with no subtitles, whereas vocabulary acquisition and retention were higher in the French subtitle and audio condition compared with the other two conditions. Similarly, Guichon and McLornan (2008) found that multimodal input benefited French language comprehension for their group of undergraduates, and that subtitles in that target language were also beneficial for comprehension. Hayati and Mohmedi (2009) supported these findings in their study with English as the target language and Persian as the native language in a study conducted in Iran. Jubran (2012) also showed significant effects from multi-sensory learning of English for high-school age students in Jordan. Sydorenko (2010) studied a group of undergraduate beginning Russian students and found that captions aided in written recognition of words and learning word meaning, while the group that was shown the video without captions scored higher on aural recognition of words.

Computer technology in language learning

Computers have enabled learners to use language learning software and programs that contain activities and media that are commonly available in classrooms, such as exercises, assessments, and audio and video. These can be presented in an individualized and interactive environment via the computer. Processes that are important to language learning are easily facilitated through technology. Chapelle (2003) discusses two such processes related to salient input: marked input, that is, information that specifically draws the learner’s attention, such as highlighting words, as well as repetition of words, are easily managed in a technological environment, where supplementary information can be provided via hyperlinks or audio clips of the relevant words can be repeated.

Challenges of learning English as a foreign language

One major challenge with English being learned around the world is that each culture has its own approach to education: how a learner should acquire new information and what the definition of knowledge is, the nature of the social relationships within the learning environment (especially between the teacher and learner), and how the situation or time itself is perceived by the learner (Parrish and Linder-VanBerschot 2010). All of these factors influence the learner’s attitude and expectation of what learning is. The intersection of the cultural definition of learning and language learning can be found often in materials and instructor interaction. Depending on the intended audience of the course material, a learner may have no knowledge of some content references, such as cultural or local knowledge, or they might have difficulty understanding presented scenarios due to different cultural values, such as having different reactions to situations. Cultural learning differences can also have an impact on a learner if the instructor comes from a different learning background and is unfamiliar with the learner’s strategies (Liu 1999, Syed 2003). One educational factor that can make learning English particularly difficult is that in places where the language has status but is not commonly spoken, the grammar might be the primary educational focus rather than conversational proficiency, so that even if a learner performs well on acquiring the material, he or she cannot effectively communicate in English (e.g. Liu 1999).

While every learner is influenced by his or her cultural context, there are also individual factors that impact a learner’s study of English. Aptitude and motivation are two features that greatly affect a person’s capacity to acquire a foreign language. People have different aptitudes for language learning, which usually correlates with their development of their native language—if the first language was acquired relatively rapidly, the person will show an aptitude for learning a second language later in life (Skehan 2014). Skehan also explains that aptitude itself is comprised of several aspects, such as strengths and weaknesses amongst various capacities such as language analysis, memory, and phonemic coding, as well as the individual’s learning type, which can tend towards a more analytic or naturalistic view of language. Learners can also have different cognitive reactions to decontextualized learning situations as well, where people adjust differently to learning a language as a subject compared to learning it as a means of natural communication.

Learner motivation is another major factor in a person’s English learning performance that can present a challenge. Noels (2001) describes different types of learner motivation, including orientations where the learner takes pleasure and interest in learning the language (intrinsic motivation), those where learning the language is a means to accomplish some other goal (extrinsic motivation), and the orientation when the learner is not invested in learning the language, but continues to do so (e.g. it is a requirement for a student to graduate, though they would not be learning the language otherwise, known as amotivation). Other social factors that can affect motivation include the learner’s identification within their ethnolinguistic group, the cultural diversity within the local society, and the status of the target language within the learner’s culture (Kruidenier and Clement 1986). Learners with intrinsic or extrinsic orientations towards language acquisition will perform better in their study than learners who exhibit amotivation.

When acquiring a new language, adult learners typically have a harder time than children, a phenomenon known as the critical period hypothesis, where after a certain age (about 7-8 years old), a learner’s capacity to acquire a language to a native-like level decreases over time (e.g. Birdsong 1999). It is not impossible, due to the learner characteristics discussed above, such as aptitude and motivation, but studies show that adults do not acquire language features such as phonology, syntax, or grammar as well as children (e.g. Johnson and Newport 1989). Other common challenges include lack of access to learning materials, minimal time available to devote to language learning, or limited support resources, especially computers or internet.

One challenge with English in particular as a foreign language is that in many places around the world, English orthography is completely different from that of the local language. This means that in order to learn English, an entire new writing system must also be learned. Blomert and Froyen (2010) show that the letter-sound relationship in English orthography is arbitrary and therefore takes a long time to learn, and is also neurologically processed in a different way from other, more naturalistic audio-visual pairings. This process is, however, inherently multi-sensory.

Multi-sensory language learning technology as a solution

Multi-sensory learning has been shown to be effective for many different learners. Because it is a neurological process and therefore likely inherent in all humans, the process can operate independently from educational practices that vary across cultures. Evidence of this is discussed above, where learners from different cultural backgrounds and learning practices showed significant benefit from multi-sensory second language input. The studies mentioned also indicate that multi-sensory learning is effective for learners who are beyond the approximate age of the critical period of language acquisition, indicating that a multi-sensory approach is beneficial for all ages. While the multi-sensory learning process itself has little effect on a learner’s aptitude, motivation, or learning strategies, it can make English language acquisition more efficient, thus enabling those with less aptitude for language learning or those who may have some difficulty with maintaining motivation to learn English. For those whose acquisition of English also involves learning the alphabet, the fundamentally multi-sensory process of learning letter-sound association can be facilitated by further multi-sensory input.

Computer technology enables learners to access interactive and personalized platforms through which a variety of content and methodologies can be delivered and operate independent of the learner’s context. Because computers can be used in environments outside the local cultural learning environment, for example, at home instead of in the classroom, the learner can operate in a context that works well for them, and where their personal learning styles are emphasized over culturally-imposed practices. Computers are also easily accessible in the modern technology-driven world, from desktop units to smartphones and tablets, and internet connectivity is gradually becoming more universal and affordable. This gives language learners access and exposure to many different resources for language learning, including text, video, and interactive activities.

Garimella and Srinivasan (2014) report on a study of multi-sensory reading technology, ReadToMe, implemented in classrooms of government schools in India. The study covered students in classes 6, 7, and 8, in 100 schools (a total of about 20,000 students) across six states for one school year. The results show statistically significant gains in reading, reading comprehension, vocabulary, spelling, and grammar between the beginning and end of the school year; in some cases, the classes showed as much as a 20% improvement in the test score. This indicates that multi-sensory technology can be a beneficial learning method even in the face of limitations like poor classroom infrastructure, as is often found in Indian government schools, and short amounts of time spent learning English or using the software.

Combining multi-sensory learning with computer technology enables delivery of an effective learning process with the multimedia capability and accessibility of computer technology. It allows learners to facilitate their own language acquisition process in an individualized and efficient way, thus optimizing their language learning experience.

Erica Schramma

EnglishHelper Inc.

January 2016

References

Baltova, I. (1999). The effect of subtitled and staged video input on the learning and retention of content and vocabulary. Doctoral dissertation, University of Toronto.

Birdsong, D. (Ed.). (1999). Second language acquisition and the critical period hypothesis. Routledge.

Birsh, J. R. (2011). Multisensory teaching of basic language skills. Brookes Publishing Company.

Blomert, L., & Froyen, D. (2010). Multi-sensory learning and learning to read. International journal of psychophysiology, 77(3), 195-204.

Chapelle, C. (2003). English language learning and technology: Lectures on applied linguistics in the age of information and communication technology. (Vol. 7). John Benjamins Publishing.

Clark, R. C., Nguyen, F., & Sweller, J. (2011). Efficiency in learning: Evidence-based guidelines to manage cognitive load. John Wiley & Sons.

Davies, G. (2011) Introduction to multimedia CALL. Module 2.2 in Davies G. (ed.) Information and Communications Technology for Language Teachers (ICT4LT), Slough, Thames Valley University [Online]: http://www.ict4lt.org/en/en_mod2-2.htm

Garimella, S., & Srinivasan, V. EnglishHelper, Inc. (2014). A Large Scale Study of the Effectiveness of Multi-Sensory Learning Technology for Learning English as a Second Language. http://www.englishhelper.com/whitepaper.pdf.

Ghazanfar, A. A., & Schroeder, C. E. (2006). Is neocortex essentially multisensory?. Trends in cognitive sciences, 10(6), 278-285.

Guichon, N., & McLornan, S. (2008). The effects of multimodality on L2 learners: Implications for CALL resource design. System, Elsevier, 36 (1), pp.85-93. <hal-00356243>

Hayati, A., & Mohmedi, F. (2011). The effect of films with and without subtitles on listening comprehension of EFL learners. British Journal of Educational Technology, 42(1), 181-192.

Johnson, J. S., & Newport, E. L. (1989). Critical period effects in second language learning: The influence of maturational state on the acquisition of English as a second language. Cognitive psychology, 21(1), 60-99.

Jubran, S. (2012). Using multisensory approach for teaching English skills and its effect on students’ achievement at Jordanian schools. European Scientific Journal, 8(22).

Kruidenier, B. G., & Clément, R. (1986). The Effect of Context on the Composition and Role of Orientations in Second Language Acquisition.

Liu, D. (1999). Training non-native TESOL students: Challenges for TESOL teacher education in the West. Non-native educators in English language teaching, 197-210.

Mayer, R. E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational psychologist, 38(1), 43-52.

Noels, K. A. (2001). New orientations in language learning motivation: Towards a model of intrinsic, extrinsic, and integrative orientations and motivation. Motivation and second language acquisition, 23, 43-68.

Paivio, A. (2006). Dual Coding Theory and Education; Draft chapter for the conference on “Pathways to Literacy Achievement for High Poverty Children,” The University of Michigan School of Education, September 29-October 1, 2006. Recuperado el, 7.

Parrish, P., & Linder-VanBerschot, J. (2010). Cultural dimensions of learning: Addressing the challenges of multicultural instruction. The International Review of Research in Open and Distributed Learning, 11(2), 1-19.

Plass, J. L., & Jones, L. (2005). Multimedia learning in second language acquisition. The Cambridge handbook of multimedia learning, 467-488.

Posner, M.I. and Raichle, M.E. (1994). Images of Mind. 2nd Ed., New York: W H Freeman & Co.

Shams, L., & Seitz, A. R. (2008). Benefits of multisensory learning. Trends in cognitive sciences, 12(11), 411-417.

Shams, L., Wozny, D. R., Kim, R., & Seitz, A. (2011). Influences of multisensory experience on subsequent unisensory processing. Frontiers in psychology, 2.

Skehan, P. (2014). Individual Differences in Second Language Learning. Routledge.

Sydorenko, T. (2010). Modality of input and vocabulary acquisition. Language Learning & Technology, 14(2), 50-73.

Syed, Z. (2003). The sociocultural context of English language teaching in the Gulf. TESOL quarterly, 337-341.