back

A Study of the Impact of Multi-Sensory AI Technology Blended with Online Teacher Classes on the English Proficiency of Adult Learners

EnglishBolo™

EnglishBolo™ is an online spoken English program that blends technology-enabled self-learning with live, video-delivered teacher classes. The self-learning lessons are ‘micro-duration’ (15 minutes each) and teacher classes can be availed any time, any day making this a convenient, effective method for improving English proficiency and enhancing spoken English skill.

The program is now widely deployed across the country and was recently approved by All India Council for Technical Education (AICTE) as also suitable for college students across India.

Multi-sensory learning on EnglishBolo™

Studies from neuroscience suggest that multi-sensory structured learning education has significant potential to improve the systematic acquisition of reading skills. The brain creates specialized neural pathways for recognizing arbitrary cultural inventions such as letter-sound relationships. Multi-sensory stimulation enables the speedy creation of such a neural network. [1]

Learning English as a foreign language is particularly challenging for adults as the capacity to learn a new language to native-level proficiency decreases significantly with time (read age). 

Deploying technology in combination with multi-sensory learning can enable English language acquisition among such learners, facilitating their learning experience. [2]

Repeated multi-sensory exposure to words enables word recognition and facilitates the development of pronunciation / oral skills – the principle that guides the design of EnglishBolo™.

Methodology of Impact Assessment

Impact Assessment

EnglishBolo™ is a 100-day program. Learners complete 100 self-paced lessons (Learn Yourself Lessons or LYLs) and 10 sessions that are live, virtual and teacher-facilitated (Teacher Classes or TCs).

The program is adapted to learners’ proficiency. Before commencing the program, learners take a test (Placement Test) that establishes the proficiency and places them at a certain level of the program (Initial Level). Learners are offered the appropriate program based on the Initial Level at which they are placed.

After completion of the program (100 LYLs and 10 TCs), learners take another test to determine their level (Final Level) at the end of the program (Final Test).  The Placement and the Final Tests are equated; the Initial Levels are correspondingly equated with the Final Levels.

Movement from the Initial Level to the Final Level helps measurement of improvement in learners’ proficiency through the program.

Metrics of Impact & Engagement

The impact of the program is measured through the following parameters:

  • Percentage of learners who have completed at least 75% of the program (completion of 75% of the program is expected to increase learners’ proficiency as evinced from anecdotal evidence)
  • Percentage of learners who have completed the program
  • Percentage of learners who have remained at the same level but exhibited improvement in the score (measured as an increase in the percentage of correctly answered questions: ‘Score Improvement’)
  • Percentage of learners who have moved upward by at least one level after completion of the program

This document provides evidence of the impact of the EnglishBolo™ program.

Sampling

Measurement of improvement of learners’ proficiency through EnglishBolo™ is a continuous process. Additionally, periodic measurement on cohorts of learners is carried out to assess for sustained impact on different cross-sections of learners. Findings from this study, for instance, is consistent with cohort studies conducted earlier.

For this impact study, the cohort of learners enrolled in the EnglishBolo™ program from July 2020 to December 2020 was selected. A sample was drawn such that it ensured:

  • • Adequate representation of learners from different channels of acquisition (considering that channel of acquisition may influence learner engagement); and
  • • Learners have adequate opportunity to exhibit improvement (learners already at the highest levels in the Placement test being excluded from this examination).

A total of 1,320 learners satisfying the above criteria were sampled.

EnglishBolo™ Impact                         

Program Completion

Completion Rate Count %
Sampled learners 1,320
Completed >75% of Program 692 52%
Completed 100% of Program 609 46%

52% of EnglishBolo™ learners from the sampled cohort had completed more than 75% of the program; 46% (a subset of 52%) completed the entire program. This completion rate is much higher than the 5% to 15% rate that is typically reported for online learning classes [3-6].

The high completion rate is especially significant considering that engagement with and completion of the program are entirely discretionary actions, initiated and undertaken by learners of their own accord. The set of learners sampled is an unbiased, diverse group that has been subjected to little to no influence or coercion in a high-stakes setting.

Improvement

Improvement Total % Level 1* % Level 2* % Level 3* %
Improvement of at least 1 level 560 92% 166 98% 234 95% 160 83%
Improvement of score 28 5% 3 2% 10 4% 15 8%
No improvement 21 3% 1 ~0% 2 1% 18 9%
Total 609 170 246 193
* Level at Placement Test

97% of the learners exhibited improvement upon completion of the program; 92% showed improvement of at least 1 level, an additional 5% showed an improvement in score between the Placement Test and Final Test while staying at the same level. All but one learner in Level 1 (learners who started off with the lowest proficiency) have benefitted from the program.

21 (3%) learners did not exhibit improvement after the program. Closer examination revealed that these learners’ engagement and assessment responses were characterized by:

  • • Placement at a higher level
  • • High Placement Test scores
  • • Unwillingness to complete the Final Test / Abandonment of test in progress
  • • Frequent requests for extension of the program

The Voice of Learners

In addition to evidence of improvement in learners, learners mention having gained greater confidence in conversing in English after completing the EnglishBolo™ program. They find teacher classes particularly effective: they mention that teachers are empathetic and encouraging and are very detailed in correcting sentences and pronunciation. Overall, learners have found the program useful whether at work, with a customer or peer-to-peer interactions, or at school.

Presented below are verbatims of some learners who were part of the sample; the names of learners have been changed:

“This is the best app! There are some words which I thought I am pronouncing correctly but after the class, the teacher told me my mistake and corrected me. Whenever I made some mistakes in class, teachers corrected me and guided me very well.”

Veena, student of B.Pharm, Final Year

“I found the content in “Learn Yourself’ lessons very interesting. It helped in improving my vocabulary and pronunciation. The teachers were very supportive and I feel more fluent and confident after completing EnglishBolo.”

Meena, student of Computer Engineering, Final Year

“I found the program to be excellent and feel that the content is very useful and easy to understand. In teacher classes, teacher-guided all the students very nicely and encouraged us to make sentences on our own. I feel that my English speaking skills have drastically improved since I started the program.”

Sheela, found EnglishBolo through a search

Conclusion

In conclusion, EnglishBolo™ sees a higher-than-industry standards completion ratio. 97% of learners have improved in proficiency upon completion of the program. Thus, EnglishBolo™ program is impactful in improving learners’ proficiency as evidenced through assessments and from anecdotal sources.

Reference

[1] Mousavi, S.Y, Low, R, and Sweller, J, 1995, Reducing Cognitive Load by Mixing Auditory and Visual Presentation Modes. Journal of Educational Psychology, (87(2),317-334)

[2] Schramma, E., 2016, Multi-Sensory Learning and Technology in the Acquisition of English as a Second Language

[3] Jordan, K. , 2015, http://www.katyjordan.com/MOOCproject.html

[4] Lederman, D., 2019, https://www.insidehighered.com/digital-learning/article/2019/01/16/study-offers-data-show-moocs-didnt-achieve-their-goals

[5] Ahearn, A., 2019, https://www.edsurge.com/news/2019-06-06-moving-from-5-to-85-completion-rates-for-online-courses#:~:text=Industry%20reports%20and%20instructional%20designers,end%20up%20earning%20a%20certificate

[6] Basu, T., 2018 https://www.influencive.com/no-one-finishes-online-course-doesnt-matter/