A Case Study

Background

The Government of Andhra Pradesh plans to introduce English as the medium of instruction in all government schools.  With this objective in mind, the Government has introduced ‘Anglodayam’, an initiative to implement EnglishHelper’s multi-sensory AI reading & comprehension solution, ReadToMe^®. EnglishHelper and SchoolNet are implementing ReadToMe^® Solutions in residential Gurukulam schools governed by the Tribal Welfare and Social Welfare departments of the Government.

The project has been jointly sponsored by Amazon Web Services and Intel India. This paper details the solution framework deployed using the Tribal Welfare schools as a case study.

About the Initiative

a. ReadToMe School Edition is deployed in 149 Tribal Welfare schools and 61 Social Welfare schools.

b. ReadToMe® Solutions are being implemented for students of Grades 5, 6, 7 & 8.

c. More than 40,500 students benefit from the program.

d. EnglishBolo™, a 100-day English improvement program, is being used by up to 3 teachers at each school.

e. ReadToMe Student Edition is available to all students with access to Android devices.

Solution Framework

EnglishHelper has deployed the following Solution framework to complete the LearningLoop™. The deployed solutions enable learning in the classroom and self-learning beyond the classroom.

1. ReadToMe School Edition is deployed in class to enable reading of the prescribed English Textbooks.
2. ReadToMe Student Edition Android app enables students to practice reading & comprehending their English textbooks on their own.
3. Teachers give homework to students on ReadToMe Student Edition, promoting engagement and self-learning.
4. ReadToMe School Edition in class also helps teachers improve English reading & comprehension skills.
5. EnglishBolo™ further enhances teachers’ English speaking skills.

ReadToMe^®

ReadToMe^® is an easy-to-use multi-sensory AI reading and comprehension software. The software has various features like Dictionary, Picture Dictionary, Translation, Pronunciation, Syllabification and Spelling that make learning easy and fun. By training itself on students’ textbooks, ReadToMe^® adapts seamlessly into the school curriculum. ReadToMe^® is available in various learning environments to support the learning needs of students across multiple use cases.

EnglishBolo™

EnglishBolo^™ is a 100-day program. Learners complete 100 self-paced lessons (Learn Yourself Lessons or LYLs) and 10 sessions that are online, live and trainer-facilitated.

The program is adapted to learners’ proficiency. Before commencing the program, learners take a test (Placement Test) that establishes their proficiency and places them at a certain level of the program (Initial Level). Learners are offered the appropriate program based on the Initial Level at which they are placed.

After completion of the program, learners take another test to determine their level (Final Level) at the end of the program (Final Test).

Students’ Proficiency

The Annual Status of Education Reports (ASER) reveal that:

a. 79% of students in Grade 3 & 40% of students in Grade 5 in Andhra Pradesh cannot read Grade 2-level text.^[i]

b. 44% of students in Grade 3 cannot read even simple words in English.^[ii]

As part of the project, Baseline assessments were conducted among a sample of the Tribal Welfare schools. From the tests, we learned that:

a. Across grades 5 to 8, students were generally able to answer only 1 in 3 questions correctly.

b. Students have difficulty in answering questions assessing their Reading and Comprehension ability.

c. The Common European Framework of Reference for Languages (CEFR)^[iii] levels of students were significantly short of the required proficiency. (While students of grades 5 to 8 are required to be in the proficiency range of A2-B2 to successfully read and comprehend their textbooks, nearly 90% of the students assessed across grades are still at CEFR level A1, lower than the A2 requirement for Grade 5.)

Teachers’ Proficiency

Proficiency Requirement of Teachers

The Government intends to implement the CBSE system in Government schools. The Andhra Pradesh Teacher Eligibility Test (APTET) requires teachers to be proficient in Grade 10-level English.^[iv] The syllabus and testing criteria correspond to the CEFR level C1.

Proficiency Level of Textbooks

EnglishHelper examined the textbooks that are prescribed by the Andhra Pradesh SCERT as well as by the NCERT, with respect to their CEFR levels. The CEFR enables a standardized comparison of elements across grades in the form of Levels. CEFR levels A (A1 and A2) define the abilities of a Basic User of the language, B (B1 and B2) define the abilities of an Independent User of the language and levels C (C1 and C2) define the abilities of a Proficient User of the language. The CEFR defines specific competencies at each of these levels in the form of “can-do” statements.^[v]

The textbooks of Grade 6 are at CEFR level B1 with more than 1,000 words at level B1 or higher. The elements of sentence structure, prose and grammar are also at level B1 or higher.

The textbooks of Grade 10 (which is the level of proficiency required for teachers) are at CEFR level C1 with more than 500 words at level C1 or higher. In these textbooks too, the elements of sentence structure, prose and grammar are at level C1 or higher. 

The ‘Can do’ descriptors for CEFR level C1 are:

1. Can understand a wide range of demanding, longer texts, and recognize implicit meaning.
2. Can express him/herself fluently and spontaneously without much obvious searching for expressions.
3. Can use language flexibly and effectively for social, academic and professional purposes.
4. Can produce clear, well-structured, detailed text on complex subjects, showing controlled use of organizational patterns, connectors and cohesive devices.

As part of the EnglishBolo^™ program offered, 388 teachers from the Tribal Welfare schools undertook a Placement Test to determine their CEFR level at the start of the program. The Placement levels of the teachers and their corresponding CEFR levels and Can do descriptors are presented in the following figures.

CEFR LEVEL	A1	A1 and some A2	A2 and some B1	B1 and some B2
Can Do DESCRIPTOR	Can recognize the first letter from a familiar picture, missing letters from a familiar word, and repeated letters. Can complete sentences meaningfully by recognizing missing words. Can identify the name of a given picture. Can identify synonyms and antonyms of familiar words. Can identify contrasting words. Can retrieve explicitly stated information from a familiar text.	Can understand and use familiar everyday expressions and very basic phrases aimed at the satisfaction of needs of a concrete type. Can introduce him/herself and others and can ask and answer questions about personal details such as where he/she lives, people he/she knows and things he/she has. Can interact in a simple way provided the other person talks slowly and clearly and is prepared to help.	Can understand sentences and frequently used expressions related to areas of most immediate relevance (e.g. very basic personal and family information, shopping, local geography, employment). Can communicate in simple and routine tasks requiring a simple and direct exchange of information on familiar and routine matters. Can describe in simple terms aspects of his/her background, immediate environment and matters in areas of immediate need.	Can understand the main points of clear standard input on familiar matters regularly encountered in work, school, leisure, etc. Can deal with most situations likely to arise whilst travelling in an area where the language is spoken. Can produce simple connected text on topics that are familiar or of personal interest. Can describe experiences and events, dreams, hopes & ambitions and briefly give reasons and explanations for opinions and plans.

77.42% of the teachers are at an English proficiency that is lower than the requirement for teachers at Andhra Pradesh schools; 53.37% are inadequately prepared for the English medium of instruction being at the lower Basic level of CEFR proficiency.

Structured telephonic conversational evaluations with 237 teachers revealed that only 9% of the teachers were able to speak English fluently and confidently. Most of these were English teachers, Science teachers or Principals. Of the 71 English teachers interviewed, only 14% (10) could speak English fluently. This situation calls for immediate remediation to prepare teachers for the English medium of instruction.

Engagement with EnglishBolo™

Of the 388 teachers, 65 teachers (representing 52 schools) teach English and the remaining 323 teachers teach subjects other than English, as elicited through telephonic conversations with them.

Completion rates of EnglishBolo^™ among the teachers were significantly higher than industry averages with 71% of English teachers and 51% of the teachers teaching other subjects completing the program. These are remarkably high considering that the typical completion rates of Massive Open Online Courses (MOOCs) range from 0.7% to 52.1%, averaging at 12.6% ^[vi].

Thus, English teachers demonstrate significantly higher engagement with the program, compared to other subject teachers.

Teachers’ Distribution by Subject taught and Completion of EnglishBolo^™:

Subject Taught	Number of Teachers	Teachers completing 100% of EnglishBoloTM	Teachers completing 75% or more of EnglishBoloTM
English	65	46 (71%)	52 (80%)
Other than English	323	165 (51%)	172 (53%)
Total	388	211 (54%)	224 (58%)

Distribution of Schools based on Whether English Teachers participated in EnglishBolo^™:

Subject taught	Number of schools	At least one Teacher Completed 100% of EnglishBoloTM	At least one Teacher Completed 75% or more of EnglishBoloTM
English Teacher underwent EnglishBoloTM	52	34*	43
Only other Subject Teacher underwent EnglishBoloTM	87	64	70
All Teachers	139	105	118

Impact of EnglishBolo™ Engagement on ReadToMe^® Usage in class

We examined the average (mean) number of ReadToMe^® classes per school taken by teachers who underwent EnglishBolo^™. The classes were conducted from August 2021 to January 2022 through a period of intermittent school closure due to the COVID-19 pandemic and scheduled holidays. 

We then looked for trends in the average as they would relate to the following:

1. Does the average number of classes that teachers conduct with ReadToMe^® differ based on whether teachers complete EnglishBolo^™?

2. Does it differ based on whether English teachers undergo the program?

3. Does it differ further based on whether English teachers complete the program?

To identify a trend, we compared each of these averages with the average number of classes when all teachers undergo EnglishBolo^™.

In the graph below, the light area indicates the number of schools falling in each of these examination sets. This establishes that there is a sizeable base for meaningful comparison. The bars indicate the average number of classes per school conducted using ReadToMe^® in the period under examination.

From the graph, it is evident that usage of ReadToMe^® in class increases with increased engagement with EnglishBolo^™. Additionally, usage is remarkably higher when an English teacher undergoes EnglishBolo^™.

Evidence suggests that when ReadToMe^® is used regularly in class, students’ learning outcomes increase significantly.^[vii]

There is also evidence indicating that when students learn with ReadToMe^® in class, they exhibit greater engagement with the self-learning app at home.^[viii]

Impact of EnglishBolo™ on Teachers’ Proficiency

An examination of change in level between Placement and Final Tests reveals that 94% of the teachers completing EnglishBolo™ have exhibited improvement of at least one level. Of the remaining teachers, all but one exhibited improvement in scores. This is in line with studies conducted on the impact of EnglishBolo™ in learners’ proficiency.^[ix]

Key Takeaways

a. Students are at low English proficiency and require immediate intervention – 90% of students are below desired proficiency.

b. Teachers’ proficiency in English is low impacting their readiness to transform English learning in school.

c. Teachers exhibit high engagement with EnglishBolo™. Specifically, English teachers exhibit significantly higher engagement.

d. When English teachers use EnglishBolo™, they have a greater propensity to teach with ReadToMe^® in class.

e. Teachers’ English proficiency improves significantly after completing EnglishBolo™.

f. There is evidence that the use of ReadToMe^® in class and at home by students completes the LearningLoop™ leading to enhanced learning outcomes.

References

[i] Annual Status of Education Report 2018: http://img.asercentre.org/docs/ASER%202018/Release%20Material/aserreport2018.pdf

[ii] Annual Status of Education Report 2016: http://img.asercentre.org/docs/Publications/ASER%20Reports/ASER%202016/aser_2016.pdf

[iii] Common European Framework of Reference for Languages (CEFR): https://www.coe.int/en/web/common-european-framework-reference-languages/level-descriptions

[iv] Government of Andhra Pradesh, Department of School Education 2021:  https://aptet.apcfss.in/Documents/APTET%202021%20Syllabus.pdf

[v] Common Reference Levels: https://www.coe.int/en/web/common-european-framework-reference-languages/table-1-cefr-3.3-common-reference-levels-global-scale

[vi] A Study of Engagement Rates of Learners in EnglishBolo™, July 2021: https://www.englishhelper.com/resources/research-center/2021/07/16/a-study-of-engagement-rates-of-learners-in-englishbolo/

[vii]  Improving reading and comprehension in K-12: Evidence from a large-scale AI technology intervention in India, May 2021: https://www.sciencedirect.com/science/article/pii/S2666920X21000138

[viii] Teachers Influence Students’ Engagement with ReadToMe Student Edition – an Initial Study, September 15, 2021: https://www.englishhelper.com/resources/research-center/2021/09/15/teachers-influence-students-engagement-with-readtome-student-edition-an-initial-study/

[ix  A Study of the Impact of Multi-Sensory AI Technology Blended with Online Teacher Classes on the English Proficiency of Adult Learners, January 11, 2021: https://www.englishhelper.com/resources/research-center/2021/01/11/a-study-of-the-impact-of-multi-sensory-ai-technology-blended-with-online-teacher-classes-on-the-english-proficiency-of-adult-learners/