Early Trends: Students’ Engagement with ReadToMe Student Edition
Homework as an Effective Nudge of Students’ Engagement with ReadToMe Student Edition
Background
EnglishHelper is deploying ReadToMe® Solutions in all the government schools of Maharashtra and Punjab. ReadToMe® is a multi-sensory AI reading and comprehension software. ReadToMe® is deployed in class to enable reading of the prescribed English textbooks. It is also available as an Android App (ReadToMe Student Edition) and enables students to practice reading and comprehension, outside the classroom.
Initial studies have indicated that teachers play a significant role in influencing students to revise at home using ReadToMe Student Edition, thus promoting self-learning.[i],[ii] Informed by this evidence, EnglishHelper has prepared pre-packaged homework that empowers teachers to share with their students enabling revision and self-learning using ReadToMe Student Edition.
Principles of Homework Preparation
Homework is a proven tool that helps students work independently and encourages self-learning. Studies indicate that homework has a positive impact on student learning outcomes. However, for homework to be effective, it is important to be matched to students‘ abilities, require the right quantum of effort, and encourage the student with support for completing the homework and independent learning.[iii]
In the context of English language acquisition, it is important to be cognizant that students are most likely to be at lower than grade level proficiency. The pre-packaged homework is designed by taking this into consideration. Students have access to all comprehension tools available on ReadToMe Student Edition by way of learning support. These factors enhance students’ compliance with the task of homework completion.
In preparing pre-packaged homework that teachers can readily distribute to their students, the following principles are followed:
1. Homework is developed for each chapter of the student’s textbook. Multiple sets of homework are created for each chapter and each grade.
2. The homework is based on the principle of increasing CEFR proficiency. Successive homework from the same chapter is designed to build higher proficiency [CEFR] of students.
3. The homework given is aligned with the curricular transaction in class in every week. This was done by continuous collection and validation of information on the chapter being taught in class each week in every grade.
4. The homework once given to students, nudges them to use the reading and comprehension tools available on ReadToMe® for fully understanding the chapter. In addition, specific words (‘homework words’) are shared. Students are expected to look up and de-code the words using the various comprehension tools available on the app. To aid learning, students are encouraged to write the meanings of the words in their notebooks.
5. The homework is designed as a ‘ready-to-forward’ WhatsApp message.
6. Teachers are encouraged to validate the homework done, either through a return WhatsApp image by the student, or in the subsequent classroom session.
Presented below are two examples of pre-packaged homework.
Homework Delivery
In the early months of piloting the homework delivery program, EnglishHelper sent group WhatsApp messages to block / district education officials and teachers who further cascaded the messages to ultimately reach the students. The homework was sent weekly. It was essential to follow this multi-step process to obtain feedback on the effectiveness of the messages, as well as obtain validation of alignment with the curricular calendar.
Subsequently, seamless systems of homework delivery have been developed, which will be explained towards the end of this paper.
Impact of Receiving Homework on Student Engagement
We report the impact of the homework based on activity observed in the months of January and February 2022. During these months, schools had re-opened after the pandemic but with intermittent closures and up to 50% attendance, when open.
We examined the activity of students from Grades 6 to 10, in both Maharashtra and Punjab. We evaluated the following activities around the day when homework was expected to be cascaded by the teacher to students:
1. Activity on the homework words
2. Time spent on ReadToMe Student Edition by students
3. Number of students spending time on ReadToMe Student Edition
It was observed that the frequency of reading ‘homework words’ peaked within two days of EnglishHelper initiating the WhatsApp forward. Additionally, higher numbers of students were seen to spend greater amounts of time on ReadToMe Student Edition, around the period of the WhatsApp forward / receipt of homework by students.
This observation is consistent across grades and for both states examined; when students receive homework from teachers, they are likely to engage more with ReadToMe Student Edition.
A trend analysis of activity on homework words, time spent on ReadToMe Student Edition and the number of students engaging, is presented in the Appendix, delineated by grade and state.
In addition, a statistical comparison of time spent by students (in minutes) within two days of the homework forward and after two days of homework forward revealed that students spent a significantly higher amount of time on ReadToMe Student Edition, after receiving homework, as shown below. These observations hold true at 80% confidence levels.
Null Hypothesis: There is no impact of Homework on user engagement.
Alternative Hypothesis: There is an impact of Homework on user engagement.
For Maharashtra,
Alpha = 0.18
t-Test: Two-Sample Assuming Equal Variances
| Variable 1 (Time spent by student within two days of homework forward) | Variable 2 (Time spent by student after two days of homework forward) | |
| Mean time spent | 1693.031 | 1407.78 |
| Variance | 2647028 | 1355515 |
| Observations | 75 | 100 |
| Pooled Variance | 1907954 | |
| Hypothesized Mean Difference | 0 | |
| df | 173 | |
| t Stat | 1.351931 | |
| P(T<=t) one-tail | 0.089081 | |
| t Critical one-tail | 0.917803 | |
| P(T<=t) two-tail | 0.178162 | |
| t Critical two-tail | 1.346198 |
We now compare the calculated value of t with the tabulated value of a certain level of significance (.18). If calculated |t|> tabulated t, the null hypothesis is rejected.
Tabulated t = 1.3462,
For the above table, we can see that:
calculated |t| > tabulated t. Hence, we reject the null hypothesis for a level of significance (0.18).
That means, there is an impact of homework on user engagement.
For Punjab,
Alpha = 0.12
t-Test: Two-Sample Assuming Equal Variances
| Variable 1 (Time spent by student within two days of homework forward) | Variable 2 (Time spent by student after two days of homework forward) | |
| Mean time spent | 133.7419 | 101.7531 |
| Variance | 27850.08 | 15955.06 |
| Observations | 90 | 120 |
| Pooled Variance | 21044.76 | |
| Hypothesized Mean Difference | 0 | |
| df | 208 | |
| t Stat | 1.581351 | |
| P(T<=t) one-tail | 0.057659 | |
| t Critical one-tail | 1.178359 | |
| P(T<=t) two-tail | 0.115317 | |
| t Critical two-tail | 1.561186 |
We now compare the calculated value of t with the tabulated value of a certain level of significance (.12). If calculated |t|> tabulated t, the null hypothesis is rejected.
Tabulated t = 1.5612,
For above table, we can see that:
calculated |t| > tabulated t. Hence, we reject the null hypothesis for a level of significance (0.12).
That means, there is an impact of homework on user engagement.
Voice of Student and Teacher on Homework
EnglishHelper conducted telephonic conversations with a sample of the students (n=100, across both states) who demonstrated activity on the “homework words”. This was undertaken to verify that their activity was initiated because of the homework program and to distinguish it from any coincidental activity arising from other awareness initiatives that are constantly undertaken in each state.
Of the students contacted, more than 90% recall receiving pre-packaged homework from the teacher. Eighty-eight percent or more engage with ReadToMe Student Edition to finish the given homework. Students typically receive homework in the classroom, some students mentioned receiving it through WhatsApp. This is likely a function of intermittent school closures and re-opening.
In addition, to verify claimed recall, students were asked to recall specific elements from the homework received. The numbers presented below are based on such verified recall.
| Maharashtra | Punjab | |
| Number of students telephonically surveyed | 57 | 43 |
| % students recall receiving homework | 93% | 91% |
| % students doing homework on ReadToMe Student Edition | 98% | 88% |
In addition, EnglishHelper contacted teachers of the students who exhibit engagement with ReadToMe Student Edition after receiving homework. 6 of the 7 teachers contacted, confirmed having shared the pre-packaged homework with their students through WhatsApp.
Developing Improved Mechanisms for Homework Delivery
Given the teacher acceptance and student compliance with the pre-packaged homework program, EnglishHelper has empowered the teacher by enabling homework delivery through WhatsApp from the ReadToMe Teacher Dashboard Android App. With this integration, teachers can choose appropriately from a set of pre-packaged homework at any time and deliver it to their students through a WhatsApp forward.
Subsequent homework delivery features are currently under development. These will allow the teacher to directly send the homework to her students registered on ReadToMe Student Edition, without an intermediate step or platform. With these improvements, teacher acceptance as well as student compliance is expected to increase, providing a seamless teaching-learning experience. This is expected to positively impact students’ self-learning on ReadToMe Student Edition.
Key Takeaways
Homework is an effective tool to promote self-learning and encourage independent study among students. For students to derive benefit from homework, it should be matched to their ability, require the right quantum of effort and the student should be adequately supported to complete the given task. ReadToMe Student Edition related homework is designed keeping these tenets in mind. When this homework is assigned to students by teachers in class or through other messaging mechanisms (e.g., WhatsApp) it is an effective nudge to students to use the learning resources available on the ReadToMe Student Edition Android app to complete the homework.
The homework nudge is effective as evidenced from the behaviour exhibited by students upon receiving homework from teachers:
1. Students look up homework words more frequently on ReadToMe Student Edition
2. They spend more time engaging with ReadToMe Student Edition
3. More students engage with ReadToMe Student Edition
Improved mechanisms for homework delivery have been released in subsequent versions of the app, with more under development. These are expected to improve the homework delivery by providing a seamless teaching-learning experience and are expected to positively impact students’ engagement with ReadToMe Student Edition, and favorably improve learning.
Appendix
References
[iii] Roschelle, J. & Hodkowski, N.M. (2020). Using research on homework to improve remote and hybrid learning [White Paper]. San Mateo, CA: Digital Promise. http://hdl.handle.net/20.500.12265/107