Texting Students and Study Supporters
Texting Students and Study Supporters (Project Success) was developed by the Behavioural Insights Team (BIT) and aimed to use text messages to improve GCSE English and maths re-sit pass rates by prompting college students to attend classes and exams, engage with study materials and form better study habits, either through direct contact with the learner or through prompting a dialogue with a nominated study supporter, e.g. a family member. Over the course of the academic year, weekly text messages were sent to students (a total of 36 for English or 37 for maths) and/or their study supporters via the BIT Promptable text messaging service.
Testing a low-cost way of engaging students at further education colleges
Using text messages to improve parental engagement and increase outcomes is a well-evidenced approach. The EEF previously funded an evaluation of text messages in secondary school, which found small positive impact on GCSE maths outcomes. This evaluation explored whether the approach could successfully improve the outcomes of students re-sitting GCSE exams in further education settings.
Our trial of Project Success included 3,779 students across 31 further education settings. This independent evaluation found that students who received text messages or had study supporters that received messages did not, on average, have higher pass rates in GCSE re-sits. These results have a high security rating.
There are some important limitations to the evaluation which may explain the different results from the Texting Parents studies that have taken place with younger age groups. Firstly, the measure of impact was whether students passed or failed the GCSE resits. This binary measure means that it is harder to detect the type of small impacts seen in the Texting Parents trial. A second limitation is that students had to opt into the trial – the process evaluation found that these students were already highly motivated. Text messages were less likely to have an impact on students who were already motivated.
The EEF has no further plans for a trial of Texting Study supporters.
There is no evidence that the Project Success intervention had any impact on the GCSE English or maths re-sit pass rate for further education college students.
There is no evidence that the Project Success intervention had any impact on the attendance of further education college students re-sitting GCSE English or maths.
The intervention did not have a differential impact on the GCSE re-sit pass rate by gender or by eligibility for free school meals (at the end of KS4). The subject being re-examined or the number of re-sits being taken also did not lead to differential effects from the intervention.
The use of mobile phone technology was perceived as a highly appropriate, effective, and low risk means of engaging with the target student cohort, though mobile phone use was less popular among study supporters.
There were significant limitations to the programme’s ability to engage those who may need it the most as it was the highly motivated students that were more engaged with their studies and with college generally who were more likely to sign up to the intervention.
Full project description
Project Success was developed by the Behavioural Insights Team (BIT) and aimed to use text messages to improve GCSE English and maths re-sit pass rates by prompting students to attend classes and exams, engage with study materials, and form better study habits, either through direct contact with the learner or through prompting a dialogue with a nominated study supporter such as a family member. The text messages were targeted at further education college students aged between 16 and 18 years and who were re-sitting maths or English. Over the course of the academic year, weekly text messages (a total of 36 for English or 37 for maths) were sent to students or their study supporters (or both) via the BIT Promptable text messaging service. This project was funded by the Education Endowment Foundation (EEF) as part of a joint initiative with J.P. Morgan to explore how to improve outcomes for disadvantaged 16- to 18-year-old students who achieve below a grade 4 in GCSE English or maths.
The evaluation included 3,779 students across 31 further education (FE) colleges in England. The efficacy trial used a four-armed, multi-site, randomised controlled design with individual random assignment to each trial arm. The four trial arms were: student received text messages, study supporter received text messages, both student and study supporter received text messages, and control (no text messages). The efficacy trial investigated the extent to which the receipt of text messages (either by the student, a study supporter, or both) improved students’ college attendance and GCSE maths or English re-sit results. The trial also explored whether any effect of receiving text messages varied according to the student’s gender or whether they had ever been eligible for free school meals. The primary outcome was obtaining a pass grade (4–9) in GCSE English or maths upon re-sitting, with lesson attendance being assessed as a secondary outcome. Where students were re-sitting both English and maths, the subject used for the intervention was randomly selected; the same subject was analysed as the primary outcome.
Alongside the impact evaluation, a mixed-methods implementation and process evaluation (IPE) was carried out. This included: a short, online diagnostic survey of students developed by BIT and NatCen, which was completed as part of the recruitment process; an observation of a tutor workshop; and interviews with college project and subject leads, study supporters, and students. The trial started in September 2017 and concluded in October 2019 with the intervention being delivered throughout the academic year 2017/2018.