Project Based Learning (PBL) is a pedagogical approach that seeks to provide Year 7 pupils with independent and group learning skills to meet both the needs of the Year 7 curriculum as well as support their learning in future stages of their education. It particularly aims to improve their engagement in learning as well as practical literacy skills. This trial evaluates a specific type of PBL known as ‘Learning through REAL Projects’, developed by the Innovation Unit—an independent social enterprise that aims to improve public sector services.
A year long pilot project in 2013/2014 with eight schools (763 pupils) established the feasibility of a main trial. It provided the opportunity to develop the intervention and test the research procedures. The results contained in this report relate to a randomised controlled trial that took place between September 2014 and April 2016. Twelve intervention schools (2,101 pupils) and 12 control schools (1,973 pupils) were involved in the trial.
REAL projects was delivered by teachers, supported in many instances by teaching assistants (TAs), with further support from senior leadership colleagues. It was delivered for a year in the intervention schools with a relatively large proportion of timetabled teaching (varying between 20% and 50%). In almost all cases it was delivered to mixed-ability Year 7 classes.
The intervention used structured cross-subject ‘REAL Projects’ planned by the delivery teachers who were supported by Classroom Coaches from the Innovation Unit. Schools were also supported by Leadership Coaches. REAL Projects are driven by an essential question which has significant educational content. The projects encouraged pupils to create an ‘excellent’ product through drafting and redrafting and then to exhibit their work to an ‘authentic’ audience.
Visits were made to intervention schools at two time points and consisted of lesson observations, interviews, and focus groups with school leadership, project leads, class teachers, and pupils. Two case studies were also conducted in schools that it was agreed delivered the intervention well.
Testing the impact of project-based learning in secondary schools.
Durham University, The York Trials Unit
Developing effective learners
Organising your school
Language and literacy
The following conclusions summarise the project outcome
Adopting PBL had no clear impact on either literacy (as measured by the Progress in English assessment) or student engagement with school and learning.
The impact evaluation indicated that PBL may have had a negative impact on the literacy attainment of pupils entitled to free school meals. However, as no negative impact was found for low-attaining pupils, considerable caution should be applied to this finding.
The amount of data lost from the project (schools dropping out and lost to follow-up) particularly from the intervention schools, as well as the adoption of PBL or similar approaches by a number of control group schools, further limits the strength of any impact finding.
From our observations and feedback from schools, we found that PBL was considered to be worthwhile and may enhance pupils' skills including oracy, communication, teamwork, and self-directed study skills.
PBL was generally delivered with fidelity but requires substantial management support and organisational change. The Innovation Unit training and support programme for teachers and school leadership was found to be effective in supporting this intervention.
What is the impact?
The trial results did not find that the PBL programme had an impact either on the pupils’ literacy performance (as measured by Progress in English 12 tests), engagement, or attendance. The analysis did find a statistically significant negative impact on students eligible for free school meals (FSM), however no negative impact was found for lower-attaining pupils more generally, which makes it difficult to hypothesise why PBL might negatively impact FSM pupils specifically. This adds to the uncertainty of the finding. Given the inherent design limitations, high attrition rate, and cross-over in terms of group allocation (with some control schools also involved in PBL) we should further limit the confidence we have in the above findings as bias in either direction could have been introduced.
The process evaluation, which was based on classroom observations and feedback from headteachers, project leads, teachers, and pupils in the schools, as well as the Innovation Unit delivery team, did provide evidence of positive benefits from doing PBL, in particular in terms of developing oracy, communication, team working, and research skills.
The existing international evidence on the effectiveness of PBL is relatively weak, and this research contributes to the evidence base from an English perspective. In summary, although PBL is unlikely to improve children’s literacy outcomes or engagement, it may enhance the quality of children’s learning, particularly improving some of the skills required for future learning and employment.
|Group||Number of schools||Effect size (95% confidence interval)||Estimated months’ progress||EEF security rating||EEF cost rating|
|PBL vs. control||24||-0.13 (-0.31, 0.05)||-2|
|PBL (FSM-eligible) vs. control||24||-0.24 (-0.41, -0.06)||-3||NA|
How secure is the finding?
Overall, the findings have low security. The trial was designed as a two-armed randomised controlled trial with schools being allocated to intervention or control groups. It was set up as an efficacy trial which aimed to test if the intervention can succeed under ideal conditions. However, 47% of the pupils in the intervention and 16% in the control group were not included in the final analysis. Therefore there were some potentially important differences in characteristics between the intervention and control groups. This undermines the security of the result. The reason that so many pupils from schools implementing PBL are missing from the analysis is largely due to five of these schools leaving the trial before it finished. Many of the schools in this trial were experiencing challenging conditions, and some of the schools that dropped out had a change of leadership team during the trial. It may be that successful implementation of the PBL programme is difficult for schools in such circumstances. For some of these pupils without Year 7 outcome data, Year 11 data will be available in the future from national data sources. This will enable EEF to do a future analysis of the effectiveness of PBL which does not suffer from such high rates of data loss.
How much does it cost?
The cost per pupil is about £58 per year for the support package from the Innovation Unit. The main additional costs are teacher time for training which varied to some degree between schools. The Innovation Unit advised a minimum of three days each for three teachers (initial training) plus ongoing planning time as necessary including between one and three hours per week for the school PBL team.
We evaluated a version of Project Based Learning (PBL) called ‘Learning through REAL projects’, which aims to improve attainment by delivering the curriculum through cross-subject projects accounting for 20-50% of timetabled teaching. We funded this trial because PBL is a popular teaching approach, even though there is limited evidence on its effectiveness.
We found no evidence that PBL had a positive impact on pupils’ literacy or their engagement with school and learning. However, the evaluation indicated that PBL may have had a negative impact on literacy for pupils eligible for free school meals. This study uses a high-quality evaluation design and is among the best evidence to date on the impact of PBL. However, because of the high number of schools that dropped out, these findings are less secure than those from most EEF trials, and caution should be exercised when interpreting them.
For schools thinking of adopting PBL, the implications on timetabling and staffing should be considered alongside the findings in this report. It is also important to consider the opportunity-costs of implementing a new, large scale whole-school approach to teaching, in particular the time it takes to train teachers to deliver a new pedagogical approach.