by Stacey Dingwall

The Education Endowment Foundation (EEF) is an independent charity that works to break the link between family income and educational achievement, ensuring that children from all backgrounds can fulfil their potential and make the most of their talents. Funded by the Sutton Trust and Department of Education, the EEF aims to raise the attainment of children facing disadvantage by:

• Identifying and funding promising educational innovations that address the needs of disadvantaged children in primary and secondary schools in England.
• Evaluating these innovations to extend and secure the evidence on what works and can be made to work at scale.
• Encouraging schools, government, charities, and others to apply evidence and adopt innovations found to be effective.

At the beginning of October, the EEF published reports outlining findings from seven of their projects. Writing on the EEF blog, Dr Kevan Collins explained that “we investigate the strategies schools use to improve their students’ attainment. We use independent evaluators to assess whether these methods really make a difference, with a focus on their impact on pupils eligible for free school meals”. On the National Foundation for Education Research’s (NFER) blog, Marian Sainsbury praised the work of the EEF in the area of evaluation, particularly their use of the randomised control trial (RCT) method of evaluation, a method which she argued is not used often enough in education.

One of the EEF’s evaluation reports received particular attention in the media: an evaluation of the ‘Increasing Pupil Motivation’ initiative, which aimed to improve attainment at GCSE level by providing incentives to Year 11 pupils in England. The initiative was largely targeted at relatively deprived schools, and offered pupils two incentives: one financial, in which the amount was reduced if they did not maintain standards in four measures of effort (attendance, behaviour, classwork and homework); and the other offered the incentive of a trip or event, under which pupils were allocated a certain number of tickets, to be reduced if they did not meet the four measures of effort.

Despite previous research indicating that the provision of incentives directly for test scores had a positive effect, the EEF evaluation of the provision of incentives specifically for effort found no significant positive impact of either type of incentive on GCSE attainment in English, maths or science. The findings did, however, suggest that the financial incentive had a significant positive impact on classwork effort in each of the three subjects, while both incentives had a positive impact on GCSE maths for pupils with low levels of prior attainment.

Findings from evaluations of another six initiatives were also published:

• Hampshire Hundreds – a local authority-led intervention that brought together lead teachers from Hampshire primary schools to provide them with evidence and support for effective teaching strategies to decrease the attainment gap between disadvantaged pupils and their peers. The EEF’s evaluation found that the initiative had no significant impact on raising attainment for disadvantaged pupils during the evaluation period, and highlighted the difficulty in converting research evidence into effective action within schools as well as the importance of careful piloting of an intervention before attempting an impact evaluation.

• Literacy Intervention Toolkit (LIT) – a programme which aimed to improve the reading ability of children in Year 7 in London and the south west and south east of England who scored below Level 4 at the end of primary school, using a method known as reciprocal teaching (reciprocal teaching encourages children to ‘become the teacher’ and teaches them how to apply four comprehension strategies – summarising, clarifying, questioning, and predicting – which enables them to understand the content of the material they are reading and make inferences based on what they have read). The evaluation of LIT found that while there was no evidence of a significant impact on the reading ability of pupils, teachers felt that the programme facilitated ‘healthy debate’ within the classroom, increased confidence in pupils who struggled with core literacy skills, and promoted independent learning.

• Mind the Gap – a project which sought to improve the metacognition and academic attainment of pupils in Year 4 (metacognition is based on the idea that effective learning depends on skills, attitudes and dispositions to learning, and that these can themselves be learned). The project consisted of two strands: the first involved training teachers in how to embed metacognitive approaches in their work and how to continue to effectively and strategically involve parents; while the second focused on parental engagement and offered families the opportunity to participate in a series of facilitated workshops where children and parents work together to create an animated film. Again, no evidence of a statistically significant impact of Mind the Gap on attainment was found, however the initiative’s impact on pupils’ metacognition was found to be positive and statistically significant, and participating families and staff felt the intervention enhanced home–school relationships and strengthened the learning relationship between children and parents.

• Summer Active Reading – this programme aimed to improve reading skills and particularly comprehension by raising children’s engagement in, and enjoyment of, reading at the transition from primary school to secondary school. The evaluation examined the impact of the programme on 205 pupils from 10 schools in the north of England who had been identified as unlikely to achieve Level 4a or above by the end of Key Stage 2, and found that, on average, pupils who participated in the programme made slightly more progress in reading comprehension that those who did not, and that the programme had a positive impact on the enjoyment of reading.

• TextNow Transition – an initiative which aimed to improve the reading comprehension skills of pupils at the transition from primary to secondary school by encouraging engagement in and enjoyment of reading through one-to-one sessions with a volunteer coach. While the evaluation did not find any evidence that the programme improved reading comprehension or attitudes towards reading for pleasure during the transition from primary to secondary school, it was noted that on average, pupils who participated in the programme made slightly less progress than similar pupils who did not; the programme had a differential effect for pupils eligible for free school meals compared to their peers; and higher attendance at the daily coaching sessions was found to have a positive impact on reading comprehension.

• Vocabulary Enrichment Full Programme (VEFP) – devised by Bolton local education authority, with the aim of improving the reading ability of pupils in Year 7. The evaluation of VEFP found that while pupils generally appeared to engage well with the intervention, it had no effect on reading ability when compared to schools’ normal Year 7 English curriculum.

As raising attainment is a complex issue, with many variables, it’s good to see these evaluations being disseminated and the findings discussed widely in the teaching profession.

---

The Idox Information Service has a wealth of research reports, articles and case studies on the importance of evaluation and evidence-based policy, as well as tips on how to carry out evaluations. These include:

7 essays on impact (DESCRIBE project: definitions, evidence, and structures to capture research impact and benefits)

Making evidence useful: the case for new institutions

The evidence base for an outcomes approach (Briefing paper 18)

A practical guide to outcome evaluation

Evidence exchange: learning from social policy from across the UK

N.B. Abstracts and access to journal articles are only available to members.