Best Practice

Evidence-based resources: Do they make a difference?

Resources and projects CPD Action research Pedagogy
Many teachers find it hard to engage with and act on research evidence, even when they know it can help them to improve teaching practice and student outcomes. Dorothy Lepkowska explains

Two new studies undertaken by the National Foundation for Educational Research (NFER) and known as The Literacy Octopus Trials, named after their multi-armed design, have looked at the impact of research dissemination on achievement in schools. The studies were funded by the Education Endowment Foundation (EEF), the Department for Education (DfE) and the Mayor of London’s Schools Excellence Fund as part of a bigger scheme of exploring the use of research in schools.

As some of the largest education trials to date, a total of more than 13,000 primary schools in England were involved in the studies which looked at a range of evidence-based resources and events designed to support the teaching and learning of literacy at key stage 2. These included printed and online research summaries, evidence-based practice guides, webinars, face-to-face professional development events and access to online tools.

The first study, which involved 12,500 primary schools, looked at different approaches to the dissemination and communication of research materials to support literacy teaching at key stage 2, using a randomised controlled trial (RCT). The materials for this Literacy Dissemination Trial were drawn up by four leading organisations with experience of engaging schools in the use of evidence. The schools were split into five groups of 2,500 schools each. One was a control group which did not receive any of the materials. The remaining groups were sent resources from one of the four organisations each using a different dissemination method:

  • Improved Reading: A Guide for Teachers (published by the Centre for Evaluation and Monitoring, Durham University).
  • Better Evidence-based Education (a magazine from the Institute for Effective Education, University of York).
  • A link to an archived webinar and materials from a conference on research evidence relating to key stage 2 literacy (from ResearchED in collaboration with NatCen).
  • A subscription to the Teaching How2s website, which focuses on evidence-based techniques and visual guides for teachers (delivered by the Campaign for Learning with Train Visual).

The resources were simply sent to the schools with a note about the trial. The purpose of this large-scale trial was to assess the impact of disseminating evidence-based resources on pupil outcomes, rather than to explore if or how the materials were used.

A second RCT involved 823 schools, and tested whether combining resources with light-touch support on their use would have greater impact. Some of the schools received light-touch support, including invitations to seminars on using the materials in the classroom, webinars before and after conferences to provide support on how to use the materials, and introductory events to using online resources.

Neither trial found evidence of improved literacy attainment at key stage 2 and the second study, which included a survey, showed no difference in teachers’ use of research, despite the additional support. The results suggest that, in general, light-touch interventions and resources need more support to make a difference.

The initial engagement of schools with the disseminated materials varied. Even where the materials included guidance on how to apply evidence in practice, the report suggested that trying to get teachers to engage with the resources or support was a challenge.

Six out of 10 primary schools engaged only a little or not at all according to monitoring data, citing a lack of time and a preference for more interactive support.

However, around one in six schools engaged and used the materials to a greater degree than expected, for example by hosting CPD sessions and requesting further materials. Through case studies, the study found a small number of schools that went further and implemented change after using the resources, or trialled new ideas and embedded them in their classroom practice.

Researchers plan to publish pupil outcomes in a year’s time from the 2016/17 cohort to establish whether there was any long-term effect from interventions.

The report concluded that the findings had raised two key issues. The first was centred on what could be done to encourage schools to engage more in research evidence, when they are already over-burdened with information. Second, it said it was important to consider what level of additional activity around research would help schools to use and refer to research in order to improve pupil outcomes.

To examine these questions further, the EEF has launched two campaigns to promote the effective use of evidence, focusing on primary-age literacy in the North East of England and better use of teaching assistants in the classroom in 1,000 schools in Yorkshire and Lincolnshire.

Dr Ben Styles, head of NFER’s Education Trials Unit, explained: “Although the findings of this research are of critical importance to organisations involved in the translation of research evidence into practice, there are also aspects of these trials that should warm the hearts of those doing rigorous evaluation research.

“The fact that more than 800 primary schools were willing to sign up to a trial that required various levels of proactive engagement by teachers, in combination with the ability to analyse data from the National Pupil Database has resulted in a rare example of robust quantitative evaluation in this field.”

Commenting on the reports, Sir Kevan Collins, chief executive of the EEF, said: “Teachers and school leaders now have access to a significant and growing body of academic research with enormous potential to improve pupil attainment and save schools money. But to do this, we need to make sure that research findings get into the hands of teachers in ways that are most likely to have an impact.

“We know how challenging this can be. Light-touch interventions are unlikely to have an impact on pupil attainment and getting teachers to engage with research is far from straightforward. We need to focus our efforts on more targeted and structured approaches to disseminate evidence and support teachers.”

  • Dorothy Lepkowska is a freelance education journalist.

Further information

NFER Research Insights

This article was published as part of Headteacher Update’s NFER Research Insights series. A free pdf of the latest Research Insights best practice and advisory articles can be downloaded from the supplements page of this website: www.headteacher-update.com/supplements/