International studies of academic performance are sometimes seen as controversial when education systems, teaching methods and compulsory education starting ages vary so much between different countries. But these studies can also provide useful information to inform policy-making and classroom practice.
For example, the Trends in International Mathematics and Science Study (TIMSS) provides insights into a range of factors that can affect performance, such as pupils’ home backgrounds and learning environments, and the quality and content of teaching.
The involvement of NFER in TIMSS goes back to its inception in 1995. Since then, NFER has been analysing the outcomes to find out what educators and policy-makers can learn.
TIMSS focuses on the performance of pupils aged 9 to 10 (year 5) and 13 to 14 (year 9), but here we will concentrate on the outcomes for primary pupils in England.
The most recent study, published for the year 2015, shows that year 5s in England scored 546 for maths. Although this is slightly higher than the previous two scores in 2011 and 2007 (542 and 541 respectively), it cannot be taken as evidence that standards have improved.
Researchers at NFER urge caution when analysing international performance because comparisons, such as this trend over time, often don’t stand up to scrutiny.
Their latest briefing report, entitled Twenty Years of TIMSS in England, states: “Simply looking at whether the score for maths or science is higher or lower than in a previous TIMSS cycle does not tell us accurately whether achievement has improved.
“It is crucial to consider whether a score is statistically significantly different – in other words, the differences have not arisen solely by chance.”
The authors also say that while it might be tempting to focus on country rankings, these can be misleading because they might be affected by small changes in pupils’ scores that are not in themselves statistically significant.
Furthermore, rankings can be volatile and may vary according to which countries are participating. The position of one country in the tables is not just based on its own performance, but that of all the others taking part, and how they had fared in that cycle. For example, even though performance among year 5s in England had not changed significantly in 2015 from 2011, it was outperformed by three additional countries during this period.
Even where differences are statistically significant (considering for example high-performers such as Hong Kong and Singapore), caution is needed in interpreting the results. We cannot know for sure whether these results are due to teaching practices and aspects of the education system or whether they are due to other environmental or cultural factors. So it is important to consider TIMSS alongside other sources of evidence.
One of the important pieces of information that the TIMSS results provide is the range of scores among the highest and lowest achieving pupils. Merely taking an average score for a country doesn’t necessarily tell us everything we need to know.
For example, Singapore is the highest achieving country for both subjects but it has a wide spread of achievement (of 280 scale points) in year 5.
England’s performance
So how did year 5s in England fare in the TIMSS rankings in 2015? An analysis of outcomes shows that primary pupils in England have maintained similar levels of performance since 2007. The proportion of youngsters achieving at the highest international level has been stable for nearly a decade and, since 2003, the proportion of year 5s reaching this level has been greater than in year 9.
England was ranked 10th in maths in year 5 in 2015 and was only significantly outperformed by seven countries. The TIMSS surveys found no gender differences in maths achievement for year 5 for the previous 20 years, though in 2015, boys outperformed girls significantly.
In science, England was ranked 15th overall in year 5. However, the performance of year 5s has fluctuated, having improving slightly between 1995 and 2003 but declined significantly between 2007 and 2011. Since 2011, it has remained largely stable.
The proportion of year 5s achieving the highest international benchmark in 2015 was significantly lower than in previous surveys, but similar to 2011. Researchers found no significant differences in the performance of boys and girls in science at this age.
Looking at the contextual evidence, the teachers of year 5 pupils were, generally speaking, satisfied with their job, though the number of teachers who were “very satisfied” was lower than in other countries. In maths, levels of participation in professional development were slightly higher for primary teachers compared with secondary teachers in England, though in science the reverse was true.
Around three-quarters of primary pupils attended schools with few discipline challenges and considered “very safe and orderly”, which was above the international average. Pupils in England also experienced bullying less frequently than in several other TIMSS-participating countries.
Primary pupils in England were also more likely to be engaged in science and maths learning than secondary pupils, with more than 80 per cent of year 5 pupils reporting a positive attitude towards those subjects, which is similar to the international average. Year 5 pupils also felt more confident with maths than they did about science.
The NFER briefing paper highlights how TIMSS enables policy-makers to examine how past policies relate to achievement over time. It also offers insights into how the most successful and most improved countries achieved their successes. It might also inform how changes in curricula in different countries had affected achievement.
Northern Ireland’s performance
In Northern Ireland, the NFER’s TIMSS analysis looked at year 6 pupils and their achievements over a four-year cycle. Northern Ireland took part in TIMSS for the second time in 2015 so comparisons can only be made with 2011.
An analysis of the country’s performance shows that, in maths, pupils in Northern Ireland out-performed 42 out of 50 participating countries but was significantly out-performed by five of those. Overall, science scores were lower than in maths, and Northern Ireland is outperformed in this subject by 22 countries, though it is in a group with seven others scoring similar results.
However, scores in both maths and science have remained stable and were not significantly different in 2015 than in 2011. There were also no significant gender differences. More than a quarter of year 6 pupils in Northern Ireland reached the TIMSS Advanced International Benchmark in maths, the sixth highest percentage overall. Over the four-year period there was an increase from 24 per cent in 2011 to 27 per cent in 2015.
In science, however, only five per cent of year 6s reached the Advanced International Benchmark. There was a wide spread of attainment in maths but a smaller gap between the best and worst achievers in science.
When it came to content, Northern Ireland pupils did better on numbers than on geometric shapes and measures and, in cognitive domains, better on knowing and applying than reasoning.
Researchers found that youngsters who most liked maths and science, or were confident in them, were more likely to excel in the subjects. This was also true of children whose parents had positive attitudes towards those subjects. Where pupils found teaching engaging this was also associated with their likelihood to succeed. Generally speaking, year 6s were more likely to be taught by a subject specialist in maths, than in science.
More teaching time was devoted to maths in Northern Ireland than in other countries, but for science this was the reverse. Classrooms in the country were also described as well-resourced, and where children had access to good home resources, this was also positively associated with achievement.
- Dorothy Lepkowska is a freelance education journalist.
Further information
- NFER Education Briefings: Twenty years of TIMSS in England, 2016: www.nfer.ac.uk/publications/99958/99958.pdf
- TIMSS 2015 in Northern Ireland: Mathematics and science: NFER, November 2016: www.nfer.ac.uk/publications/TMSS01/
NFER Research Insights
This article was published as part of Headteacher Update’s NFER Research Insights series. A free pdf of the latest Research Insights best practice and advisory articles can be downloaded from the supplements page of this website: www.headteacher-update.com/supplements/