‘I think the educational and psychological studies I mentioned are examples of what I would like to call Cargo Cult Science. In the South Seas there is a Cargo Cult of people. During the war they saw airplanes land with lots of good materials, and they want the same thing to happen now. So they’ve arranged to make things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head like headphones and bars of bamboo sticking out like antennas – he’s the controller – and they wait for the airplanes to land. They’re doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn’t work. No airplanes land. So I call these things Cargo Cult Science, because they follow all the apparent precepts and forms of scientific investigation, but they’re missing something essential, because the planes don’t land.’ Richard Feynman’s Caltech commencement address on Education and Cargo Cult Science (1974). 

‘Let’s put behind us once and for all the old sterile debate about dumbing down. I want to end young people being told that the GCSE or A-level grades they are proud of aren’t worth what they used to be.’ Ed Balls to the Labour Party Conference,  2007. 

‘It is undeniable that the last Labour government dramatically improved school standards in secondary education.’ Tristram Hunt, 26 January 2015.

‘Despite the apparently plausible and widespread belief to the contrary, the evidence that levels of attainment in schools in England have systematically improved over the last 30 years is unconvincing. Much of what is claimed as school improvement is illusory… standards have not risen; teaching has not improved… The question, therefore, is not whether there has been grade inflation, but how much…’ (Professor Robert Coe, 2013, here.)

Summary

This series of blogs will discuss: 1) what we know about standards in English schools including the effect of the introduction of the National Curriculum and GCSEs; 2) how ‘ability’ and ‘standards’ should be defined; 3) what can be learned from the 2010-15 reforms and what incentives now dominate the system; 4) what research and policy agenda is needed; 5) what materials are there for those interested in standards beyond those of the National Curriculum and state controlled exams.

Introduction

The debate about ‘standards in English schools’ is obviously of great importance but it suffers from many fundamental problems. Ironically for a debate that often involves the word ‘rigour’, the debate is itself unrigorous.

The main concepts are not properly defined. Politicians, policy people, officials, and journalists speak and write daily using phrases such as ‘we must drive up standards so that [X% of schools or pupils] hit the standard of [Y]’ when Y has no objective definition. Most obviously there has been enormous debate about grades in GCSEs and A Levels but these grades themselves are arbitrarily created according to criteria that would not impress physical scientists. The ‘standards’ are circular. Exams are regulated by the DfE and Ofqual in order that there is a very high chance that at least X% ‘pass’, then people say ‘more than X% should pass’, or ‘X% is too tough’. But the X% is just based in the first place on where the system happens to be which is historically contingent – it is not based on any scientific judgement about what children of different abilities (rigorously defined) are capable of doing given certain teaching.

In the recent debate over reforming GCSEs, when we tried to drop their use in the accountability system in 2012, Nick Clegg insisted, and Cameron agreed, that the entire reform process be based on the principles that i) about 95% of the cohort should do the same exams at 16 and ii) not many more would fail to pass than now (2012). Definitions of a ‘pass’ were therefore set in order to fit with an a priori desire for a certain percentage to pass – a political desire of one party rather than an educational judgement. (The other two parties have had the same approach over the past thirty years – my point is not that the LibDems are particularly bad.)

Despite having a circular process for defining standards, it has been a central feature of education debates for politicians to set targets for what proportion of pupils every school must get to ‘pass’ – targets that have high stakes for school management and teachers. One can understand the motivation, given the bad effects for individuals of being in really bad schools, but the process as a whole does not make sense. Further, Ofqual imposes a system (‘comparable outcomes’) which is intended to combat grade inflation but which also seems to operate deliberately against the goal of significant rises in the proportion passing GCSEs. Further, Ofsted’s reports add noise, not signal, given, as Professor Coe has said, ‘its judgements have little scientific credibility’ (some argue this is too generous).

Similarly, people in the education world use the word ‘ability’ but they almost never define or have an objective measure for ‘ability’. The work of scientists on this subject has been almost entirely ignored and has had practically no effect on policy in England. Many teacher training colleges promote ‘cargo cult’ science on the subject of ‘ability’ to thousands of teachers who are therefore confident in views that are the opposite of what the science says.

As far as I am aware, there is no serious research agenda in English schools attempting to a) discover what pupils of different ability, using objective measures, are capable of achieving given certain teaching and b) use this knowledge to shape the curriculum, tests, and objective measures of school performance in an iterative feedback loop that can improve its accuracy over time.

The main point of these blogs is to help make the case for such a research programme (see below). Since first becoming involved in education debates in 2007, I have had many discussions about this. I have said to many people, including in the Royal Society, the home of British science, that we need a scientific approach to the issue of standards and ability. I wrote about it in my essay that became public in 2013. I argued for it in the DfE, with subject associations, with those responsible for teacher training (‘the most bankrupt institution I know’, said Hattie), and with many people who talk about ‘research’ and ‘evidence’.

Few have wanted to engage in this subject because it is so politically fraught. Even fewer have done so publicly and I have personal experience of severe pressure put on many academics by university administrators not to tell the truth. However, a very positive development in English education is the growth of support for thinking seriously about evidence. In the DfE, there was a long battle on this issue that ended suddenly when the new Permanent Secretary arrived and immediately agreed with the appointment of Ben Goldacre to do a review of the Department’s handling of evidence, research, and data, which was published in 2013. (I have written many critical things about officials, such as HERE, so it is worth noting that Wormald, and other officials particularly younger ones, took this enlightened view.) There is no doubt that the culture inside the Department changed as a result though there is a very long way to go in this area and it is reasonable to be doubtful about any of the three parties’ commitment to this approach and about civil service commitment. Tom Bennett’s efforts with ResearchEd have been fantastic and are one of the most hopeful things I’ve seen since 2007. There is also now a discussion about a possible College of Teachers – an institution that will only be credible if it has high standards on the subject of cargo cult research. Unsurprisingly, therefore, more people are starting to ask: what do we know about standards? (E.g. Sam Freedman recently blogged on it.)

I therefore thought I would jot down in a series of blogs various bits of evidence, history, thoughts, discussions and so on that I have accumulated since 2007.

Five broad areas

This series of blogs will consider inter alia these questions grouped in five rough areas (which may change as I go along).

A. What is the evidence concerning ‘standards in English schools’? What was the effect of the introduction of 1) GCSEs and 2) the National Curriculum with its connected testing regime? What were the cascading effects on A Levels and higher education? What do comparisons with international tests and other academic studies tell us? What do subject associations and organisations such as the Royal Society say? What do universities and subject experts say?

B How should ‘ability’ and ‘standards’ be defined? What undermines sensible discussion about this?

C. What was Gove’s team trying to do 2010-14? How effective were reforms concerning the curriculum, exams, and accountability (including the role of Ofsted)? What lessons might be learned from the period 2010-15? What incentives dominate the system now?

D. What should come next? What can we reasonably infer from the period since 1985 about what is very unlikely to work? What should the parties not put in their manifestos? What are the main reasons why political and policy discussion of this subject has been so controversial? How does the transformation of the technological landscape since the mid-1980s change arguments? How could a focus on evidence and empiricism help improve the system?

E. What materials are there that can be used by schools that are focused more on education and learning than the official accountability system?

The goal

The goal of these blogs is not to ‘defend the Gove reforms’. When I get onto them, I will try to explain as clearly as I can why we tried to do certain things and what went wrong.  GCSE reform (along with the disaster of Ofsted) is arguably the biggest failure of our team and therefore particularly needs analysis. The goal is not to affect party manifestos – it is possible but unlikely that someone reading this may be able to nudge things off a party or bureaucratic agenda. It is reasonable to assume that whatever the parties promise their plans will crumble on contact with reality. My main hope is that people outside SW1 at the coalface of education take matters into their own hands and develop their own approaches to scientific experimentation with the curriculum, exams, and training.

In my opinion, the only real hope for large improvements in learning is if 1) a critical mass of people become convinced of the need for an empirical approach and the rejection of ‘cargo cult science’ that has dominated education, and 2) an empirical programme emerges that iteratively a) tests what children of different abilities can learn and b) uses this information to alter curricula, tests, and teacher training. We need experiments and Grand Prizes in education that have brought dramatic breakthroughs in other areas, such as DARPA’s Grand Challenge that led to breakthroughs in basic science and then to driverless cars. Imagine what well-defined Grand Challenges could bring to English schools.

Improvements in education do not need to be justified as goals with reference to other things such as economic growth. Learning and education are fundamental aspects of being human. However, it is obvious that humans will have to grapple with profound challenges over the next thirty years. The population will grow by another few billion, mainly in cities and connected to the mobile internet and ‘the internet of things’. Energy and other resource demands will put the global system under huge pressure. We face old security threats like nuclear weapons and new threats such as the use of genetic engineering techniques empowering garage bio-hackers, for good and evil. For example, the revolutionary genome ‘cut and paste’ engineering tool, CRISPR, may soon be used to ‘de-extinct’ species and eradicate diseases but the same techniques could be used destructively. Much progress in machine intelligence and robotics is being driven by research controlled by militaries and intelligence agencies but little research is done on the profound dangers.

If we are to cope with these things, we will need new technologies, new institutions, and new ideas. Improving our education system is therefore obviously central. I have proposed that it ought to become the central organising principle for the British state, as an answer to Dean Acheson’s famous quip that Britain had failed to find a post-imperial role.

Hopefully the discussion of standards in English schools will be useful regardless of whether you agree with this broader argument or not.

Please leave comments, corrections, research reports, complaints etc below. I will add things people leave as I go along and at the end try to produce something short and rigorous…