Wormald

by

Bureaucratic cancer and the sabotage of A Level reform

‘Bureaucracy is cancerous in head and limbs; only its belly is sound and the laws it excretes are the most straightforward shit in the world… With this bureaucracy including the judges on the bench we can have press laws written by angels and they cannot lift us from the swamp. With bad laws and good civil servants one can still govern, with bad civil servants the best laws cannot help.’ Otto von Bismarck, 1850.

‘I had the agreement in principle of my colleagues; I had the agreement in principle of the entire Landtag; and yet, although minister-president, I found myself absolutely unable to bring the matter one step further along. Agreement does not help me at all when passive resistance – from what direction in this complicated machine is impossible to learn – is conducted with such success that I am scarcely in a position after two to three years to answer even the most basic questions.’ Otto von Bismarck, 1878.

If the most effective political operator of the modern world frequently complained about the difficulty of enforcing policy against a hostile bureaucracy, we should not be surprised if similar problems recur over and over again.

Here is an interesting example of how education policy is made and how Whitehall works.

In 2012, we announced that the DfE would step back from controlling A Levels and give universities control. (Allegra Stratton ran the original story on Newsnight.) The main mechanism was ALCAB. It was a nightmare to set up partly because although subject experts very much wanted to be involved the administrators who control universities wanted to stay out of the controversy and said to us in the DfE ‘we don’t want to have to say publicly that A Level papers are bad’.

We forced ALCAB to be created. MG and I spent a lot of time in awful meetings forcing it through. Its main role was supposed to be an annual review of specific A Level papers so that professors XYZ could say ‘hopeless question in the Edexcel physics paper, it gets the definition of entropy wrong again, it fails to test XXX’ etc.

The DfE has closed this committee down. It emerged via this Times Higher Education story.

I pointed a few hacks to it. They have called the DfE press office and spads. Both of those entities were given a line from officials saying ‘ALCAB’s work is done, no story here’. (Cf. Forsyth’s blog here.)

This is a lie. The main role was an annual review process. This should have been conducted this year and 2016 in preparation for new A Levels in 2017. It was envisaged as a permanent role. Interestingly, the letters completely elide this main role out of existence and present ALCAB as having only a temporary role.

Now this annual review won’t happen.

This is almost a Jedi-level operation from DfE officials. The DfE hated giving away control, obviously, and hated ALCAB. The very point of the process – a sword of Damocles in the form of eminent professors saying ‘crap questions’ each year – was supposed to force the DfE, exam boards, and Ofqual to raise their game. You can imagine how popular this was. Now the situation will revert to the status quo – the DfE firmly in charge and those pesky professors who point out things like – specific papers do not test the maths skills in the specifications – are happily excluded, with no ‘unhelpful’ public scrutiny of standards.

I very much doubt that poor Nicky Morgan Nicky Morgan [*see end] realises what she has done. It was probably a letter buried deep in her box weeks ago that she had no reason to suspect meant she was being used to subvert reform and entrench Whitehall’s power. It is impossible for a new minister to spot all such things – you don’t know what you don’t know. We can also safely bet that No10 has not the faintest idea about what ALCAB is or what the annual review process was supposed to do.

This is how Whitehall closes down threats to its power. Although it is systemically incompetent viz policy and implementation, its real focus is on its own power, jobs, and money. To these, it pays careful attention and deploys its real skills.

It is possible that the hard struggle to improve A Levels and remove politicians’ and Whitehall’s grip of them is now substantially lost, without the MPs having a clue as to why and the details lost in a miasma of untraceable decisions and discussions.

Nicky Morgan and her spads should ask Rose (head of private office) and Wormald (Perm Sec) not just ‘how did this happen?’, but also ‘why were we and the press office given lies to tell the media?’ They would also be well advised to make clear that a repetition of this fancy footwork will mean someone fired. But of course this will have little effect. The officials are lining up their holidays and their own plans for the future, safe in the happy knowledge that whoever ‘wins’ the election, they will remain in charge. The MPs of all parties are largely content for this situation to continue. In the focus groups, swing voters will continue to say ‘they’re all the same’ with much more accuracy than they realise, but few in Westminster are really listening and even fewer know what is to be done…

I will blog a few reflections on No10’s ‘schools week’ tomorrow. NB. notice how, just as I wrote in The Hollow Men, this No10 ‘schools week’ is like all the others – two days of rubbish gimmicks, a self-inflicted cockup (‘real terms cuts to the budget’), followed by silence such that by Friday the 8 people who knew it was ‘schools week’ have themselves forgotten? Plus ca change…

Ps. If you want details on the devaluation of exams since 1988, and therefore why the annual review process was so important, read THIS.

 

UPDATE. Some have asked ‘how much confidence did you have in ALCAB doing a good job?’ Answer? Initially not much. They are all under huge pressure to say everything is fine. Initially for example, despite physics departments across the country  complaining about the removal of calculus from Physics A Level (complaints that practically none of them will repeat publicly because of fear of their VC office), it did not look like ALCAB would be much use and they rejected calls from various professors I know on this subject. There is massive political pressure to focus exclusively on the numbers taking an A Level rather than the quality  of the A Level.

But my hope was that by creating something that would be seen as the ‘voice of the university subject experts’, they would have to listen and adapt in order to maintain credibility and avoid embarrassing challenges. There are more and more enraged academics fed up of VC offices lying to the media and misrepresenting academics’ opinions. I thought that creating something would push the debate in increasingly sensible directions where the emphasis would be on the skills needed on arrival at university. Now, everything to do with A Levels is dominated by political not educational concerns about the numbers doing them and ‘access’. This has helped corrupt the exam system. If we had professors of physics, French, music etc every year publicly humiliating exam boards for errors, this would soon improve things from a low base and make it much harder for MPs and Whitehall to keep corrupting public exams.

[* I wrote ‘poor Nicky Morgan’ with the feeling – poor her, I know what it’s like to be pottering around in the DfE dealing with all sorts of problems before the horror of Question Time then someone walks in with a new bigger problem… But a few people email to say it sounds patronising which was not deliberate, hence deletion…]

by

Standards In English Schools Part 0: Introduction

‘I think the educational and psychological studies I mentioned are examples of what I would like to call Cargo Cult Science. In the South Seas there is a Cargo Cult of people. During the war they saw airplanes land with lots of good materials, and they want the same thing to happen now. So they’ve arranged to make things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head like headphones and bars of bamboo sticking out like antennas – he’s the controller – and they wait for the airplanes to land. They’re doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn’t work. No airplanes land. So I call these things Cargo Cult Science, because they follow all the apparent precepts and forms of scientific investigation, but they’re missing something essential, because the planes don’t land.’ Richard Feynman’s Caltech commencement address on Education and Cargo Cult Science (1974). 

‘Let’s put behind us once and for all the old sterile debate about dumbing down. I want to end young people being told that the GCSE or A-level grades they are proud of aren’t worth what they used to be.’ Ed Balls to the Labour Party Conference,  2007. 

‘It is undeniable that the last Labour government dramatically improved school standards in secondary education.’ Tristram Hunt, 26 January 2015.

‘Despite the apparently plausible and widespread belief to the contrary, the evidence that levels of attainment in schools in England have systematically improved over the last 30 years is unconvincing. Much of what is claimed as school improvement is illusory… standards have not risen; teaching has not improved… The question, therefore, is not whether there has been grade inflation, but how much…’ (Professor Robert Coe, 2013, here.)

Summary

This series of blogs will discuss: 1) what we know about standards in English schools including the effect of the introduction of the National Curriculum and GCSEs; 2) how ‘ability’ and ‘standards’ should be defined; 3) what can be learned from the 2010-15 reforms and what incentives now dominate the system; 4) what research and policy agenda is needed; 5) what materials are there for those interested in standards beyond those of the National Curriculum and state controlled exams.

Introduction

The debate about ‘standards in English schools’ is obviously of great importance but it suffers from many fundamental problems. Ironically for a debate that often involves the word ‘rigour’, the debate is itself unrigorous.

The main concepts are not properly defined. Politicians, policy people, officials, and journalists speak and write daily using phrases such as ‘we must drive up standards so that [X% of schools or pupils] hit the standard of [Y]’ when Y has no objective definition. Most obviously there has been enormous debate about grades in GCSEs and A Levels but these grades themselves are arbitrarily created according to criteria that would not impress physical scientists. The ‘standards’ are circular. Exams are regulated by the DfE and Ofqual in order that there is a very high chance that at least X% ‘pass’, then people say ‘more than X% should pass’, or ‘X% is too tough’. But the X% is just based in the first place on where the system happens to be which is historically contingent – it is not based on any scientific judgement about what children of different abilities (rigorously defined) are capable of doing given certain teaching.

In the recent debate over reforming GCSEs, when we tried to drop their use in the accountability system in 2012, Nick Clegg insisted, and Cameron agreed, that the entire reform process be based on the principles that i) about 95% of the cohort should do the same exams at 16 and ii) not many more would fail to pass than now (2012). Definitions of a ‘pass’ were therefore set in order to fit with an a priori desire for a certain percentage to pass – a political desire of one party rather than an educational judgement. (The other two parties have had the same approach over the past thirty years – my point is not that the LibDems are particularly bad.)

Despite having a circular process for defining standards, it has been a central feature of education debates for politicians to set targets for what proportion of pupils every school must get to ‘pass’ – targets that have high stakes for school management and teachers. One can understand the motivation, given the bad effects for individuals of being in really bad schools, but the process as a whole does not make sense. Further, Ofqual imposes a system (‘comparable outcomes’) which is intended to combat grade inflation but which also seems to operate deliberately against the goal of significant rises in the proportion passing GCSEs. Further, Ofsted’s reports add noise, not signal, given, as Professor Coe has said, ‘its judgements have little scientific credibility’ (some argue this is too generous).

Similarly, people in the education world use the word ‘ability’ but they almost never define or have an objective measure for ‘ability’. The work of scientists on this subject has been almost entirely ignored and has had practically no effect on policy in England. Many teacher training colleges promote ‘cargo cult’ science on the subject of ‘ability’ to thousands of teachers who are therefore confident in views that are the opposite of what the science says.

As far as I am aware, there is no serious research agenda in English schools attempting to a) discover what pupils of different ability, using objective measures, are capable of achieving given certain teaching and b) use this knowledge to shape the curriculum, tests, and objective measures of school performance in an iterative feedback loop that can improve its accuracy over time.

The main point of these blogs is to help make the case for such a research programme (see below). Since first becoming involved in education debates in 2007, I have had many discussions about this. I have said to many people, including in the Royal Society, the home of British science, that we need a scientific approach to the issue of standards and ability. I wrote about it in my essay that became public in 2013. I argued for it in the DfE, with subject associations, with those responsible for teacher training (‘the most bankrupt institution I know’, said Hattie), and with many people who talk about ‘research’ and ‘evidence’.

Few have wanted to engage in this subject because it is so politically fraught. Even fewer have done so publicly and I have personal experience of severe pressure put on many academics by university administrators not to tell the truth. However, a very positive development in English education is the growth of support for thinking seriously about evidence. In the DfE, there was a long battle on this issue that ended suddenly when the new Permanent Secretary arrived and immediately agreed with the appointment of Ben Goldacre to do a review of the Department’s handling of evidence, research, and data, which was published in 2013. (I have written many critical things about officials, such as HERE, so it is worth noting that Wormald, and other officials particularly younger ones, took this enlightened view.) There is no doubt that the culture inside the Department changed as a result though there is a very long way to go in this area and it is reasonable to be doubtful about any of the three parties’ commitment to this approach and about civil service commitment. Tom Bennett’s efforts with ResearchEd have been fantastic and are one of the most hopeful things I’ve seen since 2007. There is also now a discussion about a possible College of Teachers – an institution that will only be credible if it has high standards on the subject of cargo cult research. Unsurprisingly, therefore, more people are starting to ask: what do we know about standards? (E.g. Sam Freedman recently blogged on it.)

I therefore thought I would jot down in a series of blogs various bits of evidence, history, thoughts, discussions and so on that I have accumulated since 2007.

Five broad areas

This series of blogs will consider inter alia these questions grouped in five rough areas (which may change as I go along).

A. What is the evidence concerning ‘standards in English schools’? What was the effect of the introduction of 1) GCSEs and 2) the National Curriculum with its connected testing regime? What were the cascading effects on A Levels and higher education? What do comparisons with international tests and other academic studies tell us? What do subject associations and organisations such as the Royal Society say? What do universities and subject experts say?

B How should ‘ability’ and ‘standards’ be defined? What undermines sensible discussion about this?

C. What was Gove’s team trying to do 2010-14? How effective were reforms concerning the curriculum, exams, and accountability (including the role of Ofsted)? What lessons might be learned from the period 2010-15? What incentives dominate the system now?

D. What should come next? What can we reasonably infer from the period since 1985 about what is very unlikely to work? What should the parties not put in their manifestos? What are the main reasons why political and policy discussion of this subject has been so controversial? How does the transformation of the technological landscape since the mid-1980s change arguments? How could a focus on evidence and empiricism help improve the system?

E. What materials are there that can be used by schools that are focused more on education and learning than the official accountability system?

The goal

The goal of these blogs is not to ‘defend the Gove reforms’. When I get onto them, I will try to explain as clearly as I can why we tried to do certain things and what went wrong.  GCSE reform (along with the disaster of Ofsted) is arguably the biggest failure of our team and therefore particularly needs analysis. The goal is not to affect party manifestos – it is possible but unlikely that someone reading this may be able to nudge things off a party or bureaucratic agenda. It is reasonable to assume that whatever the parties promise their plans will crumble on contact with reality. My main hope is that people outside SW1 at the coalface of education take matters into their own hands and develop their own approaches to scientific experimentation with the curriculum, exams, and training.

In my opinion, the only real hope for large improvements in learning is if 1) a critical mass of people become convinced of the need for an empirical approach and the rejection of ‘cargo cult science’ that has dominated education, and 2) an empirical programme emerges that iteratively a) tests what children of different abilities can learn and b) uses this information to alter curricula, tests, and teacher training. We need experiments and Grand Prizes in education that have brought dramatic breakthroughs in other areas, such as DARPA’s Grand Challenge that led to breakthroughs in basic science and then to driverless cars. Imagine what well-defined Grand Challenges could bring to English schools.

Improvements in education do not need to be justified as goals with reference to other things such as economic growth. Learning and education are fundamental aspects of being human. However, it is obvious that humans will have to grapple with profound challenges over the next thirty years. The population will grow by another few billion, mainly in cities and connected to the mobile internet and ‘the internet of things’. Energy and other resource demands will put the global system under huge pressure. We face old security threats like nuclear weapons and new threats such as the use of genetic engineering techniques empowering garage bio-hackers, for good and evil. For example, the revolutionary genome ‘cut and paste’ engineering tool, CRISPR, may soon be used to ‘de-extinct’ species and eradicate diseases but the same techniques could be used destructively. Much progress in machine intelligence and robotics is being driven by research controlled by militaries and intelligence agencies but little research is done on the profound dangers.

If we are to cope with these things, we will need new technologies, new institutions, and new ideas. Improving our education system is therefore obviously central. I have proposed that it ought to become the central organising principle for the British state, as an answer to Dean Acheson’s famous quip that Britain had failed to find a post-imperial role.

Hopefully the discussion of standards in English schools will be useful regardless of whether you agree with this broader argument or not.

Please leave comments, corrections, research reports, complaints etc below. I will add things people leave as I go along and at the end try to produce something short and rigorous…