On the referendum #24K: Observer fake news and a suggestion — I and the MPs discuss this with all of us under oath

Very short blog re the Observer story…

As usual the news these days is very confusing.

I have never refused to give evidence to MPs.

I offered to give evidence to MPs in writing.

Collins refused to discuss a suitable date, insisted on a date he knew I could not do, and chose to issue a formal summons instead of figure out when I could come.

I therefore said that his Committee was plainly behaving unreasonably, that I would not attend it, but I would consider speaking to another Committee:

‘If another Committee behaves reasonably and I can give evidence without compromising various legal actions then I will consider it. Once these legal actions have finished, presumably this year, it will be easy to arrange if someone else wants to do it.’ HERE.

Since I wrote that, MPs have not taken me up on their offer. Collins’ Committee has continued to spread fake news across the internet. Collins has misled Parliament about Vote Leave.

If MPs want to hear what actually happened — from AIQ to Cambridge Analytica to BeLeave — all they have to do is ask and I will be VERY happy to talk to them. Surely there is a Committee there that can make time if MPs are really so desperate to know? The Collins Committee is only into ‘fake news’ and data whereas these issues in many ways are much more fundamental than that and touch on many committees. They could even form an ad hoc Special Committee — and I don’t mind if they stack it with all Remainers if they like, as they have with Collins’ Committee. Collins could be on it too.

There is only one complicating factor. The Electoral Commission told me last week that I have never been investigated by them. This will seem astonishing given their report essentially is a claim that although they cannot provide any evidence for it I somehow must have devised a ‘common plan’ with Grimes. They have concluded this without ever investigating me or interviewing me and after rejecting my offer to talk to them. Further, the EC says they have referred ‘others’ to the police as well as the VL ‘responsible person’. I am told this refers to me. But the EC is under legal obligations to investigate people before they refer them to the police and they have not investigated me, according to them. Baffling…

If the EC confirms that they have not referred me to the police, and my only future involvement is therefore as a witness in the VL/Grimes appeals and further investigations, then as far as I understand it there is no barrier to me giving evidence to Parliament.

I would also be happy to do this UNDER OATH which the whistleblowers have not done — and THE MPs SHOULD ALSO BE UNDER OATH and therefore obliged not to say things they know, or reasonably should know, are false. Collins could therefore be on this Committee but he could not repeat falsehoods. Ditto Bradshaw, Lammy, Grieve or any of the rest who tell all sorts of lies about the referendum. Wouldn’t it be an improvement over the usual Parliamentary process for everybody there to be under an obligation to tell the truth, particularly at a time when so many in public life lie as a matter of course?! We’d all be forced to up our game for many hours on end and avoid lots of the usual silliness in such events. This could only be a good thing.

Also note that the whistleblowers have provably lied — so much that the Observer has been forced to delete many of their original allegations from their website and forced Carole to delete many of her tweets and the Observer has coughed up substantial legal fees (not mine, I have not hired any lawyers or threatened any hacks with lawyers).

The normal pattern is for people to say ‘Cummings is bluffing’. When people say this it always turns out I wasn’t. Remember last year when I said the EC had written to VL saying we could make donations? Carole and @Jolyon said I was lying/bluffing for a year. What happened? The documents were produced in the High Court as part of the judicial review: I wasn’t lying and I wasn’t bluffing, though much of the media has not realised this fact and still reports fake news. I don’t bluff when my bluff will obviously be called.

If the EC confirms they have not referred me to the police — which logically they cannot have done given they have never investigated me (unless they are lying which surely is extremely improbable?) — and MPs get in touch to fix a date I will be more than happy to answer every question any of them have.

I will blog further about this weird affair this week.

On the referendum #27: Banks, Russia, conspiracies and Vote Leave

Dear Tory MPs, ministers, donors and peers who supported the January 2016 coup against Vote Leave…

Remember how I and Victoria Woodcock told you repeatedly Banks was not someone who should play a significant role, that his conduct would destroy the credibility of an official campaign, and a ‘unified campaign’ with him would be a ‘total disaster’?

Remember how I and Victoria Woodcock told you repeatedly that he could not be trusted?

Remember how in horrific meeting after horrific meeting you said that we didn’t understand politics and we needed to ‘unite’ and ‘use his social media operation cos he’s got hundreds of thousands of Likes’?

Remember how we clutched our heads and said ‘Facebook doesn’t work like that, he’s spinning you all bullshit, the media will sink the whole campaign if Banks is involved and we refuse to contemplate it’?

Remember how you then tried to engineer the coup, partly also because Banks had told so many of you (cunningly) that the most important factor in winning was ‘you must represent us in the debates on stage with Nigel in front of millions’ and ‘we need your experience, not all these kids Cummings has hired’?

Of course it’s true that the Remain Establishment are doing whatever they can to discredit the referendum, the Observer has invented stuff for two years (including loony conspiracy theories about Banks, me, Mercer, AIQ, Russia etc), and Banks’s actual role in the 10 week campaign was trivial other than causing us embarrassment. Yes it’s true that Banks was a net drag on the result and we’d have won by more if he’d been dropped down one of his defunct mines in summer 2015 and the effort wasted dealing with him had been spent making Vote Leave much stronger earlier. From grassroots to digital, everything would have happened earlier, bigger and better but for that debilitating distraction which meant VL staff had to fight Banks and the entire Establishment simultaneously.

But all that does not change how close you all came to destroying our chance of winning by putting him in charge of the whole thing.

I know you’ll all be wanting to write to those Vote Leave staff who called your bluff and who, unlike you, displayed moral courage under pressure and made many personal sacrifices while you were on the beach or shooting, in order to apologise and thank them personally so here are some of the names of those who told you on 25 January 2016 they would all be out the door in 5 minutes if you persisted and handed power to Banks, and thereby nudged reality down a different branching history:

  • Richard Howell
  • Oliver Lewis
  • Rob Oxley
  • Stephen Parkinson
  • James Starkie
  • Paul Stephenson
  • Jonny Suart
  • Nick Varley
  • Cleo Watson
  • Victoria Woodcock

But for their actions that day, Vote Leave would have been destroyed, Farage and Banks would have run the official campaign with Bill Cash as legal adviser fighting with DD to be on the Today programme, Boris and Gove would have gone on a long holiday rather than flush their reputations down the toilet, Remain would have won 60-40 and Osborne would today be scanning the horizon for the right moment to take over before the 2020 election.

You’re welcome…

Dominic

Ps. Another branching history… If Cameron and Osborne had simply delayed the vote to 2017, Vote Leave would have ceased to exist in spring 2016, Banks and Farage would have been in charge with Cash/DD et al, and Remain would very likely have cruised to victory last year. Our extreme action on and after 25 January only worked because of the time pressure imposed by the Government. Without it, the consensus was, as people said at the time, ‘we’d have a year to rebuild without you and your crazy ideas’. In history books, luck is always underplayed and the talent of individuals is usually overplayed. As I’ve said many times, Vote Leave could only win because the Establishment’s OODA loops are broken — as the Brexit negotiations painfully demonstrate daily — and they are systematically bad at decisions, and this created just enough space for us to win.

Pps. Although Banks and Leave.EU HQ were hopeless, many of its volunteers did great work and ignored ‘the horror, the horror’ in London among the egomaniacs. Although Farage told them not to help VL post-designation, most of them ignored him and did help us (see comments below).

‘Politics is a job that can really only be compared with navigation in uncharted waters. One has no idea how the weather or the currents will be or what storms one is in for. In politics, there is the added fact that one is largely dependent on the decisions of others, decisions on which one was counting and which then do not materialise; one’s actions are never completely one’s own. And if the friends on whose support one is relying change their minds, which is something that one cannot vouch for, the whole plan miscarries… One’s enemies one can count on – but one’s friends!’ Otto von Bismarck.

‘Everything in war is very simple, but the simplest thing is difficult. The difficulties accumulate and end by producing a kind of friction that is inconceivable unless one has experienced war… Countless minor incidents – the kind you can never really foresee – combine to lower the general level of performance, so that one always falls short of the intended goal.  Iron will-power can overcome this friction … but of course it wears down the machine as well… Friction is the only concept that … corresponds to the factors that distinguish real war from war on paper.  The … army and everything else related to it is basically very simple and therefore seems easy to manage. But … each part is composed of individuals, every one of whom retains his potential of friction… This tremendous friction … is everywhere in contact with chance, and brings about effects that cannot be measured… Friction … is the force that makes the apparently easy so difficult… Finally … all action takes place … in a kind of twilight, which like fog or moonlight, often tends to make things seem grotesque and larger than they really are.  Whatever is hidden from full view in this feeble light has to be guessed at by talent, or simply left to chance.’ Clausewitz.

 

Effective action #4b: ‘Expertise’, prediction and noise, from the NHS killing people to Brexit

In part A I looked at extreme sports as some background to the question of true expertise and the crucial nature of fast high quality feedback.

This blog looks at studies comparing expertise in many fields over decades, including work by Tetlock and Kahneman, and problems like — why people don’t learn to use even simple tools to stop children dying unnecessarily. There is a summary of some basic lessons at the end.

The reason for writing about this is that we will only improve the performance of government (at individual, team and institutional levels) if we reflect on:

  • what expertise really is and why do some very successful fields cultivate it effectively while others, like government, do not;
  • how to select much higher quality people (it’s insane people as ignorant and limited as me can have the influence we do in the way we do — us limited duffers can help in limited ways but why do we deliberately exclude ~100% of the most intelligent, talented, relentless, high performing people from fields with genuine expertise, why do we not have people like Fields Medallist Tim Gowers or Michael Nielsen as Chief Scientist  sitting ex officio in Cabinet?);
  • how to train people effectively to develop true expertise in skills relevant to government: it needs different intellectual content (PPE/economics are NOT good introductory degrees) and practice in practical skills (project management, making predictions and in general ‘thinking rationally’) with lots of fast, accurate feedback;
  • how to give them effective tools: e.g the Cabinet Room is worse in this respect than it was in July 1914 — at least then the clock and fireplace worked, and Lord Salisbury in the 1890s would walk round the Cabinet table gathering papers to burn in the grate — while today No10 is decades behind the state-of-the-art in old technologies like TV, doesn’t understand simple tools like checklists, and is nowhere with advanced technologies;
  • and how to ‘program’ institutions differently so that 1) people are more incentivised to optimise things we want them to optimise, like error-correction and predictive accuracy, and less incentivised to optimise bureaucratic process, prestige, and signalling as our institutions now do to a dangerous extent, and, connected, so that 2) institutions are much better at building high performance teams rather than continue normal rules that make this practically illegal, and so that 3) we have ‘immune systems’ to minimise the inevitable failures of even the best people and teams .

In SW1 now, those at the apex of power practically never think in a serious way about the reasons for the endemic dysfunctional decision-making that constitutes most of their daily experience or how to change it. What looks like omnishambles to the public and high performers in technology or business is seen by Insiders, always implicitly and often explicitly, as ‘normal performance’. ‘Crises’ such as the collapse of Carillion or our farcical multi-decade multi-billion ‘aircraft carrier’ project occasionally provoke a few days of headlines but it’s very rare anything important changes in the underlying structures and there is no real reflection on system failure.

This fact is why, for example, a startup created in a few months could win a referendum that should have been unwinnable. It was the systemic and consistent dysfunction of Establishment decision-making systems over a long period, with very poor mechanisms for good accurate feedback from reality, that created the space for a guerrilla operation to exploit.

This makes it particularly ironic that even after Westminster and Whitehall have allowed their internal consensus about UK national strategy to be shattered by the referendum, there is essentially no serious reflection on this system failure. It is much more psychologically appealing for Insiders to blame ‘lies’ (Blair and Osborne really say this without blushing), devilish use of technology to twist minds and so on. Perhaps the most profound aspect of broken systems is they cannot reflect on the reasons why they’re broken  — never mind take effective action. Instead of serious thought, we have high status Insiders like Campbell reduced to bathos with whining on social media about Brexit ‘impacting mental health’. This lack of reflection is why Remain-dominated Insiders lurched from failure over the referendum to failure over negotiations. OODA loops across SW1 are broken and this is very hard to fix — if you can’t orient to reality how do you even see your problem well? (NB. It should go without saying that there is a faction of pro-Brexit MPs, ‘campaigners’ and ‘pro-Brexit economists’ who are at least as disconnected from reality, often more, as the May/Hammond bunker.)

Screenshot 2018-06-05 10.05.19

In the commercial world, big companies mostly die within a few decades because they cannot maintain an internal system to keep them aligned to reality plus startups pop up. These two factors create learning at a system level — there is lots of micro failure but macro productivity/learning in which useful information is compressed and abstracted. In the political world, big established failing systems control the rules, suck in more and more resources rather than go bust, make it almost impossible for startups to contribute and so on. Even failures on the scale of the 2008 Crash or the 2016 referendum do not necessarily make broken systems face reality, at least quickly. Watching Parliament’s obsession with trivia in the face of the Cabinet’s and Whitehall’s contemptible failure to protect the interests of millions in the farcical Brexit negotiations is like watching the secretary to the Singapore Golf Club objecting to guns being placed on the links as the Japanese troops advanced.

Neither of the main parties has internalised the reality of these two crises. The Tories won’t face reality on things like corporate looting and the NHS, Labour won’t face reality on things like immigration and the limits of bureaucratic centralism. Neither can cope with the complexity of Brexit and both just look like I would look like in the ring with a professional fighter — baffled, terrified and desperate for a way to escape. There are so many simple ways to improve performance — and their own popularity! — but the system is stuck in such a closed loop it wilfully avoids seeing even the most obvious things and suppresses Insiders who want to do things differently…

But… there is a network of almost entirely younger people inside or close to the system thinking ‘we could do so much better than this’. Few senior Insiders are interested in these questions but that’s OK — few of them listened before the referendum either. It’s not the people now in power and running the parties and Whitehall who will determine whether we make Brexit a platform to contribute usefully to humanity’s biggest challenges but those that take over.

Doing better requires reflecting on what we know about real expertise…

*

How to distinguish between fields dominated by real expertise and those dominated by confident ‘experts’ who make bad predictions?

We know a lot about the distinction between fields in which there is real expertise and fields dominated by bogus expertise. Daniel Kahneman, who has published some of the most important research about expertise and prediction, summarises the two fundamental tests to ask about a field: 1) is there enough informational structure in the environment to allow good predictions, and 2) is there timely and effective feedback that enables error-correction and learning.

‘To know whether you can trust a particular intuitive judgment, there are two questions you should ask: Is the environment in which the judgment is made sufficiently regular to enable predictions from the available evidence? The answer is yes for diagnosticians, no for stock pickers. Do the professionals have an adequate opportunity to learn the cues and the regularities? The answer here depends on the professionals’ experience and on the quality and speed with which they discover their mistakes. Anesthesiologists have a better chance to develop intuitions than radiologists do. Many of the professionals we encounter easily pass both tests, and their off-the-cuff judgments deserve to be taken seriously. In general, however, you should not take assertive and confident people at their own evaluation unless you have independent reason to believe that they know what they are talking about.’ (Emphasis added.)

In fields where these two elements are present there is genuine expertise and people build new knowledge on the reliable foundations of previous knowledge. Some fields make a transition from stories (e.g Icarus) and authority (e.g ‘witch doctor’) to quantitative models (e.g modern aircraft) and evidence/experiment (e.g some parts of modern medicine/surgery). As scientists have said since Newton, they stand on the shoulders of giants.

How do we assess predictions / judgement about the future?

‘Good judgment is often gauged against two gold standards – coherence and correspondence. Judgments are coherent if they demonstrate consistency with the axioms of probability theory or propositional logic. Judgments are correspondent if they agree with ground truth. When gold standards are unavailable, silver standards such as consistency and discrimination can be used to evaluate judgment quality. Individuals are consistent if they assign similar judgments to comparable stimuli, and they discriminate if they assign different judgments to dissimilar stimuli.

‘Coherence violations range from base rate neglect and confirmation bias to overconfidence and framing effects (Gilovich, Griffith & Kahneman, 2002; Kahneman, Slovic & Tversky, 1982). Experts are not immune. Statisticians (Christensen-Szalanski & Bushyhead, 1981), doctors (Eddy, 1982), and nurses (Bennett, 1980) neglect base rates. Physicians and intelligence professionals are susceptible to framing effects and financial investors are prone to overconfidence.

‘Research on correspondence tells a similar story. Numerous studies show that human predictions are frequently inaccurate and worse than simple linear models in many domains (e.g. Meehl, 1954; Dawes, Faust & Meehl, 1989). Once again, expertise doesn’t necessarily help. Inaccurate predictions have been found in parole officers, court judges, investment managers in the US and Taiwan, and politicians. However, expert predictions are better when the forecasting environment provides regular, clear feedback and there are repeated opportunities to learn (Kahneman & Klein, 2009; Shanteau, 1992). Examples include meteorologists, professional bridge players, and bookmakers at the racetrack, all of whom are well-calibrated in their own domains.‘ (Tetlock, How generalizable is good judgment?, 2017.)

In another 2017 piece Tetlock explored the studies furtherIn the 1920s researchers built simple models based on expert assessments of 500 ears of corn and the price they would fetch in the market. They found that ‘to everyone’s surprise, the models that mimicked the judges’ strategies nearly always performed better than the judges themselves’ (Tetlock, cf. ‘What Is in the Corn Judge’s Mind?’, Journal of American Society for Agronomy, 1923). Banks found the same when they introduced models for credit decisions.

‘In other fields, from predicting the performance of newly hired salespeople to the bankruptcy risks of companies to the life expectancies of terminally ill cancer patients, the experience has been essentially the same. Even though experts usually possess deep knowledge, they often do not make good predictions

When humans make predictions, wisdom gets mixed with “random noise.”… Bootstrapping, which incorporates expert judgment into a decision-making model, eliminates such inconsistencies while preserving the expert’s insights. But this does not occur when human judgment is employed on its own…

In fields ranging from medicine to finance, scores of studies have shown that replacing experts with models of experts produces superior judgments. In most cases, the bootstrapping model performed better than experts on their own. Nonetheless, bootstrapping models tend to be rather rudimentary in that human experts are usually needed to identify the factors that matter most in making predictions. Humans are also instrumental in assigning scores to the predictor variables (such as judging the strength of recommendation letters for college applications or the overall health of patients in medical cases). What’s more, humans are good at spotting when the model is getting out of date and needs updating…

Human experts typically provide signal, noise, and bias in unknown proportions, which makes it difficult to disentangle these three components in field settings. Whether humans or computers have the upper hand depends on many factors, including whether the tasks being undertaken are familiar or unique. When tasks are familiar and much data is available, computers will likely beat humans by being data-driven and highly consistent from one case to the next. But when tasks are unique (where creativity may matter more) and when data overload is not a problem for humans, humans will likely have an advantage…

One might think that humans have an advantage over models in understanding dynamically complex domains, with feedback loops, delays, and instability. But psychologists have examined how people learn about complex relationships in simulated dynamic environments (for example, a computer game modeling an airline’s strategic decisions or those of an electronics company managing a new product). Even after receiving extensive feedback after each round of play, the human subjects improved only slowly over time and failed to beat simple computer models. This raises questions about how much human expertise is desirable when building models for complex dynamic environments. The best way to find out is to compare how well humans and models do in specific domains and perhaps develop hybrid models that integrate different approaches.‘ (Tetlock)

Kahneman also recently published new work relevant to this.

Research has confirmed that in many tasks, experts’ decisions are highly variable: valuing stocks, appraising real estate, sentencing criminals, evaluating job performance, auditing financial statements, and more. The unavoidable conclusion is that professionals often make decisions that deviate significantly from those of their peers, from their own prior decisions, and from rules that they themselves claim to follow.’

In general organisations spend almost no effort figuring out how noisy the predictions made by senior staff are and how much this costs. Kahneman has done some ‘noise audits’ and shown companies that management make MUCH more variable predictions than people realise.

‘What prevents companies from recognizing that the judgments of their employees are noisy? The answer lies in two familiar phenomena: Experienced professionals tend to have high confidence in the accuracy of their own judgments, and they also have high regard for their colleagues’ intelligence. This combination inevitably leads to an overestimation of agreement. When asked about what their colleagues would say, professionals expect others’ judgments to be much closer to their own than they actually are. Most of the time, of course, experienced professionals are completely unconcerned with what others might think and simply assume that theirs is the best answer. One reason the problem of noise is invisible is that people do not go through life imagining plausible alternatives to every judgment they make.

‘High skill develops in chess and driving through years of practice in a predictable environment, in which actions are followed by feedback that is both immediate and clear. Unfortunately, few professionals operate in such a world. In most jobs people learn to make judgments by hearing managers and colleagues explain and criticize—a much less reliable source of knowledge than learning from one’s mistakes. Long experience on a job always increases people’s confidence in their judgments, but in the absence of rapid feedback, confidence is no guarantee of either accuracy or consensus.’

Reviewing the point that Tetlock makes about simple models beating experts in many fields, Kahneman summarises the evidence:

‘People have competed against algorithms in several hundred contests of accuracy over the past 60 years, in tasks ranging from predicting the life expectancy of cancer patients to predicting the success of graduate students. Algorithms were more accurate than human professionals in about half the studies, and approximately tied with the humans in the others. The ties should also count as victories for the algorithms, which are more cost-effective…

‘The common assumption is that algorithms require statistical analysis of large amounts of data. For example, most people we talk to believe that data on thousands of loan applications and their outcomes is needed to develop an equation that predicts commercial loan defaults. Very few know that adequate algorithms can be developed without any outcome data at all — and with input information on only a small number of cases. We call predictive formulas that are built without outcome data “reasoned rules,” because they draw on commonsense reasoning.

‘The construction of a reasoned rule starts with the selection of a few (perhaps six to eight) variables that are incontrovertibly related to the outcome being predicted. If the outcome is loan default, for example, assets and liabilities will surely be included in the list. The next step is to assign these variables equal weight in the prediction formula, setting their sign in the obvious direction (positive for assets, negative for liabilities). The rule can then be constructed by a few simple calculations.

The surprising result of much research is that in many contexts reasoned rules are about as accurate as statistical models built with outcome data. Standard statistical models combine a set of predictive variables, which are assigned weights based on their relationship to the predicted outcomes and to one another. In many situations, however, these weights are both statistically unstable and practically unimportant. A simple rule that assigns equal weights to the selected variables is likely to be just as valid. Algorithms that weight variables equally and don’t rely on outcome data have proved successful in personnel selection, election forecasting, predictions about football games, and other applications.

‘The bottom line here is that if you plan to use an algorithm to reduce noise, you need not wait for outcome data. You can reap most of the benefits by using common sense to select variables and the simplest possible rule to combine them…

‘Uncomfortable as people may be with the idea, studies have shown that while humans can provide useful input to formulas, algorithms do better in the role of final decision maker. If the avoidance of errors is the only criterion, managers should be strongly advised to overrule the algorithm only in exceptional circumstances.

Jim Simons is a mathematician and founder of the world’s most successful ‘quant fund’, Renaissance Technologies. While market prices appear close to random and are therefore extremely hard to predict, they are not quite random and the right models/technology can exploit these small and fleeting opportunities. One of the lessons he learned early was: Don’t turn off the model and go with your gut. At Renaissance, they trust models over instincts. The Bridgewater hedge fund led by Ray Dalio is similar. After near destruction early in his career, Dalio explicitly turned towards explicit model building as the basis for decisions combined with radical attempts to create an internal system that incentivises the optimisation of error-correction. It works.

*

People fail to learn from even the great examples of success and the simplest lessons

One of the most interesting meta-lessons of studying high performance, though, is that simply demonstrating extreme success does NOT lead to much learning. For example:

  • ARPA and PARC created the internet and PC. The PARC research team was an extraordinary collection of about two dozen people who were managed in a very unusual way that created super-productive processes extremely different to normal bureaucracies. XEROX, which owned PARC, had the entire future of the computer industry in its own hands, paid for by its own budgets, and it simultaneously let Bill Gates and Steve Jobs steal everything and XEROX then shut down the research team that did it. And then, as Silicon Valley grew on the back of these efforts, almost nobody, including most of the billionaires who got rich from the dynamics created by ARPA-PARC, studied the nature of the organisation and processes and copied it. Even today, those trying to do edge-of-the-art research in a similar way to PARC right at the heart of the Valley ecosystem are struggling for long-term patient funding. As Alan Kay, one of the PARC team, said, ‘The most interesting thing has been the contrast between appreciation/exploitation of the inventions/contributions [of PARC] versus the almost complete lack of curiosity and interest in the processes that produced them. ARPA survived being abolished in the 1970s but it was significantly changed and is no longer the freewheeling place that it was in the 1960s when it funded the internet. In many ways DARPA’s approach now is explicitly different to the old ARPA (the addition of the ‘D’ was a sign of internal bureaucratic changes).

Screenshot 2018-06-05 14.55.00

  • ‘Systems management’ was invented in the 1950s and 1960s (partly based on wartime experience of large complex projects) to deal with the classified ICBM project and Apollo. It put man on the moon then NASA largely abandoned the approach and reverted to being (relative to 1963-9) a normal bureaucracy. Most of Washington has ignored the lessons ever since — look for example at the collapse of ObamaCare’s rollout, after which Insiders said ‘oh, looks like it was a system failure, wonder how we deal with this’, mostly unaware that America had developed a successful approach to such projects half a century earlier. This is particularly interesting given that China also studied Mueller’s approach to systems management in Apollo and as we speak is copying it in projects across China. The EU’s bureaucracy is, like Whitehall, an anti-checklist to high level systems management — i.e they violate almost every principle of effective action.
  • Buffett and Munger are the most successful investment partnership in world history. Every year for half a century they have explained some basic principles, particularly concerning incentives, behind organisational success. Practically no public companies take their advice and all around us in Britain we see vast corporate looting and politicians of all parties failing to act — they don’t even read the Buffett/Munger lessons and think about them. Even when given these lessons to read, they won’t read them (I know this because I’ve tried).

Perhaps you’re thinking — well, learning from these brilliant examples might be intrinsically really hard, much harder than Cummings thinks. I don’t think this is quite right. Why? Partly because millions of well-educated and normally-ethical people don’t learn even from much simpler things.

I will explore this separately soon but I’ll give just one example. The world of healthcare unnecessarily kills and injures people on a vast scale. Two aspects of this are 1) a deep resistance to learning from the success of very simple tools like checklists and 2) a deep resistance to face the fact that most medical experts do not understand statistics properly and their routine misjudgements cause vast suffering, plus warped incentives encourage widespread lies about statistics and irrational management. E.g People are constantly told things like ‘you’ve tested positive for X therefore you have X’ and they then kill themselves. We KNOW how to practically eliminate certain sorts of medical injury/death. We KNOW how to teach and communicate statistics better. (Cf. Professor Gigerenzer for details. He was the motivation for including things like conditional probabilities in the new National Curriculum.) These are MUCH simpler than building ICBMs, putting man on the moon, creating the internet and PC, or being great investors. Yet our societies don’t do them.

Why?

Because we do not incentivise error-correction and predictive accuracy. People are not incentivised to consider the cost of their noisy judgements. Where incentives and culture are changed, performance magically changes. It is the nature of the systems, not (mostly) the nature of the people, that is the crucial ingredient in learning from proven simple success. In healthcare like in government generally, people are incentivised to engage in wasteful/dangerous signalling to a terrifying degree — not rigorous thinking and not solving problems.

I have experienced the problem with checklists first hand in the Department for Education when trying to get the social worker bureaucracy to think about checklists in the context of avoiding child killings like Baby P. Professionals tend to see them as undermining their status and bureaucracies fight against learning, even when some great officials try really hard (as some in the DfE did such as Pamela Dow and Victoria Woodcock). ‘Social work is not the same as an airline Dominic’. No shit. Airlines can handle millions of people without killing one of them because they align incentives with predictive accuracy and error-correction.

Some appalling killings are inevitable but the social work bureaucracy will keep allowing unnecessary killings because they will not align incentives with error-correction. Undoing flawed incentives threatens the system so they’ll keep killing children instead — and they’re not particularly bad people, they’re normal people in a normal bureaucracy. The pilot dies with the passengers. The ‘CEO’ on over £150,000 a year presiding over another unnecessary death despite constantly increasing taxpayers money pouring in? Issue a statement that ‘this must never happen again’, tell the lawyers to redact embarrassing cockups on the grounds of ‘protecting someone’s anonymity’ (the ECHR is a great tool to cover up death by incompetence), fuck off to the golf course, and wait for the media circus to move on.

Why do so many things go wrong? Because usually nobody is incentivised to work relentlessly to suppress entropy, never mind come up with something new.

*

We can see some reasonably clear conclusions from decades of study on expertise and prediction in many fields.

  • Some fields are like extreme sport or physics: genuine expertise emerges because of fast effective feedback on errors.
  • Abstracting human wisdom into models often works better than relying on human experts as models are often more consistent and less noisy.
  • Models are also often cheaper and simpler to use.
  • Models do not have to be complex to be highly effective — quite the opposite, often simpler models outperform more sophisticated and expensive ones.
  • In many fields (which I’ve explored before but won’t go into again here) low tech very simple checklists have been extremely effective: e.g flying aircraft or surgery.
  • Successful individuals like Warren Buffett and Ray Dalio also create cognitive checklists to trap and correct normal cognitive biases that degrade individual and team performance.
  • Fields make progress towards genuine expertise when they make a transition from stories (e.g Icarus) and authority (e.g ‘witch doctor’) to quantitative models (e.g modern aircraft) and evidence/experiment (e.g some parts of modern medicine/surgery).
  • In the intellectual realm, maths and physics are fields dominated by genuine expertise and provide a useful benchmark to compare others against. They are also hierarchical. Social sciences have little in common with this.
  • Even when we have great examples of learning and progress, and we can see the principles behind them are relatively simple and do not require high intelligence to understand, they are so psychologically hard and run so counter to the dynamics of normal big organisations, that almost nobody learns from them. Extreme success is ‘easy to learn from’ in one sense and ‘the hardest thing in the world to learn from’ in another sense.

It is fascinating how remarkably little interest there is in the world of politics/government, and social sciences analysing politics/government, about all this evidence. This is partly because politics/government is an anti-learning and anti-expertise field, partly because the social sciences are swamped by what Feynman called ‘cargo cult science’ with very noisy predictions, little good feedback and learning, and a lot of chippiness at criticism whether it’s from statistics experts or the ‘ignorant masses’. Fields like ‘education research’ and ‘political science’ are particularly dreadful and packed with charlatans but much of economics is not much better (much pro- and anti-Brexit mainstream economics is classic ‘cargo cult’).

I have found there is overwhelmingly more interest in high technology circles than in government circles, but in high technology circles there is also a lot of incredulity and naivety about how government works — many assume politicians are trying and failing to achieve high performance and don’t realise that in fact nobody is actually trying. This illusion extends to many well-connected businessmen who just can’t internalise the reality of the apex of power. I find that uneducated people on 20k living hundreds of miles from SW1 generally have a more accurate picture of daily No10 work than extremely well-connected billionaires.

This is all sobering and is another reason to be pessimistic about the chances of changing government from ‘normal’ to ‘high performance’ — but, pessimism of the intellect, optimism of the will…

If you are in Whitehall now watching the Brexit farce or abroad looking at similar, you will see from page 26 HERE a checklist for how to manage complex government projects at world class levels (if you find this interesting then read the whole paper). I will elaborate on this. I am also thinking about a project to look at the intersection of (roughly) five fields in order to make large improvements in the quality of people, ideas, tools, and institutions that determine political/government decisions and performance:

  • the science of prediction across different fields (e.g early warning systems, the Tetlock/IARPA project showing dramatic performance improvements),
  • what we know about high performance (individual/team/organisation) in different fields (e.g China’s application of ‘systems management’ to government),
  • technology and tools (e.g Bret Victor’s work, Michael Nielsen’s work on cognitive technologies, work on human-AI ‘minotaur’ teams),
  • political/government decision making affecting millions of people and trillions of dollars (e.g WMD, health), and
  • communication (e.g crisis management, applied psychology).

Progress requires attacking the ‘system of systems’ problem at the right ‘level’. Attacking the problems directly — let’s improve policy X and Y, let’s swap ‘incompetent’ A for ‘competent’ B — cannot touch the core problems, particularly the hardest meta-problem that government systems bitterly fight improvement. Solving the explicit surface problems of politics and government is best approached by a more general focus on applying abstract principles of effective action. We need to surround relatively specific problems with a more general approach. Attack at the right level will see specific solutions automatically ‘pop out’ of the system. One of the most powerful simplicities in all conflict (almost always unrecognised) is: ‘winning without fighting is the highest form of war’. If we approach the problem of government performance at the right level of generality then we have a chance to solve specific problems ‘without fighting’ — or, rather, without fighting nearly so much and the fighting will be more fruitful.

This is not a theoretical argument. If you look carefully at ancient texts and modern case studies, you see that applying a small number of very simple, powerful, but largely unrecognised principles (that are very hard for organisations to operationalise) can produce extremely surprising results.

How to jump from the Idea to Reality? More soon…


Ps. Just as I was about to hit publish on this, the DCMS Select Committee released their report on me. The sentence about the Singapore golf club at the top comes to mind.

On the referendum #24J: Collins, grandstanding, empty threats & the plan for a rematch against the public

The DCMS Select Committee has just sent me the following letter.

Screenshot 2018-05-24 13.51.14

Here is my official reply…

Dear Damian et al

As you know I agreed to give evidence.

In April, I told you I could not do the date you suggested. On 12 April I suggested July.

You ignored this for weeks.

On 3 May you asked again if I could do a date I’d already said I could not do.

I replied that, as I’d told you weeks earlier, I could not.

You then threatened me with a Summons.

On 10 May, Collins wrote:

Dear Dominic

We have offered you different dates, and as I said previously we are not prepared to wait until July for you to give evidence to the committee. We have also discussed this with the Electoral Commission who have no objection to you giving evidence to us.

We are asking you to give evidence to the committee following evidence we have received that relates to the work of Vote Leave. We have extended a similar invitiation to Arron Banks and Andy Wigmore, to respond to evidence we have received about Leave.EU, and they have both agreed to attend.

The committee will be sending you a summons to appear and I hope that you are able to respond positively to this

best wishes

I replied:

The EC has NOT told me this.

Sending a summons is the behaviour of people looking for PR, not people looking to get to the bottom of this affair.

A summons will have ZERO positive impact on my decision and is likely only to mean I withdraw my offer of friendly cooperation, given you will have shown greater interest in grandstanding than truth-seeking, which is one of the curses of the committee system.

I hope you reconsider and put truth-seeking first.

Best wishes

d

You replied starting this charade.

 

You talk of ‘contempt of Parliament’.

You seem unaware that most of the country feels contempt for Parliament and this contempt is growing.

  • You have failed miserably over Brexit. You have not even bothered to educate yourselves on the basics of ‘what the Single Market is’, as Ivan Rogers explained in detail yesterday.
  • We want £350 million a week for the NHS plus long-term consistent funding and learning from the best systems in the world and instead you funnel our money to appalling companies like the parasites that dominate defence procurement.
  • We want action on unskilled immigration and you give us bullshit promises of ‘tens of thousands’ that you don’t even believe yourselves plus, literally, free movement for murderers, then you wonder why we don’t trust you.
  • We want a country MORE friendly to scientists and people from around the world with skills to offer and you give us ignorant persecution that is making our country a bad joke.
  • We want you to take money away from corporate looters (who fund your party) and fund science research so we can ‘create the future’, and you give us Carillion and joke aircraft carriers.
  • We want to open government to the best people and ideas in the world and you keep it a closed dysfunctional shambles that steals our money and keeps power locked within two useless parties and a closed bureaucracy that excludes ~100% of the most talented people. We want real expertise and you don’t even think about what that means.
  • You spend your time on this sort of grandstanding instead of serving millions of people less fortunate than you and who rely on you.

If you had wanted my evidence you would have cooperated over dates.

You actually wanted to issue threats, watch me give in, then get higher audiences for your grandstanding.

I’m calling your bluff. Your threats are as empty as those from May/Hammond/DD to the EU. Say what you like, I will not come to your committee regardless of how many letters you send or whether you send characters in fancy dress to hand me papers.

If another Committee behaves reasonably and I can give evidence without compromising various legal actions then I will consider it. Once these legal actions have finished, presumably this year, it will be easy to arrange if someone else wants to do it.

Further, I’m told many of your committee support the Adonis/Mandelson/Campbell/Grieve/Goldman Sachs/FT/CBI campaign for a rematch against the country.

Do you know what Vote Leave 2 would feel like for the MPs who vote for that (and donors who fund it)?

It would feel like having Lawrence Taylor chasing you and smashing you into the ground over and over and over again.

Vote Leave 2 would not involve me — nobody will make that mistake again — but I know what it would feel like for every MP who votes for a rematch against the public.

Lawrence Taylor: relentless 

So far you guys have botched things on an epic scale but it’s hard to break into the Westminster system — you rig the rules to stop competition. Vote Leave 1 needed Cameron’s help to hack the system. If you guys want to run with Adonis and create another wave, be careful what you wish for. ‘Unda fert nec regitur’ and VL2 would ride that wave right at the gates of Westminster.

A second referendum would be bad for the country and I hope it doesn’t happen but if you force the issue, then Vote Leave 2 would try to create out of the smoking wreck in SW1 something that can deliver what the public wants. Imagine Amazon-style obsession on customer satisfaction (not competitor and media obsession which is what you guys know) with Silicon Valley technology/scaling and Mueller-style ‘systems politics’ combined with the wave upon wave of emotion you will have created. Here’s some free political advice: when someone’s inside your OODA loop, it feels to them like you are working for them. If you go for a rematch, then this is what you will be doing for people like me. 350m would just be the starter.

‘Mixed emotions, Buddy, like Larry Wildman going off a cliff — in my new Maserati.’

I will happily discuss this with your colleagues on a different committee if they are interested, after the legal issues are finished…

 

Best wishes

Dominic

Ps. If you’re running an inquiry on fake news, it would be better to stop spreading fake news yourselves and to correct your errors when made aware of them. If you’re running an inquiry on issues entangled with technologies, it would be better to provide yourself with technological expertise so you avoid spreading false memes. E.g your recent letter to Facebook asked them to explain to you the operational decision-making of Vote Leave. This is a meaningless question which it is impossible for Facebook to answer and could only be asked by people who do not understand the technology they are investigating.

On the referendum #25: a letter to Tory MPs & donors on the Brexit shambles

[NB. As has become usual, whenever I write something critical about an aspect of Brexit, Remain-supporting media like the FT/Economist/Guardian etc portray this dishonestly as a general statement about Brexit. So for example, below I say that the Government ‘irretrievably botched’ the process of preparing to be a ‘third country’ under EU law in line with official policy. This has been widely quoted as ‘Brexit is irretrievably botched’. This is not at all my view as I have said many times. The referendum was explicitly presented to the country by Parliament as a ‘choice for a generation’. Whether Brexit is a success will not be determined by the ‘deal’. The deal is now sure to be much worse than it could have been. This means we will start off outside the EU in a state worse than we might have done. But whether we make the most of things over a 10/20/30 year timescale is a completely different question and unknowable to anybody. Ignore the fanatics on both sides who are ‘sure’, from Chris Giles to Bill Cash.]

Dear Tory MPs and donors

I’ve avoided writing about the substance of Brexit and the negotiations since the anniversary last year but a few of you have been in touch recently asking ‘what do you think?’ so…

Vote Leave said during the referendum that:

1) promising to use the Article 50 process would be stupid and the UK should maintain the possibility of making real preparations to leave while NOT triggering Article 50 and

2) triggering Article 50 quickly without discussions with our EU friends and without a plan ‘would be like putting a gun in your mouth and pulling the trigger’. 

Following this advice would have maintained the number of positive branching histories of the future, including a friendly departure under Article 50.

The Government immediately accepted bogus legal advice and triggered Article 50 quickly without discussions with our EU friends and without a plan. This immediately closed many positive branching histories and created major problems. The joy in Brussels was palpable. Hammond and DD responded to this joy with empty sabre rattling which Brussels is now enjoying shoving down their throats.  

The government’s nominal policy, which it put in its manifesto and has repeated many times, is to leave the Single Market and Customs Union and the jurisdiction of the ECJ.

This requires preparing to be a ‘third country’ for the purposes of  EU law. It requires building all the infrastructure and facilities that are normal around the world to manage trade.

This process should have started BEFORE triggering A50 but the government has irretrievably botched this.

Having botched it, it could have partially recovered its blunder by starting to do it afterwards.

No such action has been taken.

Downing Street, the Treasury, the Cabinet Office and the Cabinet have made no such preparations and there is no intention of starting.

The Cabinet has never asked for and never been given a briefing from responsible officials on these preparations. Some of them understand this and are happy (e.g Hammond). Most of them don’t understand this and/or prefer not to think about it. It will be trashed in the history books as the pre-1914 Cabinet has been for its failure to discuss what its military alliance with France actually meant until after it was too late.

The few ministers who try to make preparations are often told ‘it’s illegal’ and are blocked by their own Departments, the Cabinet Office and Treasury. The standard officials device of ‘legal advice’ is routinely deployed to whip cowed ministers and spads into line. But given officials now know the May/Hammond plan is surrender, it’s hardly surprising they are not preparing for a Potemkin policy. 

The Treasury argues, with a logic that is both contemptible and reasonable in the comical circumstances, that given the actual outcome of the negotiations will be abject surrender, it is pointless wasting more money to prepare for a policy that has no future and therefore even the Potemkin preparations now underway should be abandoned (NB. the Chancellor has earmarked half of the money for a ‘no deal’ for the fiscal year after we leave the EU).

Instead, Whitehall’s real preparations are for the continuation of EU law and the jurisdiction of the ECJ. The expectation is that MPs will end up accepting the terrible agreement as voting it down would be to invite chaos.

In short, the state has made no preparations to leave and plans to make no preparations to leave even after leaving.

Further, the Government promised in the December agreement to do a number of things that are logically, legally and practically incompatible including leaving the Single Market and Customs Union, avoiding ‘friction’ and changing nothing around the Irish border (as defined by the EU), and having no border in the Irish Sea.

The Government has also aided and abetted bullshit invented by Irish nationalists and Remain campaigners that the Belfast Agreement prevents reasonable customs checks on trade between Northern Ireland and the Republic. Read the agreement. It does no such thing. This has fatally undermined the UK’s negotiating position and has led to the false choice of not really leaving the EU (‘the Government’s backstop’) or undermining the UK’s constitutional integrity (‘the EU’s backstop’). Barwell promised ministers in December that the text did not mean what it plainly did mean. Now he argues ‘you agreed all this in December’. Whenever you think ‘it can’t be this bad’, the internal processes are always much worse than you think.

Parliament and its Select Committees have contributed to delusions. They have made almost no serious investigation of what preparations to be a third country under EU law should be and what steps are being taken to achieve it.

A small faction of pro-Brexit MPs (which also nearly destroyed Vote Leave so they could babble about ‘Global Britain’ in TV debates) could have done one useful thing — forced the government to prepare for their official policy. Instead this faction has instead spent its time trying to persuade people that all talk of ‘preparations’ is a conspiracy of Brussels and Heywood. They were an asset to Remain in the referendum and they’ve helped sink a viable policy since. A party that treats this faction (or Dominic Grieve) as a serious authority on the law deserves everything it gets. (I don’t mean ‘the ERG’ — I mean a subset of the ERG.)

All this contributes to current delusional arguments over supposed ‘models’ (hybrid/max fac etc) that even on their own terms cannot solve the problem of multiple incompatible promises. ‘Compromise proposals’ such as that from Boles which assume the existence of ‘third country’ planning are just more delusions. It doesn’t matter which version of delusion your gangs finally agree on if none of them has a basis in reality and so long as May/Hammond continue they will have no basis in reality.

You can dance around the fundamental issues all you want but in the end ‘reality cannot be fooled’.

The Government effectively has no credible policy and the whole world knows it. By not taking the basic steps any sane Government should have taken from 24 June 2016, including providing itself with world class legal advice, it’s ‘strategy’ has imploded. It now thinks its survival requires surrender, it thinks that admitting this risks its survival, it thinks that the MPs can be bullshitted by clever drafting from officials, and that once Leave MPs and donors — you guys — are ordering your champagne in the autumn for your parties on 30 March 2019 you will balk at bringing down the Government when you finally have to face that you’ve been conned. Eurosceptics are full of shit and threats they don’t deliver, they say in No10, and on this at least they have a point.

This set of problems cannot be solved by swapping ‘useless X’ for ‘competent Y’ or ‘better spin’.

This set of problems cannot be solved by listening to charlatans such as the overwhelming majority of economists and ‘trade experts’ who brand themselves pro-Brexit, live in parallel universes, and spin fantasies to you.

This set of problems derives partly from the fact that the wiring of power in Downing Street is systemically dysfunctional and, worse, those with real institutional power (Cabinet Office/HMT officials etc) have as their top priority the maintenance of this broken system and keeping Britain as closely tied to the EU as possible. There is effectively zero prospect of May’s team, totally underwater, solving these problems not least because they cannot see them — indeed, their only strategy is to ‘trust officials to be honest’, which is like trusting Bernie Madoff with your finances. Brexit cannot be done with the traditional Westminster/Whitehall system as Vote Leave  warned repeatedly before 23 June 2016.

Further, lots of what Corbyn says is more popular than what Tory think tanks say and you believe (e.g nationalising the trains and water companies that have been run by corporate looters who Hammond says ‘we must defend’). You are only at 40% in the polls because a set of UKIP voters has decided to back you until they see how Brexit turns out. You only survived the most useless campaign in modern history because Vote Leave killed UKIP. You’re now acting like you want someone to create a serious version of it.

Ask yourselves: what happens when the country sees you’ve simultaneously a) ‘handed over tens of billions for fuck all’ as they’ll say in focus groups (which the UK had no liability to pay), b) failed to do anything about unskilled immigration, c) persecuted the high skilled immigrants, such as scientists, who the public wants you to be MORE welcoming to, and d) failed to deliver on the nation’s Number One priority — funding for the NHS which is about to have a very high profile anniversary? And what happens if May staggers to 30 March 2019 and, as Barwell is floating with some of you, they then dig in to fight the 2022 campaign?

If you think that babble about ‘the complexity of the Irish border / the Union / peace’ will get you all off the hook, you must be listening to the same people who ran the 2017 campaign. It won’t. The public, when they tune back in at some point, will consider any argument based on Ireland as such obvious bullshit you must be lying. Given they already think you lie about everything, it won’t be a stretch.

Yes there are things you can do to mitigate the train wreck. For example, it requires using the period summer 2019 to autumn 2021 to change the political landscape, which is incompatible with the continuation of the May/Hammond brand of stagnation punctuated by rubbish crisis management. If you go into the 2022 campaign after five years of this and the contest is Tory promises versus Corbyn promises, you will be maximising the odds of Corbyn as PM. Since 1945, only once has a party trying to win a third term increased its number of seats. Not Thatcher. Not Blair. 1959 — after swapping Eden for Macmillan and with over ~6% growth the year before the vote. You will be starting without a majority (unlike others fighting for a third term). You won’t have half that growth — you will need something else. Shuffling some people is necessary but extremely far from sufficient. 

Of course it could have worked out differently but that is now an argument over branching histories for the history books. Yes it’s true that May, Hammond, Heywood and Robbins are Remain and have screwed it up but you’re deluded if you think you’ll be able to blame the debacle just on them. Whitehall is better at the blame game than you are, officials are completely dominant in this government, ministers have chosen to put Heywood/Robbins in charge, and YOU will get most of the blame from the public.

The sooner you internalise these facts and face reality, the better for the country and you.

Every day that you refuse to face reality increases the probability not only of a terrible deal but also of Seumas Milne shortly casting his curious and sceptical eyes over your assets and tax affairs.

It also increases the probability that others will conclude your party is incapable of coping with this situation and, unless it changes fast, drastic action will be needed including the creation of new forces to reflect public contempt for both the main parties and desire for a political force that reflects public priorities.

If revolution there is to be, better to undertake it than undergo it…

Best wishes

Former campaign director of Vote Leave

Ps. This explains part of what needs to be done and as you will see it will not be done by a normal UK party operating with the existing Whitehall system –‘a change of perspective is worth 80 IQ points’ and ‘how to capture the heavens’.

PPS. I should also add there are many officials who wanted to deliver government policy and MPs have let them down appallingly too. The less I say about that the better for them.

On the referendum #24H: Facebook, data science, technology, elections, and transparency

This blog has two short parts: A) a simple point about Wednesday’s committee hearing, B) some interesting evidence from a rare expert on the subject of data and campaigns, and a simple idea to improve regulation of elections. (And a PS. on hack Jane Merrick spreading more fake news.) There is a very short UPDATE re Facebook posted the next day, highlighted in BOLD below.

A. Re Wednesday’s Select Committee and Facebook letter

Correspondence from Facebook was published and used by the Committee to suggest that Vote Leave/AIQ have lied about when they started working together.

Henry de Zoete was introduced to AIQ on 31 March 2016. (This is all clear in emails that I think have been given to the Electoral Commission — if not they easily could be.)

AIQ did zero work for VL before then and, obviously, did not have access to VL’s Facebook page before we had even spoken to them.

If Facebook is saying that AIQ was running ads for VL in February 2016, then Facebook is wrong. [UPDATE: actually, if you read Facebook’s letter carefully, they correct their own error in a table where they use the timeframe for AIQ activity of “15 April – 23 June”. “15 April” of course fits with the date of VL’s introduction to AIQ I gave in this blog, and is the first day of the official campaign. The MPs either didn’t read the letter properly or chose to use the date which gave them a news story.]

VL was running stuff on FB in February as Facebook says. But this was done by us, NOT by AIQ.

Probably Facebook has looked at the VL FB page, seen activity in February, seen AIQ doing stuff shortly after and wrongly concluded that the earlier activity was also done by AIQ. It wasn’t and any further investigations will show this.

This isn’t actually important viz the legal claims and the EC investigation but I make the point in the interests of trying to clarify FACTS — so far the fake news inquiry has spread fake news around the world and clarified little. Also note how the Committee drops correspondence on the day of the hearing to maximise their chances of creating embarrassing moments for witnesses. This is the behaviour of people happy to see false memes spread, not the behaviour of truth-seeking MPs.

The Committee is now threatening me with ‘contempt of Parliament’. Their behaviour in seeking headlines rather than cooperating with witnesses over dates for evidence is  the sort of behaviour that has increased the contempt of the public for MPs over the last 20 years, which of course contributed to the referendum result. The Committee doesn’t understand Vote Leave. We had to deal with threats from MPs every day for a year, including from the PM/Chancellor and their henchmen who could actually back up serious threats. We ignored that. Why would you think we’re going to worry about EMPTY threats? If you think I care about ‘reputational damage’, you are badly advised.

B. Rare expertise on the subject of data and elections from Eitan Hersh to US Senate

Eitan Hersh wrote a book in 2015 called Hacking the Electorate. It’s pretty much the best book I’ve seen on the use of data science in US elections and what good evidence shows works and does not work.

As I wrote after the referendum, we tried hard in Vote Leave to base decisions on the best EVIDENCE for what works in campaigns and we spent time tracking down a wide variety of studies. Usually in politics everything is done on hunches. Inevitably, the world of ‘communications’ / PR / advertising / marketing is full of charlatans flogging snake oil. It is therefore very easy to do things and spend money just because it’s conventional. Because we were such a huge underdog we had to take some big gambles and we wanted to optimise the effectiveness of our core message as much as possible — if you know the science, you can focus more effectively. The constraints of time, money, and the appalling in-fighting meant we never pushed this nearly as far as I wanted but we tried hard.

For example, one of the few things about advertising which seems logical and has good evidence to support it is — try to get your message in front of people as close to the decision point as possible. That’s why we spent almost the whole campaign testing things (via polls, focus groups, online etc) then dropped most of our marketing budget in the last few days of the campaign. Similarly, Robert Cialdini wrote one of the few very good books on persuasion — Influence — and ideas from that informed how we wrote campaign materials. We were happy to take risks and look stupid. We came across a study where researchers had used as a control a leaflet with zero branding only to find, much to their surprise if I remember right, that it worked much better than all the other examples. We therefore experimented with leaflets stripped of all branding (‘The Facts’) which unleashed another wave of attacks from SW1 (‘worst thing I’ve seen in politics, amateur hour’ etc), but sure enough in focus groups people loved it (the IN campaign clearly found the same because they started copying this).

Of course, all sorts of decisions could not be helped by reliable evidence. But it is a much healthier process to KNOW when you’re taking a punt. Most political operations — and government — don’t try to be rigorous about decision-making or force themselves to think about what they know with what confidence. They are dominated by seniority, not evidence. Our focus on evidence was connected to creating a culture in which people could say to senior people ‘you’re wrong’. This is invaluable. I made many awful mistakes but was mostly saved from the consequences because we had a culture in which people could say ‘you’re wrong’ and fix them fast.

This is relevant to Hersh’s evidence and the conspiracy theories…

Hersh’s evidence should be read by everybody interested in the general issues of data and elections and the recent conspiracy theories in particular. I won’t go into these conspiracies again.

Here are some quotes…

‘Based on the information I have seen from public reports about Cambridge Analytica, it is my opinion that its targeting practices in 2016 ought not to be a major cause for concern in terms of unduly influencing the election outcome…

‘In every election, the news media exaggerate the technological feats of political campaigns…

‘The latest technology used by the winning campaign is often a good storyline, even if it’s false. Finally, campaign consultants have a business interest in appearing to offer a special product to future clients, and so they are often eager to embellish their role in quotes to the media…

‘I found that commercial data did not turn out to be very useful to campaigns. Even while campaigns touted the hundreds or thousands of data points they had on individuals, campaigns’ predictive models did not rely very much on these fields. Relative to information like  age, gender, race, and party affiliation, commercial measures of product preferences did not add very much explanatory power about Americans’ voting behavior…

‘Many commercial fields simply are not highly correlated with political dispositions. And even those that are might not provide added information to a campaign’s predictive models…

‘Nearly everything Mr. Nix articulates here [in a video describing CA’s methods] is not new. Based on what we know from past work,  it is also likely to have been ineffective. Cambridge Analytica’s definition of a persuadable voter is someone who is likely to vote but the campaign isn’t sure who they will vote for. This is a common campaign convention for defining persuadability. It also bears virtually no relationship to which voters are actually persuadable, undecided, or cross-pressured on issues, as I discuss in Hacking the Electorate… Cambridge Analytica’s strategy of contacting likely voters who are not surely supportive of one candidate over the other but who support gun rights and who are predicted to bear a particular personality trait is likely to give them very little traction in moving voters’ opinions. And indeed, I have seen no evidence presented by the firm or by anyone suggesting the firm’s strategies were effective at doing this 

‘As many journalists have observed, building a psychological profile by connecting Facebook “likes” to survey respondents who took a personality test would lead to inaccurate predictions. Facebook “likes” might be correlated with traits like openness and neuroticism, but the correlation is likely to be weak. The weak correlation means that the prediction will have lots of false positives…

‘In campaign targeting models I have studied, predictions of which voters are black or Hispanic are wrong about 25-30% of the time. Models of traits such as issue positions or personality traits are likely to be much less accurate. They are less accurate because they are less stable and because available information like demographic correlates and Facebook “likes” are probably only weakly related to them…

‘In a series of experiments, a colleague and I found that voters penalize candidates for mis-targeting such that any gains made through a successful target are often canceled out by losses attributable to mistargets… 

I am skeptical that Cambridge Analytica manipulated voters in a way that affected the election 

[Hersh then says ‘The skepticism I offer comes with a high degree of uncertainty’ and describes some of the gaps in what we know about such things. He also calls on Facebook to make its data available to researchers.]

‘News, both real and fake, is disseminated among users because it feels good to share. The kinds of news and content that often piques our interest appeals to our basest instincts; we are drawn to extremism, provocation, and outrage.’

Transparency — two simple ideas to improve things

In the last section Hersh discusses some broad points about transparency and social media. These things are important as I said after the referendum. Sadly, the focus on conspiracy theories has diverted the media and MPs away from serious issues.

I have zero legal responsibility for Vote Leave now — I ceased to be a director as part of our desperate rearguard action during the coup that kicked off on 25 January 2016. But I wouldn’t mind if Facebook wanted to take ALL of Vote Leave’s Facebook data that may be still sitting in ad manager etc — data normally considered very sensitive and never published by campaigns — and put the whole lot on its website available for download by anybody (excluding personal data so no individuals could be identified, which presumably would be illegal).

Why?

  1. In principle I agree with Hersh and think serious academic scrutiny would be good.
  2. In the interests of the VL team, it would prove what I have been saying and prove aspects of the conspiracy theories wrong. We never saw/used/wanted the data improperly acquired by CA. We did practically no ‘microtargeting’ in the normal sense of the term and zero using so-called ‘psychographics’ for exactly the reason described above — we tried to base decisions on good evidence and the good evidence from experts like Hersh was that it was not a good use of time and money. We focused on other things.

Here is another idea.

Why not have a central platform (managed by a much-reformed and updated Electoral Commission with serious powers) and oblige all permitted participants in elections to upload samples of all digital ads to this platform (say daily?) for public inspection by anybody who wants to look. After the election, further data on buy size, audience etc could be made automatically available alongside each sample. This would add only a tiny admin burden to a campaign but it would ensure that there is a full and accurate public record of digital campaigning.

Of course, this idea highlights an obvious point — there has never been any requirement on the parties to do this with paper documents. Part of the reason for the rage against Vote Leave in SW1 is that the referendum victory was something done to SW1 and the parties, not something done by them, hence partly their scrutiny of our methods. (This is also partly why the MPs are struggling so much to get to grips with the consequences.) There are no silver bullets but this simple measure would do some good and I cannot see a reasonable objection. Professional campaigners and marketers would hate this as they profit from a lack of transparency and flogging snake oil but their concerns should be ignored. Will the parties support such transparency for themselves in future elections?

One of the many opportunities of Brexit, as I’ve said before, comes in how we regulate such things. American law massively reflects the interests of powerful companies. EU law, including GDPR, is a legal and bureaucratic nightmare. The UK has, thanks to Brexit, a chance to regulate data better than either. This principle applies to many other fields, from CRISPR and genetic engineering to artificial intelligence and autonomous vehicles, which in the EU will be controlled by the ECJ interpreting the Charter of Fundamental Rights (and be bad for Europe’s economies and democracies). MPs could usefully consider these great opportunities instead of nodding along as officials do their best to get ministers to promise to maintain every awful set of EU rules until judgement day.

The issues of data-technology-elections is going to become more and more important fast. While the field is dominated by charlatans, it is clear that there is vast scope for non-charlatans to exploit technology and potentially do things far more effective, and potentially dangerous for democracy, than CA has claimed (wrongly) to do. Having spent some time in Silicon Valley since the referendum, it is obvious that it is/will be possible to have a decisive impact on a UK election using advanced technology. The limiting factors will be cash and a very small number of highly able people: i.e an operation to change an election could scale very effectively and stay hidden to a remarkable degree. The laws are a joke. MPs haven’t mastered the 70 year old technology of TV. How do you think they’d cope with people using tools like Generative Adversarial Networks (GANs) — never mind what will be available within five years? The gaps in technical skill between commercial fields are extreme and getting wider as the west coast of America and coastal China suck in people with extreme skills. Old media companies already cannot compete with the likes of Google and the skill gaps  — and their consequences — grow every day.

But but but — technology alone will very rarely be the decisive factor: ‘people, ideas, and machines — in that order’, shouted Colonel Boyd at audiences, and this will remain true until/unless the machines get smarter than the people. The most important thing for campaigns (and governments) to get right is how they make decisions. If you do this right, you will exploit technology successfully. If you don’t — like the Tories in 2017 who created a campaign organisation violating every principle of effective action — no advantage in technology or cash will save you. And to get this right, you should study examples from the ancient world to modern projects like ARPA-PARC and Apollo (see here).

Anyway, I urge you to read Hersh’s evidence and ponder his warnings at the end, it will only take ~15 minutes. If interested, I also urge you to read some of the work by Rand Waltzman who ran a DARPA project on technology and social media. He has mostly been ignored in Washington as far as I can see but he should not be. He would be one of the most useful people in the world for MPs and hacks interested in these issues to speak to.

https://www.eitanhersh.com/uploads/7/9/7/5/7975685/hersh_written_testimony_senate_judiciary.pdf


PS. I’ve just been sent a blog on The Times website by Jane Merrick. It includes this regarding the latest odd news about a C4 drama:

‘Yet as with Mandelson, Cummings seems to complain about everything that is ever written about him, and so his reaction from his Twitter account — @odysseanproject (don’t ask) — was this: “What’s the betting this will be a Remain love-in and dire.” Oh how humbly he does brag!

I’ve had hacks email me asking me to ‘defend’ things on that Twitter account.

1. That is not my twitter account — it is a fake account. It’s interesting how many hacks complain about fake news while spreading it themselves. If you’re going to make claims about anonymous Twitter accounts (as she does elsewhere in her blog), try not to get confused by obvious parodies.

2. She also doesn’t mention that her husband, Toby Helm, was the SW1 equivalent of the guy in Scream chasing me and Henry de Zoete around Westminster for two years with a carving knife and a scream mask. The Observer promised the lobby I’d be marched out of the DfE in handcuffs. Nothing happened. Why? Because hate clouded their judgement, they botched the facts, and their claims were bullshit. Sound familiar?

[Update: The Times has cut that passage from the blog.]

On the referendum #24G: Grandstanding MPs and the Zoolander inquiry

The Select Committee doing the inquiry into fake news asked me to give evidence over a month ago.

I said I would be happy to but explained that I could not do those dates and also explained legal issues re the Electoral Commission’s 3rd inquiry which means that lawyers of multiple parties have told me to keep my trap shut until they’ve finished, which is unlikely for months to come.

I also made clear I was happy to give evidence once this snag was out of the way.

Instead of discussing dates in a cooperative way, they went silent for weeks then threatened me this morning with a ‘Summons’. (They have also published claims today about the attitude of the ICO and EC towards me giving evidence which those organisations have never told me.)

I said that if they took such a foolish step — and an unnecessary one given I wanted to give evidence — they would be demonstrating their priority was grandstanding PR, not truth-seeking. I also suggested they bring Wylie and Zoolander back to the Committee to explain the multiple factual errors and inconsistencies in their stories so we could all be clear about what exactly they are really claiming. So far the inquiry on fake news has helped spread fake news across the world.

Minutes later they sent their Summons and asked for confirmation I will change my mind and appear on the date they already knew I could not make.

No, of course not.

I said that if they issued a Summons instead of discussing possible dates like reasonable people, then it would be obvious they are not interested in friendly cooperation to uncover the truth. So I will not give evidence to this Committee under any circumstances. (I may to other Committees depending on behaviour.)

One of the many things about government that could be improved is changes to the Committee process and powers. They should, like in America, have the power to compel attendance (!), but they should also have processes that push them towards truth-seeking behaviour rather than the usual trivialising grandstanding. Committees also need resources for specialist help as they are unable to question witnesses properly on many issues.

If they really wanted to get to the bottom of things, they would have continued discussions over dates, not sent an ineffectual Summons to the media. Their desire for a quick headline has robbed them of hours of TV… 

Ps. For those who missed it, Facebook analysed Vote Leave advertising on Facebook during the referendum and concluded that not only did Vote Leave NOT use the data wrongly appropriated by CA, but that we COULD NOT HAVE DONE. So far, no correction from Observer/C4.