#29 On the referendum & #4c on Expertise: On the ARPA/PARC ‘Dream Machine’, science funding, high performance, and UK national strategy

Post-Brexit Britain should be considering the intersection of 1) ARPA/PARC-style science research and ‘systems management’ for managing complex projects with 2) the reform of government institutions so that high performance teams — with different education/training (‘Tetlock processes’) and tools (including data science and visualisations of interactive models of complex systems) — can make ‘better decisions in a complex world’.  

This paper examines the ARPA/PARC vision for computing and the nature of the two organisations. In the 1960s visionaries such as Joseph Licklider, Robert Taylor and Doug Engelbart developed a vision of networked interactive computing that provided the foundation not just for new technologies but for whole new industries. Licklider, Sutherland, Taylor et al provided a model (ARPA) for how science funding can work. Taylor provided a model (PARC) of how to manage a team of extremely talented people who turned a profound vision into reality. The original motivation for the vision of networked interactive computing was to help humans make good decisions in a complex world.

This story suggests ideas about how to make big improvements in the world with very few resources if they are structured right. From a British perspective it also suggests ideas about what post-Brexit Britain should do to help itself and the world and how it might be possible to force some sort of ‘phase transition’ on the rotten Westminster/Whitehall system.

For the PDF of the paper click HERE. Please correct errors with page numbers below. I will update it after feedback.

Further Reading

The Dream Machine.

Dealers of Lightning.

‘Sketchpad: A man-machine graphical communication system’, Ivan Sutherland 1963.

Oral history interview with Sutherland, head of ARPA’s IPTO division 1963-5.

This link has these seminal papers:

  • Man-Computer Symbiosis, Licklider (1960)
  • The computer as a communications device, Licklider & Taylor (1968)

Watch Alan Kay explain how to invent the future to YCombinator classes HERE and HERE.  

HERE for Kay quotes from emails with Bret Victor.

HERE for Kay’s paper on PARC, The Power of the Context.

Kay’s Early History of Smalltalk.

HERE for a conversation between Kay and Engelbart.

Alan Kay’s tribute to Ted Nelson at “Intertwingled” Fest (an Alto using Smalltalk).

Personal Distributed Computing: The Alto and Ethernet Software1, Butler Lampson. 

You and Your Research, Richard Hamming.

AI nationalism, essay by Ian Hogarth. This concerns implications of AI for geopolitics.

Drones go to work, Chris Anderson (one of the pioneers of commercial drones). This explains the economics of the drone industry.

Meditations on Moloch, Scott Alexander. This is an extremely good essay in general about deep problems with our institutions.

Intelligence Explosion Microeconomics, Yudkowsky.

Autonomous technology and the greater human good. Omohundro.

Can intelligence explode? Hutter.

For the issue of IQ, genetics and the distribution of talent (and much much more), cf. Steve Hsu’s brilliant blog.

Bret Victor.

Michael Nielsen.

For some pre-history on computers, cf. The birth of computational thinking (some of the history of computing devices before the Turing/von Neumann revolution) and The crisis of mathematical paradoxes, Gödel, Turing and the basis of computing (some of the history of ideas about mathematical foundations and logic such as the famous papers by Gödel and Turing in the 1930s)

Part I of this series of blogs is HERE.

Part II on the emergence of ‘systems management’, how George Mueller used it to put man on the moon, and a checklist of how successful management of complex projects is systematically different to how Whitehall works is HERE.

On the referendum #24M: Carole asks me questions, I answer — can MPs handle the truth?

After I published the DCMS report on fake news which itself spreads fake news, misunderstandings, wrong ideas about GDPR and the law etc, Carole commented on my blog. I replied. Both are below unedited for those following the twists of this farcical story…

Remember, the Electoral Commission REFUSED to interview me or any of the 7 Vote Leave staff over 2 years and 3 inquiries. I offered to give evidence to Collins’ committee. He refused to negotiate over dates and demanded a date he knew weeks earlier I could not do. I have made an open offer to MPs to give evidence to any other committee they want on condition only that all of us are under oath. They can make the whole committee Remainers, that’s OK with me, but they have to promise not to lie. Everybody interviewing Collins should ask him — ‘Why don’t MPs call Cummings’ bluff and get him in and all of you do it under oath to get to the bottom of all this once and for all? Wouldn’t it be good for once for MPs to promise not to lie?’

As Jack Nicholson said, ‘The truth? You can’t handle the truth!’

And everybody who interviews Wylie should ask him:

‘You tried to sell Cummings the exact same stuff you now claim is a threat to democracy and he turned you down, YOU admit YOU had access to the notorious Facebook data but Facebook has confirmed that Vote Leave COULD NOT HAVE used that data in their advertising, contrary to what you explicitly claimed, so why should anyone believe a word you say and whom did you flog all that to after Cummings told you to get lost in November 2015?’

FACT: Wylie is a liar, a fantasist and he presented himself to the media as a ‘whistleblower’ without disclosing he tried to sell me his snake oil and assured me he would keep it all ‘secret’. It’s all in writing if a proper court or the MPs ever fancy finally doing a proper investigation of all this…

I see Lionel Barber, editor of the FT which told us repeatedly that our economy would collapse unless we joined the euro, is also yapping about lies and facts. FACT: Barber is happy to publish lies when they’re his lies — I’ve seen him do it many times including about me. During the referendum when an FT’ hack was criticised by a Cabinet Minister for a lack of integrity in the FT’s coverage, his response was a shrug and ‘we’re in campaign mode’. Barber like many mentalist Remainers (and Leavers) lives in a fantasy world where his side are LIGHT AND TRUTH and the other side are DARK AND LIES. Unlike them, I’ve always thought it reasonable to support Remain. I don’t make the mistake of thinking those who disagree with me are evil morons. This is one of the reasons Vote Leave made more rational decisions than the other side which fooled themselves about their environment. These guys are used to getting their own way. They got screwed on the biggest issue in politics when they thought they couldn’t lose. They’re mad partly because they’re rightly embarrassed. And they keep fooling themselves every day…

I see Best for Britain is fundraising for a judicial review demanding a second referendum. This has no prospect of success because the relevant provisions of the EU Act 2011 have been repealed. Raising money for it is arguably fraudulent and criminal behaviour. A hack should investigate…

(For those REALLY interested in this story… There is some comment and I’ve got some emails about why there seem to be VL Facebook ads placed in India/Sri Lanka in June. I haven’t looked at the FB data dump but it’s very likely these were part of the testing for our football contest where we offered a real prize of £50 million if you could predict the Euro football tournament — £50 million a day, you see, because we couldn’t persuade an insurance company to insure us for £350 million prize which is what I wanted to do (we even got in touch with Warren Buffett’s office given he self-insured a  similar prize for his own PR stunt which had given us this idea, but being Buffett’s office they made the smart decision not to get involved damn it). We had to test the infrastructure live without the media seeing so we advertised in Asia including on Asian porn sites to see if the website worked properly, gambling that the British media would probably not see such carefully targeted ads in the middle of the night. This is probably why these otherwise odd looking ads have turned up in the FB data.)

CAROLE COMMENT

Dom. Great to have your input. If only you could have given that in person to MPs as they repeatedly asked you to. You still have not provided any cogent or reasonable answer about why you refuse to answer parliament’s questions. You campaigned for parliament’s sovereignty and yet you do not respect British laws and you deliberately seek to undermine its authority. Can you explain why?

I was posting on Twitter as I looked through those files last night It would have been great to have had yours or Matthew Elliott’s input at any stage. He was online – smearing Damian Collins – why didn’t he chip in? I knew that the law required imprints on political advertisements. I went and read the guidelines to referendum campaigners and that included digital adverts too…but as others pointed out this morning – and as I updated to make clear – the ones for digital ads are different. They *do not have the force of law*. So, you’re quite correct. There is a loophole.

A loophole that you knew about and exploited because you make no mention of the framing of the 50 million ads. They didn’t have any mention of Vote Leave, did they? Or at least from the information released by Facebook, there was nothing to say who the advert was on behalf of or what it was for? (Anonymously harvesting people’s data.) Can you please display how these were seen on Facebook and explain your rationale for apparently not disclosing who was placing the advertisement or how it was intended to be used?

You mention nothing about the toxic nature of these adverts. Their overt racism. The scaremongering. The blatant lies. Would you like to comment on those?

And can you comment too on the adverts that were shown after Jo Cox was murdered and campaigning was suspended. It appears that some of these ads – including one labeled “Breaking News” – were scheduled during that period. Certainly they reached millions of people. Can you clarify: were you campaigning during that period? Or is there another explanation?

Which of these ads were posted publicly to Vote Leave’s Facebook page? And which were dark? The majority of these have never been seen publicly before so it seems at least some are the latter. Could you clarify? And specify which.

It is really helpful and important to have input – that’s why I’ve repeatedly sent you questions to which I’ve never had any proper replies. I anticipate your fullest response so that we can be sure to get this right.

Thanks, Carole

MY REPLY

1/ I’ve never ‘refused to answer’ questions as you know. I offered to negotiate a date with Collins and he refused the offer. I’ve also offered to give evidence to a different committee — though I’ve suggested we ALL should do it UNDER OATH. Wouldn’t that be a good way to set an example to the nation — political discussions with everyone forced to be careful about the truth?!

2/ ‘I was posting on Twitter as I looked through those files last night It would have been great to have had yours or Matthew Elliott’s input at any stage.’ That’s not how journalism works. You don’t babble nonsense on Twitter accusing people of being racist criminals and expect that they’re monitoring you 24/7 and leap in to fix your repeated errors. It is YOUR RESPONSIBILITY TO FACT CHECK BEFORE MAKING CLAIMS.

As you know, the Observer and you have had to delete many defamatory claims you’ve made (about others) based on fantasies. Remember how you made claims about ‘deleting the google drive’ that I told you were lies, and you’ve had to delete all that from the Guardian website and your twitter feed AND PAY LEGAL COSTS TO THOSE YOU DEFAMED?

3/ ‘So, you’re quite correct. There is a loophole.’ Glad to see this but how about deleting all your tweets that call us criminals — or do you think it’s OK to accuse people of being ‘criminals’ on the basis of errors and leave the errors spreading across the world?

4/ I don’t remember exactly how the 50m ads were done. I remember there was a separate website. But all Facebook ads have to have a frame so they will have been identified. And remember the POINT of it was to collect data! There HAD to be an identifiable click through for the ads or the whole exercise would have been pointless. So though I can’t remember the detail I know for sure there was a clear identification and a website with a proper legal privacy policy and connection to Vote Leave etc.

5/ ‘can you comment too on the adverts that were shown after Jo Cox was murdered and campaigning was suspended. It appears that some of these ads – including one labeled “Breaking News” – were scheduled during that period. Certainly they reached millions of people.’ Wrong. As I’ve explained on my blog. Ads were not shown in that time. You think it is ‘certain’ — your certainty is unfounded.

6/ Lies? Cameron wanted to ‘pave the road from Ankara’, it’s on film. Fact!

7/ We ran no ‘dark’ posts despite repeated claims to the contrary. Everything you see in the FB data dump was a normal FB ad. Remember, I asked FB to release everything weeks ago. Are you asking Remain to do the same or are you happy for what they did to stay ‘dark’?

If you really care about facts and truth you will stop spreading fake news across the internet time after time.

Although lots of people call you a liar I don’t agree with them. The whistleblowers lied and have given multiple versions of the same events that would be shredded in open court under oath — e.g Sanni claiming he saw me have meetings with Grimes and AIQ (total invention, no such meetings happened but of course the EC never asked me or Grimes about this or anything else). But I think you just want to believe we’re baddies, you trust the wrong people, and you don’t check stuff properly. If you’re going to write about fake news on a website that reaches millions, you have a particularly strong responsibility to stop spreading misinformation about this story. If you want to be treated like Bob Woodward, you should be careful about facts. If you’d done Watergate, Nixon would have been able to trash the story and get away with it.

In the autumn, you and I should do an interview. You interview me for 90 minutes and ask whatever you want. But then I’ll interview you for 90 minutes about your reporting. All on film so no fancy editing.

And in the meantime you should ask the MPs — why not call Cummings’ bluff and accept his offer for a multi-hour session, no questions banned, with all of you under oath so you can finally nail him?

Best wishes
D

On referendum #24L: Fake news from the fake news committee, Carole, and a rematch against the public

[Update. More fake news — claims we kept advertising during the ‘pause’ after Jo Cox’s murder. Wrong. The spreadsheet data from Facebook reflects when the ads were created, not when they were shown. AIQ was putting stuff into the system during the pause, not running ads. Again this false meme is already around the world. Alistair 45 minutes Campbell is ranting about moral cesspits. But yet again it’s fake news.

Incidentally, I opposed any pause at the time. I think the right way to deal with terrorism is to carry on with normal life, like Britain used to when it was a more serious country. (I hated the way Cameron would tweet in response to ISIS, giving them just what they want. I hated the way Cameron and Blair read out in Parliament names of people killed which had the same effect. I thought it also reflected SW1’s basic ignorance about how to deal with information operations against terrorist groups in many ways more sophisticated about communication than traditional institutions — eg. Hezbollah often does TV better than the Tory Party.) But I was outvoted by MPs who downed tools and headed back to London, giving Osborne/Dre the chance to use the news as they wished. But they botched it — in a classic case study of people fooling themselves, they thought that the country reflected the mood of Inner London. They started tweeting broken hearts and ‘we love our MP’ at each other. They therefore blew their last chance to recover from strategic misjudgements. Those who would run Remain in a second referendum remain disconnected from reality and on current form would botch a second referendum which anyway would be held in circumstances much more favourable to Leave on almost every dimension.

Also NB. Carole Cadwalladr has commented below and I will answer shortly.

Also NB. contra some reports, I was not sent the report by the Committee. I’m told they did send it to ‘witnesses’ but that did not include me. I was sent it by someone in Parliament fed up with Collins’ dishonesty and blatant use of Carole’s conspiracies for his own end of overturning the referendum result.]

A few thoughts on the last 24 hours of conspiracy theories plus a copy of the DCMS Select Committee report on fake news. They gave it to Carole for Sunday, obviously, but someone appalled at their dishonesty leaked it to me so I publish it below. It is, in keeping with their general behaviour, itself fake news.

Most of SW1 has suffered a psychological and operational implosion because of the referendum. 

Many MPs, hacks and chalatan-pundits on both sides have responded to the result by retreating to psychologically appealing parallel worlds rather than face reality — ‘the frogs before the storm’ prefer the comforting Oblonsky mental fug of groupthink.

A subset of the ERG, for example, welcomed the December agreement on the Irish backstop that actually spelled doom for their central ideas about how the negotiations were being conducted. Bernard Jenkin was so confident that he and Cash understood what was happening he cheerfully wrote that he had not needed to read it before welcoming it. This is the same group now ranting about Chequers — which was programmed by the December agreement, as are the imminent further surrenders in the autumn on Free Movement and everything else! This is the same group that tells everyone that people like me who say that serious preparations are needed to leave the EU are ‘like those peddling the Millennium Bug’. Their ideas on preparations are as accurate as their ideas on the December agreement were and of course in order to avoid facing their tragi-comic blunders of judgement over two years they are constructing parallel worlds for their minds to live in.

Hardcore Remainers are similar. They want a second referendum and this requires de-legitimising the first. They therefore hysterically spread false memes while shouting ‘liars’ at Leavers. Cash and Carole have a lot in common.

The last 24 hours has illustrated again how the entire story about Vote Leave / data / digital communications has become a great case study in contemporary politics: ubiquitous accusations of lying by people who either lie or are entirely reckless about the truth, almost nobody figuring out reality before babbling all over social media setting off cascades of false information, MPs clueless about basic legal issues also spreading false memes and so on. 

A few simple points about the new wave of fake news.

Carole has spread countless factual errors for over a year. When I explained how we had followed best practice to safeguard personal data by quickly deleting the VL electoral database containing tens of millions after the referendum, she turned this professional and ethical behaviour (not copied by the Remain campaign which kept it all) into accusations of me ‘destroying evidence’ and perverting the course of justice. This sort of thing has happened repeatedly.

Over the past 24 hours she has constructed new fake memes now spreading across the world. 

1. The latest astonishing ‘crimes’ according to Carole et al is that the VL ads did not have ‘imprints’, were ‘dark’, unethical and illegal. She has tweeted dozens of times along the lines of:

‘[Vote Leave] DELIBERATELY BROKE THE LAW by leaving off who paid for it… No wonder Dominic Cummings wouldn’t come to parliament. No wonder @facebook didn’t want to release this shit. This is truly toxic, dark, absolutely undemocratic shit at the heart of the biggest election we’ll ever see… Look at this stuff. Fake fake fake news. It’s not an ad. It’s not labelled as an ad. It doesn’t say who placed it or who paid for it or who it was targeted at or way. This is the fakiest of fake news. And until today we had no idea about any of this’. 

This is totally wrong and reflects deep misunderstandings. 

a. The campaigns were NOT legally required to carry imprints in the same way as printed  material. Carole is factually wrong about the law again. 

b. This is actually irrelevant because the VL ads that Carole claims were ‘dark’ and criminal because ‘no imprint’ actually were clearly labelled as VL. The images she is pulling from the FB data dump are raw images — they are not images of the actual ads themselves. The images sat within a ‘frame’ which everyone seeing them on Facebook would see. This included ‘Vote Leave’ and other text and also had a weblink. 

E.g Carole posts this as new evidence that I should be locked up — an image ‘without imprint’:

Screenshot 2018-07-27 11.55.21

 

(By the way, you CANNOT trust David ‘pave the road from Ankara’ Cameron on Turkey! Don’t believe me? Watch this!)

 

This is how ads actually appear on Facebook:

Screenshot 2018-07-27 11.57.39

Thousands of people are now spreading Carole’s memes across the internet. They are shocked and appalled — surely the criminal Cummings will finally be jailed etc. 

2. Amid the data dump of Facebook ads, there are claims that VL promoted BeLeave ads. This is a misunderstanding and the BBC has corrected their story. These ads appear in the 0-999 impressions box in the FB spreadsheet because the actual number of impressions was ZERO. They never ran. This issue is related to AIQ’s recent explanation of an error they made with loading audiences for BeLeave. It is detailed and technical and I won’t go into it here but in a nutshell: VL did not promote BeLeave ads. Remain, however, did do this but of course nobody cares. (It is more forgivable to make mistakes about this as it is a tricky niche issue.)

3. Another criminal conspiracy Carole is spreading across the internet concerns Brexit Central. This was set up after the vote (not by me). Grimes went to work on it and merged the BeLeave page into the BC page hence FB labels them confusingly as ‘Brexit Central/BeLeave’. Without asking anybody what it means, Carole and others have screamed ‘aha this organisation secretly existed before the vote and was illegally advertising, LOCK UP CRIMINAL CUMMINGS.’ Wrong again.

There are many other false memes spreading but there’s no point going into all of them.

Also NB. I asked months ago for Facebook to publish everything in the interests of transparency. Will Will Straw do as I did and ask Facebook to publish EVERYTHING they have about the Remain campaign? I’m not holding my breath.

HERE IS THE DCMS REPORT ON FAKE NEWS. IT IS… FAKE NEWS

The report knowingly/incompetently makes false claims regarding Vote Leave, AIQ and BeLeave. Despite nobody ever producing any evidence for Carole’s original loony conspiracy theory that I was secretly coordinating with Arron Banks, Bannon and Robert Mercer, the Committee also asks for yet another inquiry of this and of course they want the police involved to give credibility to their fantasies and legitimise their campaign for a second referendum. The MPs know Facebook has explained to them that VL COULD NOT HAVE used the notorious Facebook data acquired by Cambridge Analytica but they try to provide credibility to these conspiracy theories.

Further, these MPs have littered their report with errors and misunderstandings about the legal framework for elections, thus spreading further confusion. They haven’t even bothered to understand GDPR, which they mis-explain badly. Collins et al have shown no interest in the truth. Now MPs publish a document after months of supposed work that makes basic errors about electoral law which will debase public debate even further.

NB. I HAVE SUGGESTED TO MPS THAT I COME AND GIVE EVIDENCE AND WE ALL OPERATE UNDER OATH. NOT A SQUEAK FROM THEM.

JUST LIKE THE ELECTORAL COMMISSION REFUSED TO SPEAK TO ME OR ANYBODY ELSE FROM VOTE LEAVE OVER TWO YEARS AND THREE INQUIRIES.

WHY?

AS JACK NICHOLSON SAYS, ‘THE TRUTH? YOU CAN’T HANDLE THE TRUTH!’

If the MPs really want to get to the bottom of this, all they have to do is promise to tell the truth. Come on guys, step up to the plate…

If SW1 put 1% of the effort it’s put into spreading fake news about Vote Leave into FIXING THE LAWS as I suggested BEFORE Carole’s conspiracy theories got traction, we would be in a much healthier state. But SW1 is rotten…

Hugo Rifkind says ‘Whatever you think of the referendum result, we can’t ever let there be a campaign like this again.’

Tough luck Hugo — if your side gets its way and there is another referendum, Vote Leave 2 will be much much worse for your side than VL1 was. VL2 will win by more than VL1 and the logical corollary will be to morph into a new party and fight the next election ‘to implement the promises we made in the referendum because the MPs have proved they can’t be trusted’. At a minimum VL2 will win the referendum and destroy the strategic foundations of both main parties. The Tories will be destroyed and maybe Labour too. The rotten civil service system will be replaced and the performance of government will be transformed for the better. Investment in basic science research will flow. Long-term funding for the NHS guaranteed by law. MORE high skilled immigrants, FEWER low-skilled. An agenda that could not be described as Left or Right. The public will love it. Insiders will hate it but they will have slit their own throats and have no moral credibility. Few careers will survive.

Is there enough self-awareness and self-interest among MPs to realise the consequences? Hard to say. I’m more critical of SW1 than almost any Insider and even I have been surprised by the rottenness. It will be no surprise if they slit their own throats.

So far the MPs have botched things on an epic scale but it’s hard to break into the Westminster system — they rig the rules to stop competition. Vote Leave 1 needed Cameron’s help to hack the system. If you guys want to run with Adonis and create another wave, be careful what you wish for. ‘Unda fert nec regitur’ and VL2 will ride that wave right at — and through — the gates of Parliament.

Ps. One hack who does actually pay attention to facts on this subject is Jim Waterson. It can’t be comfortable pointing out facts at the Guardian on this story so double credit to him.

[Pps. Sorry for mis-remembering Tom Cruise/Jack Nicholson to those who messaged.]

On the referendum #24K: Observer fake news and a suggestion — I and the MPs discuss this with all of us under oath

Very short blog re the Observer story…

As usual the news these days is very confusing.

I have never refused to give evidence to MPs.

I offered to give evidence to MPs in writing.

Collins refused to discuss a suitable date, insisted on a date he knew I could not do, and chose to issue a formal summons instead of figure out when I could come.

I therefore said that his Committee was plainly behaving unreasonably, that I would not attend it, but I would consider speaking to another Committee:

‘If another Committee behaves reasonably and I can give evidence without compromising various legal actions then I will consider it. Once these legal actions have finished, presumably this year, it will be easy to arrange if someone else wants to do it.’ HERE.

Since I wrote that, MPs have not taken me up on their offer. Collins’ Committee has continued to spread fake news across the internet. Collins has misled Parliament about Vote Leave.

If MPs want to hear what actually happened — from AIQ to Cambridge Analytica to BeLeave — all they have to do is ask and I will be VERY happy to talk to them. Surely there is a Committee there that can make time if MPs are really so desperate to know? The Collins Committee is only into ‘fake news’ and data whereas these issues in many ways are much more fundamental than that and touch on many committees. They could even form an ad hoc Special Committee — and I don’t mind if they stack it with all Remainers if they like, as they have with Collins’ Committee. Collins could be on it too.

There is only one complicating factor. The Electoral Commission told me last week that I have never been investigated by them. This will seem astonishing given their report essentially is a claim that although they cannot provide any evidence for it I somehow must have devised a ‘common plan’ with Grimes. They have concluded this without ever investigating me or interviewing me and after rejecting my offer to talk to them. Further, the EC says they have referred ‘others’ to the police as well as the VL ‘responsible person’. I am told this refers to me. But the EC is under legal obligations to investigate people before they refer them to the police and they have not investigated me, according to them. Baffling…

If the EC confirms that they have not referred me to the police, and my only future involvement is therefore as a witness in the VL/Grimes appeals and further investigations, then as far as I understand it there is no barrier to me giving evidence to Parliament.

I would also be happy to do this UNDER OATH which the whistleblowers have not done — and THE MPs SHOULD ALSO BE UNDER OATH and therefore obliged not to say things they know, or reasonably should know, are false. Collins could therefore be on this Committee but he could not repeat falsehoods. Ditto Bradshaw, Lammy, Grieve or any of the rest who tell all sorts of lies about the referendum. Wouldn’t it be an improvement over the usual Parliamentary process for everybody there to be under an obligation to tell the truth, particularly at a time when so many in public life lie as a matter of course?! We’d all be forced to up our game for many hours on end and avoid lots of the usual silliness in such events. This could only be a good thing.

Also note that the whistleblowers have provably lied — so much that the Observer has been forced to delete many of their original allegations from their website and forced Carole to delete many of her tweets and the Observer has coughed up substantial legal fees (not mine, I have not hired any lawyers or threatened any hacks with lawyers).

The normal pattern is for people to say ‘Cummings is bluffing’. When people say this it always turns out I wasn’t. Remember last year when I said the EC had written to VL saying we could make donations? Carole and @Jolyon said I was lying/bluffing for a year. What happened? The documents were produced in the High Court as part of the judicial review: I wasn’t lying and I wasn’t bluffing, though much of the media has not realised this fact and still reports fake news. I don’t bluff when my bluff will obviously be called.

If the EC confirms they have not referred me to the police — which logically they cannot have done given they have never investigated me (unless they are lying which surely is extremely improbable?) — and MPs get in touch to fix a date I will be more than happy to answer every question any of them have.

I will blog further about this weird affair this week.

On the referendum #27: Banks, Russia, conspiracies and Vote Leave

Dear Tory MPs, ministers, donors and peers who supported the January 2016 coup against Vote Leave…

Remember how I and Victoria Woodcock told you repeatedly Banks was not someone who should play a significant role, that his conduct would destroy the credibility of an official campaign, and a ‘unified campaign’ with him would be a ‘total disaster’?

Remember how I and Victoria Woodcock told you repeatedly that he could not be trusted?

Remember how in horrific meeting after horrific meeting you said that we didn’t understand politics and we needed to ‘unite’ and ‘use his social media operation cos he’s got hundreds of thousands of Likes’?

Remember how we clutched our heads and said ‘Facebook doesn’t work like that, he’s spinning you all bullshit, the media will sink the whole campaign if Banks is involved and we refuse to contemplate it’?

Remember how you then tried to engineer the coup, partly also because Banks had told so many of you (cunningly) that the most important factor in winning was ‘you must represent us in the debates on stage with Nigel in front of millions’ and ‘we need your experience, not all these kids Cummings has hired’?

Of course it’s true that the Remain Establishment are doing whatever they can to discredit the referendum, the Observer has invented stuff for two years (including loony conspiracy theories about Banks, me, Mercer, AIQ, Russia etc), and Banks’s actual role in the 10 week campaign was trivial other than causing us embarrassment. Yes it’s true that Banks was a net drag on the result and we’d have won by more if he’d been dropped down one of his defunct mines in summer 2015 and the effort wasted dealing with him had been spent making Vote Leave much stronger earlier. From grassroots to digital, everything would have happened earlier, bigger and better but for that debilitating distraction which meant VL staff had to fight Banks and the entire Establishment simultaneously.

But all that does not change how close you all came to destroying our chance of winning by putting him in charge of the whole thing.

I know you’ll all be wanting to write to those Vote Leave staff who called your bluff and who, unlike you, displayed moral courage under pressure and made many personal sacrifices while you were on the beach or shooting, in order to apologise and thank them personally so here are some of the names of those who told you on 25 January 2016 they would all be out the door in 5 minutes if you persisted and handed power to Banks, and thereby nudged reality down a different branching history:

  • Richard Howell
  • Oliver Lewis
  • Rob Oxley
  • Stephen Parkinson
  • James Starkie
  • Paul Stephenson
  • Jonny Suart
  • Nick Varley
  • Cleo Watson
  • Victoria Woodcock

But for their actions that day, Vote Leave would have been destroyed, Farage and Banks would have run the official campaign with Bill Cash as legal adviser fighting with DD to be on the Today programme, Boris and Gove would have gone on a long holiday rather than flush their reputations down the toilet, Remain would have won 60-40 and Osborne would today be scanning the horizon for the right moment to take over before the 2020 election.

You’re welcome…

Dominic

Ps. Another branching history… If Cameron and Osborne had simply delayed the vote to 2017, Vote Leave would have ceased to exist in spring 2016, Banks and Farage would have been in charge with Cash/DD et al, and Remain would very likely have cruised to victory last year. Our extreme action on and after 25 January only worked because of the time pressure imposed by the Government. Without it, the consensus was, as people said at the time, ‘we’d have a year to rebuild without you and your crazy ideas’. In history books, luck is always underplayed and the talent of individuals is usually overplayed. As I’ve said many times, Vote Leave could only win because the Establishment’s OODA loops are broken — as the Brexit negotiations painfully demonstrate daily — and they are systematically bad at decisions, and this created just enough space for us to win.

Pps. Although Banks and Leave.EU HQ were hopeless, many of its volunteers did great work and ignored ‘the horror, the horror’ in London among the egomaniacs. Although Farage told them not to help VL post-designation, most of them ignored him and did help us (see comments below).

‘Politics is a job that can really only be compared with navigation in uncharted waters. One has no idea how the weather or the currents will be or what storms one is in for. In politics, there is the added fact that one is largely dependent on the decisions of others, decisions on which one was counting and which then do not materialise; one’s actions are never completely one’s own. And if the friends on whose support one is relying change their minds, which is something that one cannot vouch for, the whole plan miscarries… One’s enemies one can count on – but one’s friends!’ Otto von Bismarck.

‘Everything in war is very simple, but the simplest thing is difficult. The difficulties accumulate and end by producing a kind of friction that is inconceivable unless one has experienced war… Countless minor incidents – the kind you can never really foresee – combine to lower the general level of performance, so that one always falls short of the intended goal.  Iron will-power can overcome this friction … but of course it wears down the machine as well… Friction is the only concept that … corresponds to the factors that distinguish real war from war on paper.  The … army and everything else related to it is basically very simple and therefore seems easy to manage. But … each part is composed of individuals, every one of whom retains his potential of friction… This tremendous friction … is everywhere in contact with chance, and brings about effects that cannot be measured… Friction … is the force that makes the apparently easy so difficult… Finally … all action takes place … in a kind of twilight, which like fog or moonlight, often tends to make things seem grotesque and larger than they really are.  Whatever is hidden from full view in this feeble light has to be guessed at by talent, or simply left to chance.’ Clausewitz.

 

On the referendum #26: How to change science funding post-Brexit [updated with comment by Alan Kay]

There was an excellent piece in the Telegraph yesterday by two young neuroscientists on how SW1 should be thinking about science post-Brexit. The byline says that James Phillips works at Janelia, a US lab that has explicitly tried to learn about how to fund science research from the famous successes of Bell Labs, the ARPA-PARC project that invented the internet and PC, and similar efforts. He must see every day how science funding can work so much better than is normal in Britain.

Today, the UK a) ties research up in appalling bureaucracy, such as requiring multi-stage procurement processes literally to change a lightbulb, and b) does not fund it enough. The bureaucracy around basic science is so crazy that a glitch in paper work means thousands of animals are secretly destroyed in ways the public would be appalled to learn if made public.

Few in SW1 take basic science research seriously. And in all the debates over Brexit, practically the entire focus is 1980s arguments over the mechanism for regulating product markets created by Delors to centralise power in Brussels — the Internal Market (aka Single Market). Thirty years after they committed to this mechanism and two years after the referendum that blew it up, most MPs still don’t understand what it is and how it works. Dismally, the last two years has been a sort of remedial education programme and there has been practically zero discussion about how Britain could help create the future

During the referendum, Vote Leave argued that the dreadful Cameron/Osborne immigration policy (including the net migration target) was damaging and said we should make Britain MORE welcoming to scientists. Obviously Remain-SW1 likes to pretend that the May/Hammond Remain team’s shambles is the only possible version of Brexit. Nothing could be further from the truth. If the government had funded the NHS, ditched the ‘tens of thousands’ absurdity, and, for example, given maths, physics and computer science PhDs ‘free movement’ then things would be very different now — and Corbyn would probably be a historical footnote.

Regardless of how you voted in the referendum, reasonable people outside the rancid environment of SW1 should pressure their MPs to take their responsibilities to science x100 more seriously than they do.

I strongly urge you to read it all, send it to your MP, and politely ask for action…

(Their phrase ‘creating the future’ invokes Alan Kay’s famous line — the best way to predict the future is to invent it.)


Science holds the key, by James & Matthew Phillips

The 2008 crisis should have led us to reshape how our economy works. But a decade on, what has really changed? The public knows that the same attitude that got us into the previous economic crisis will not bring us long-term prosperity, yet there is little vision from our leaders of what the future should look like. Our politicians are sleeping, yet have no dreams. To solve this, we must change emphasis from creating “growth” to creating the future: the former is an inevitable product of the latter.

Britain used to create the future, and we must return to this role by turning to scientists and engineers. Science defined the last century by creating new industries. It will define this century too: robotics, clean energy, artificial intelligence, cures for disease and other unexpected advances lie in wait. The country that gives birth to these industries will lead the world, and yet we seem incapable of action.

So how can we create new industries quickly? A clue lies in a small number of institutes that produced a strikingly large number of key advances. Bell Labs produced much of the technology underlying computing. The Palo Alto Research Centre did the same for the internet. There are simple rules of thumb about how great science arises, embodied in such institutes. They provided ambitious long-term funding to scientists, avoided unnecessary bureaucracy and chased high-risk, high-reward projects.

Today, scientists spend much of their time completing paperwork. A culture of endless accountability has arisen out of a fear of misspending a single pound. We’ve seen examples of routine purchases of LEDs that cost under £10 having to go through a nine-step bureaucratic review process.

Scientists on the cusp of great breakthroughs can be slowed by years mired in review boards and waiting on a decision from on high. Their discoveries are thus made, and capitalised on, elsewhere. We waste money, miss patents, lose cures and drive talented scientists away to high-paid jobs. You don’t cure cancer with paperwork. Rather than invigilate every single decision, we should do spot checks retrospectively, as is done with tax returns.

A similar risk aversion is present in the science funding process. Many scientists are forced to specify years in advance what they intend to do, and spend their time continually applying for very short, small grants. However, it is the unexpected, the failures and the accidental, which are the inevitable cost and source of fruit in the scientific pursuit. It takes time, it takes long-term thinking, it takes flexibility. Peter Higgs, Nobel laureate who predicted the Higgs Boson, says he wouldn’t stand a chance of being funded today for lack of a track record. This leads scientists collectively to pursue incremental, low-risk, low-payoff work.

The current funding system is also top-down, prescriptive and homogenous, administered centrally from London. It is slow to respond to change and cut off from the real world.

We should return to funding university departments more directly, allowing more rapid, situation-aware decision-making of the kind present in start-ups, and create a diversity of funding systems. This is how the best research facilities in history operated, yet we do not learn their key lesson: that science cannot be managed by central edict, but flourishes through independent inquiry.

While Britain built much of modern science, today it neglects it, lagging behind other comparable nations in funding, and instead prioritising a financial industry prone to blowing up. Consider that we spent more money bailing out the banks in a single year than we have on science in the entirety of history.

We scarcely pause to consider the difference in return on investment. Rather than prop up old industries, we should invest in world-leading research institutes with a specific emphasis on high-risk, high-payoff research.

Those who say this is not government’s role fail the test of history. Much great science has come from government investment in times of crisis. Without Nasa, there would be no SpaceX. These government investments were used to provide a long-term, transformative vision on a scale that cannot be achieved through private investment alone – especially where there is a high risk of failure but high reward in success. The payoff of previous investments was enormous, so why not replicate the defence funding agencies that led to them with peacetime civilian equivalents?

In order to be the nation where new discoveries are made, we must take decisive steps to make the UK a magnet for talented young scientists.

However, a recent report on ensuring a successful UK research endeavour scarcely mentioned young scientists at all. An increased focus on this goal, alongside simple steps like long-term funding and guaranteed work visas for their spouses, would go a long way. In short, we should be to scientific innovation what we are to finance: a highly connected nerve centre for the global economy.

The political candidate that can leverage a pro-science platform to combine economic stimulus with the reality of economic pragmatism will transform the UK. We should lead the future by creating it.

James Phillips is a PhD student in neuroscience at the HHMI Janelia Research Campus in the US and the University of Cambridge. 
Matthew Phillips is a PhD student in neuroscience at the Sainsbury Wellcome Centre, University College London


UPDATE

Alan Kay, the brilliant researcher I mentioned above, happened to read this blog and posted this comment which I will also paste below here…

[From Alan Kay]

Good advice! However, I’m afraid that currently in the US there is nothing like the fabled Bell Labs or ARPA-PARC funding, at least in computing where I’m most aware of what is and is not happening (I’m the “Alan Kay” of the famous quote).

It is possible that things were still better a few years ago in the US than in the UK (I live in London half the year and in Los Angeles the other half). But I have some reasons to doubt. Since the new “president”, the US does not even have a science advisor, nor is there any sign of desire for one.

A visit to the classic Bell Labs of its heyday would reveal many things. One of the simplest was a sign posted randomly around: “Either do something very useful, or very beautiful”. Funders today won’t fund the second at all, and are afraid to fund at the risk level needed for the first.

It is difficult to sum up ARPA-PARC, but one interesting perspective on this kind of funding was that it was both long range and stratospherically visionary, and part of the vision was that good results included “better problems” (i.e. “problem finding” was highly valued and funded well) and good results included “good people” (i.e. long range funding should also create the next generations of researchers). in fact, virtually all of the researchers at Xerox PARC had their degrees funded by ARPA, they were “research results” who were able to get better research results.

Since the “D” was put on ARPA in the early 70s, it was then not able to do what it did in the 60s. NSF in the US never did this kind of funding. I spent quite a lot of time on some of the NSF Advisory Boards and it was pretty much impossible to bridge the gap between what was actually needed and the difficulties the Foundation has with congressional oversight (and some of the stipulations of their mission).

Bob Noyce (one of the founders of Intel) used to say “Wealth is created by Scientists, Engineers and Artists, everyone else just moves it around”.

Einstein said “We cannot solve important problems of the world using the same level of thinking we used to create them”.

A nice phrase by Vi Hart is “We must insure human wisdom exceeds human power”.

To make it to the 22nd century at all, and especially in better shape than we are now, we need to heed all three of these sayings, and support them as the civilization we are sometimes trying to become. It’s the only context in which “The best way to predict the future is to invent it” makes any useful sense.

Effective action #4b: ‘Expertise’, prediction and noise, from the NHS killing people to Brexit

In part A I looked at extreme sports as some background to the question of true expertise and the crucial nature of fast high quality feedback.

This blog looks at studies comparing expertise in many fields over decades, including work by Tetlock and Kahneman, and problems like — why people don’t learn to use even simple tools to stop children dying unnecessarily. There is a summary of some basic lessons at the end.

The reason for writing about this is that we will only improve the performance of government (at individual, team and institutional levels) if we reflect on:

  • what expertise really is and why do some very successful fields cultivate it effectively while others, like government, do not;
  • how to select much higher quality people (it’s insane people as ignorant and limited as me can have the influence we do in the way we do — us limited duffers can help in limited ways but why do we deliberately exclude ~100% of the most intelligent, talented, relentless, high performing people from fields with genuine expertise, why do we not have people like Fields Medallist Tim Gowers or Michael Nielsen as Chief Scientist  sitting ex officio in Cabinet?);
  • how to train people effectively to develop true expertise in skills relevant to government: it needs different intellectual content (PPE/economics are NOT good introductory degrees) and practice in practical skills (project management, making predictions and in general ‘thinking rationally’) with lots of fast, accurate feedback;
  • how to give them effective tools: e.g the Cabinet Room is worse in this respect than it was in July 1914 — at least then the clock and fireplace worked, and Lord Salisbury in the 1890s would walk round the Cabinet table gathering papers to burn in the grate — while today No10 is decades behind the state-of-the-art in old technologies like TV, doesn’t understand simple tools like checklists, and is nowhere with advanced technologies;
  • and how to ‘program’ institutions differently so that 1) people are more incentivised to optimise things we want them to optimise, like error-correction and predictive accuracy, and less incentivised to optimise bureaucratic process, prestige, and signalling as our institutions now do to a dangerous extent, and, connected, so that 2) institutions are much better at building high performance teams rather than continue normal rules that make this practically illegal, and so that 3) we have ‘immune systems’ to minimise the inevitable failures of even the best people and teams .

In SW1 now, those at the apex of power practically never think in a serious way about the reasons for the endemic dysfunctional decision-making that constitutes most of their daily experience or how to change it. What looks like omnishambles to the public and high performers in technology or business is seen by Insiders, always implicitly and often explicitly, as ‘normal performance’. ‘Crises’ such as the collapse of Carillion or our farcical multi-decade multi-billion ‘aircraft carrier’ project occasionally provoke a few days of headlines but it’s very rare anything important changes in the underlying structures and there is no real reflection on system failure.

This fact is why, for example, a startup created in a few months could win a referendum that should have been unwinnable. It was the systemic and consistent dysfunction of Establishment decision-making systems over a long period, with very poor mechanisms for good accurate feedback from reality, that created the space for a guerrilla operation to exploit.

This makes it particularly ironic that even after Westminster and Whitehall have allowed their internal consensus about UK national strategy to be shattered by the referendum, there is essentially no serious reflection on this system failure. It is much more psychologically appealing for Insiders to blame ‘lies’ (Blair and Osborne really say this without blushing), devilish use of technology to twist minds and so on. Perhaps the most profound aspect of broken systems is they cannot reflect on the reasons why they’re broken  — never mind take effective action. Instead of serious thought, we have high status Insiders like Campbell reduced to bathos with whining on social media about Brexit ‘impacting mental health’. This lack of reflection is why Remain-dominated Insiders lurched from failure over the referendum to failure over negotiations. OODA loops across SW1 are broken and this is very hard to fix — if you can’t orient to reality how do you even see your problem well? (NB. It should go without saying that there is a faction of pro-Brexit MPs, ‘campaigners’ and ‘pro-Brexit economists’ who are at least as disconnected from reality, often more, as the May/Hammond bunker.)

Screenshot 2018-06-05 10.05.19

In the commercial world, big companies mostly die within a few decades because they cannot maintain an internal system to keep them aligned to reality plus startups pop up. These two factors create learning at a system level — there is lots of micro failure but macro productivity/learning in which useful information is compressed and abstracted. In the political world, big established failing systems control the rules, suck in more and more resources rather than go bust, make it almost impossible for startups to contribute and so on. Even failures on the scale of the 2008 Crash or the 2016 referendum do not necessarily make broken systems face reality, at least quickly. Watching Parliament’s obsession with trivia in the face of the Cabinet’s and Whitehall’s contemptible failure to protect the interests of millions in the farcical Brexit negotiations is like watching the secretary to the Singapore Golf Club objecting to guns being placed on the links as the Japanese troops advanced.

Neither of the main parties has internalised the reality of these two crises. The Tories won’t face reality on things like corporate looting and the NHS, Labour won’t face reality on things like immigration and the limits of bureaucratic centralism. Neither can cope with the complexity of Brexit and both just look like I would look like in the ring with a professional fighter — baffled, terrified and desperate for a way to escape. There are so many simple ways to improve performance — and their own popularity! — but the system is stuck in such a closed loop it wilfully avoids seeing even the most obvious things and suppresses Insiders who want to do things differently…

But… there is a network of almost entirely younger people inside or close to the system thinking ‘we could do so much better than this’. Few senior Insiders are interested in these questions but that’s OK — few of them listened before the referendum either. It’s not the people now in power and running the parties and Whitehall who will determine whether we make Brexit a platform to contribute usefully to humanity’s biggest challenges but those that take over.

Doing better requires reflecting on what we know about real expertise…

*

How to distinguish between fields dominated by real expertise and those dominated by confident ‘experts’ who make bad predictions?

We know a lot about the distinction between fields in which there is real expertise and fields dominated by bogus expertise. Daniel Kahneman, who has published some of the most important research about expertise and prediction, summarises the two fundamental tests to ask about a field: 1) is there enough informational structure in the environment to allow good predictions, and 2) is there timely and effective feedback that enables error-correction and learning.

‘To know whether you can trust a particular intuitive judgment, there are two questions you should ask: Is the environment in which the judgment is made sufficiently regular to enable predictions from the available evidence? The answer is yes for diagnosticians, no for stock pickers. Do the professionals have an adequate opportunity to learn the cues and the regularities? The answer here depends on the professionals’ experience and on the quality and speed with which they discover their mistakes. Anesthesiologists have a better chance to develop intuitions than radiologists do. Many of the professionals we encounter easily pass both tests, and their off-the-cuff judgments deserve to be taken seriously. In general, however, you should not take assertive and confident people at their own evaluation unless you have independent reason to believe that they know what they are talking about.’ (Emphasis added.)

In fields where these two elements are present there is genuine expertise and people build new knowledge on the reliable foundations of previous knowledge. Some fields make a transition from stories (e.g Icarus) and authority (e.g ‘witch doctor’) to quantitative models (e.g modern aircraft) and evidence/experiment (e.g some parts of modern medicine/surgery). As scientists have said since Newton, they stand on the shoulders of giants.

How do we assess predictions / judgement about the future?

‘Good judgment is often gauged against two gold standards – coherence and correspondence. Judgments are coherent if they demonstrate consistency with the axioms of probability theory or propositional logic. Judgments are correspondent if they agree with ground truth. When gold standards are unavailable, silver standards such as consistency and discrimination can be used to evaluate judgment quality. Individuals are consistent if they assign similar judgments to comparable stimuli, and they discriminate if they assign different judgments to dissimilar stimuli.

‘Coherence violations range from base rate neglect and confirmation bias to overconfidence and framing effects (Gilovich, Griffith & Kahneman, 2002; Kahneman, Slovic & Tversky, 1982). Experts are not immune. Statisticians (Christensen-Szalanski & Bushyhead, 1981), doctors (Eddy, 1982), and nurses (Bennett, 1980) neglect base rates. Physicians and intelligence professionals are susceptible to framing effects and financial investors are prone to overconfidence.

‘Research on correspondence tells a similar story. Numerous studies show that human predictions are frequently inaccurate and worse than simple linear models in many domains (e.g. Meehl, 1954; Dawes, Faust & Meehl, 1989). Once again, expertise doesn’t necessarily help. Inaccurate predictions have been found in parole officers, court judges, investment managers in the US and Taiwan, and politicians. However, expert predictions are better when the forecasting environment provides regular, clear feedback and there are repeated opportunities to learn (Kahneman & Klein, 2009; Shanteau, 1992). Examples include meteorologists, professional bridge players, and bookmakers at the racetrack, all of whom are well-calibrated in their own domains.‘ (Tetlock, How generalizable is good judgment?, 2017.)

In another 2017 piece Tetlock explored the studies furtherIn the 1920s researchers built simple models based on expert assessments of 500 ears of corn and the price they would fetch in the market. They found that ‘to everyone’s surprise, the models that mimicked the judges’ strategies nearly always performed better than the judges themselves’ (Tetlock, cf. ‘What Is in the Corn Judge’s Mind?’, Journal of American Society for Agronomy, 1923). Banks found the same when they introduced models for credit decisions.

‘In other fields, from predicting the performance of newly hired salespeople to the bankruptcy risks of companies to the life expectancies of terminally ill cancer patients, the experience has been essentially the same. Even though experts usually possess deep knowledge, they often do not make good predictions

When humans make predictions, wisdom gets mixed with “random noise.”… Bootstrapping, which incorporates expert judgment into a decision-making model, eliminates such inconsistencies while preserving the expert’s insights. But this does not occur when human judgment is employed on its own…

In fields ranging from medicine to finance, scores of studies have shown that replacing experts with models of experts produces superior judgments. In most cases, the bootstrapping model performed better than experts on their own. Nonetheless, bootstrapping models tend to be rather rudimentary in that human experts are usually needed to identify the factors that matter most in making predictions. Humans are also instrumental in assigning scores to the predictor variables (such as judging the strength of recommendation letters for college applications or the overall health of patients in medical cases). What’s more, humans are good at spotting when the model is getting out of date and needs updating…

Human experts typically provide signal, noise, and bias in unknown proportions, which makes it difficult to disentangle these three components in field settings. Whether humans or computers have the upper hand depends on many factors, including whether the tasks being undertaken are familiar or unique. When tasks are familiar and much data is available, computers will likely beat humans by being data-driven and highly consistent from one case to the next. But when tasks are unique (where creativity may matter more) and when data overload is not a problem for humans, humans will likely have an advantage…

One might think that humans have an advantage over models in understanding dynamically complex domains, with feedback loops, delays, and instability. But psychologists have examined how people learn about complex relationships in simulated dynamic environments (for example, a computer game modeling an airline’s strategic decisions or those of an electronics company managing a new product). Even after receiving extensive feedback after each round of play, the human subjects improved only slowly over time and failed to beat simple computer models. This raises questions about how much human expertise is desirable when building models for complex dynamic environments. The best way to find out is to compare how well humans and models do in specific domains and perhaps develop hybrid models that integrate different approaches.‘ (Tetlock)

Kahneman also recently published new work relevant to this.

Research has confirmed that in many tasks, experts’ decisions are highly variable: valuing stocks, appraising real estate, sentencing criminals, evaluating job performance, auditing financial statements, and more. The unavoidable conclusion is that professionals often make decisions that deviate significantly from those of their peers, from their own prior decisions, and from rules that they themselves claim to follow.’

In general organisations spend almost no effort figuring out how noisy the predictions made by senior staff are and how much this costs. Kahneman has done some ‘noise audits’ and shown companies that management make MUCH more variable predictions than people realise.

‘What prevents companies from recognizing that the judgments of their employees are noisy? The answer lies in two familiar phenomena: Experienced professionals tend to have high confidence in the accuracy of their own judgments, and they also have high regard for their colleagues’ intelligence. This combination inevitably leads to an overestimation of agreement. When asked about what their colleagues would say, professionals expect others’ judgments to be much closer to their own than they actually are. Most of the time, of course, experienced professionals are completely unconcerned with what others might think and simply assume that theirs is the best answer. One reason the problem of noise is invisible is that people do not go through life imagining plausible alternatives to every judgment they make.

‘High skill develops in chess and driving through years of practice in a predictable environment, in which actions are followed by feedback that is both immediate and clear. Unfortunately, few professionals operate in such a world. In most jobs people learn to make judgments by hearing managers and colleagues explain and criticize—a much less reliable source of knowledge than learning from one’s mistakes. Long experience on a job always increases people’s confidence in their judgments, but in the absence of rapid feedback, confidence is no guarantee of either accuracy or consensus.’

Reviewing the point that Tetlock makes about simple models beating experts in many fields, Kahneman summarises the evidence:

‘People have competed against algorithms in several hundred contests of accuracy over the past 60 years, in tasks ranging from predicting the life expectancy of cancer patients to predicting the success of graduate students. Algorithms were more accurate than human professionals in about half the studies, and approximately tied with the humans in the others. The ties should also count as victories for the algorithms, which are more cost-effective…

‘The common assumption is that algorithms require statistical analysis of large amounts of data. For example, most people we talk to believe that data on thousands of loan applications and their outcomes is needed to develop an equation that predicts commercial loan defaults. Very few know that adequate algorithms can be developed without any outcome data at all — and with input information on only a small number of cases. We call predictive formulas that are built without outcome data “reasoned rules,” because they draw on commonsense reasoning.

‘The construction of a reasoned rule starts with the selection of a few (perhaps six to eight) variables that are incontrovertibly related to the outcome being predicted. If the outcome is loan default, for example, assets and liabilities will surely be included in the list. The next step is to assign these variables equal weight in the prediction formula, setting their sign in the obvious direction (positive for assets, negative for liabilities). The rule can then be constructed by a few simple calculations.

The surprising result of much research is that in many contexts reasoned rules are about as accurate as statistical models built with outcome data. Standard statistical models combine a set of predictive variables, which are assigned weights based on their relationship to the predicted outcomes and to one another. In many situations, however, these weights are both statistically unstable and practically unimportant. A simple rule that assigns equal weights to the selected variables is likely to be just as valid. Algorithms that weight variables equally and don’t rely on outcome data have proved successful in personnel selection, election forecasting, predictions about football games, and other applications.

‘The bottom line here is that if you plan to use an algorithm to reduce noise, you need not wait for outcome data. You can reap most of the benefits by using common sense to select variables and the simplest possible rule to combine them…

‘Uncomfortable as people may be with the idea, studies have shown that while humans can provide useful input to formulas, algorithms do better in the role of final decision maker. If the avoidance of errors is the only criterion, managers should be strongly advised to overrule the algorithm only in exceptional circumstances.

Jim Simons is a mathematician and founder of the world’s most successful ‘quant fund’, Renaissance Technologies. While market prices appear close to random and are therefore extremely hard to predict, they are not quite random and the right models/technology can exploit these small and fleeting opportunities. One of the lessons he learned early was: Don’t turn off the model and go with your gut. At Renaissance, they trust models over instincts. The Bridgewater hedge fund led by Ray Dalio is similar. After near destruction early in his career, Dalio explicitly turned towards explicit model building as the basis for decisions combined with radical attempts to create an internal system that incentivises the optimisation of error-correction. It works.

*

People fail to learn from even the great examples of success and the simplest lessons

One of the most interesting meta-lessons of studying high performance, though, is that simply demonstrating extreme success does NOT lead to much learning. For example:

  • ARPA and PARC created the internet and PC. The PARC research team was an extraordinary collection of about two dozen people who were managed in a very unusual way that created super-productive processes extremely different to normal bureaucracies. XEROX, which owned PARC, had the entire future of the computer industry in its own hands, paid for by its own budgets, and it simultaneously let Bill Gates and Steve Jobs steal everything and XEROX then shut down the research team that did it. And then, as Silicon Valley grew on the back of these efforts, almost nobody, including most of the billionaires who got rich from the dynamics created by ARPA-PARC, studied the nature of the organisation and processes and copied it. Even today, those trying to do edge-of-the-art research in a similar way to PARC right at the heart of the Valley ecosystem are struggling for long-term patient funding. As Alan Kay, one of the PARC team, said, ‘The most interesting thing has been the contrast between appreciation/exploitation of the inventions/contributions [of PARC] versus the almost complete lack of curiosity and interest in the processes that produced them. ARPA survived being abolished in the 1970s but it was significantly changed and is no longer the freewheeling place that it was in the 1960s when it funded the internet. In many ways DARPA’s approach now is explicitly different to the old ARPA (the addition of the ‘D’ was a sign of internal bureaucratic changes).

Screenshot 2018-06-05 14.55.00

  • ‘Systems management’ was invented in the 1950s and 1960s (partly based on wartime experience of large complex projects) to deal with the classified ICBM project and Apollo. It put man on the moon then NASA largely abandoned the approach and reverted to being (relative to 1963-9) a normal bureaucracy. Most of Washington has ignored the lessons ever since — look for example at the collapse of ObamaCare’s rollout, after which Insiders said ‘oh, looks like it was a system failure, wonder how we deal with this’, mostly unaware that America had developed a successful approach to such projects half a century earlier. This is particularly interesting given that China also studied Mueller’s approach to systems management in Apollo and as we speak is copying it in projects across China. The EU’s bureaucracy is, like Whitehall, an anti-checklist to high level systems management — i.e they violate almost every principle of effective action.
  • Buffett and Munger are the most successful investment partnership in world history. Every year for half a century they have explained some basic principles, particularly concerning incentives, behind organisational success. Practically no public companies take their advice and all around us in Britain we see vast corporate looting and politicians of all parties failing to act — they don’t even read the Buffett/Munger lessons and think about them. Even when given these lessons to read, they won’t read them (I know this because I’ve tried).

Perhaps you’re thinking — well, learning from these brilliant examples might be intrinsically really hard, much harder than Cummings thinks. I don’t think this is quite right. Why? Partly because millions of well-educated and normally-ethical people don’t learn even from much simpler things.

I will explore this separately soon but I’ll give just one example. The world of healthcare unnecessarily kills and injures people on a vast scale. Two aspects of this are 1) a deep resistance to learning from the success of very simple tools like checklists and 2) a deep resistance to face the fact that most medical experts do not understand statistics properly and their routine misjudgements cause vast suffering, plus warped incentives encourage widespread lies about statistics and irrational management. E.g People are constantly told things like ‘you’ve tested positive for X therefore you have X’ and they then kill themselves. We KNOW how to practically eliminate certain sorts of medical injury/death. We KNOW how to teach and communicate statistics better. (Cf. Professor Gigerenzer for details. He was the motivation for including things like conditional probabilities in the new National Curriculum.) These are MUCH simpler than building ICBMs, putting man on the moon, creating the internet and PC, or being great investors. Yet our societies don’t do them.

Why?

Because we do not incentivise error-correction and predictive accuracy. People are not incentivised to consider the cost of their noisy judgements. Where incentives and culture are changed, performance magically changes. It is the nature of the systems, not (mostly) the nature of the people, that is the crucial ingredient in learning from proven simple success. In healthcare like in government generally, people are incentivised to engage in wasteful/dangerous signalling to a terrifying degree — not rigorous thinking and not solving problems.

I have experienced the problem with checklists first hand in the Department for Education when trying to get the social worker bureaucracy to think about checklists in the context of avoiding child killings like Baby P. Professionals tend to see them as undermining their status and bureaucracies fight against learning, even when some great officials try really hard (as some in the DfE did such as Pamela Dow and Victoria Woodcock). ‘Social work is not the same as an airline Dominic’. No shit. Airlines can handle millions of people without killing one of them because they align incentives with predictive accuracy and error-correction.

Some appalling killings are inevitable but the social work bureaucracy will keep allowing unnecessary killings because they will not align incentives with error-correction. Undoing flawed incentives threatens the system so they’ll keep killing children instead — and they’re not particularly bad people, they’re normal people in a normal bureaucracy. The pilot dies with the passengers. The ‘CEO’ on over £150,000 a year presiding over another unnecessary death despite constantly increasing taxpayers money pouring in? Issue a statement that ‘this must never happen again’, tell the lawyers to redact embarrassing cockups on the grounds of ‘protecting someone’s anonymity’ (the ECHR is a great tool to cover up death by incompetence), fuck off to the golf course, and wait for the media circus to move on.

Why do so many things go wrong? Because usually nobody is incentivised to work relentlessly to suppress entropy, never mind come up with something new.

*

We can see some reasonably clear conclusions from decades of study on expertise and prediction in many fields.

  • Some fields are like extreme sport or physics: genuine expertise emerges because of fast effective feedback on errors.
  • Abstracting human wisdom into models often works better than relying on human experts as models are often more consistent and less noisy.
  • Models are also often cheaper and simpler to use.
  • Models do not have to be complex to be highly effective — quite the opposite, often simpler models outperform more sophisticated and expensive ones.
  • In many fields (which I’ve explored before but won’t go into again here) low tech very simple checklists have been extremely effective: e.g flying aircraft or surgery.
  • Successful individuals like Warren Buffett and Ray Dalio also create cognitive checklists to trap and correct normal cognitive biases that degrade individual and team performance.
  • Fields make progress towards genuine expertise when they make a transition from stories (e.g Icarus) and authority (e.g ‘witch doctor’) to quantitative models (e.g modern aircraft) and evidence/experiment (e.g some parts of modern medicine/surgery).
  • In the intellectual realm, maths and physics are fields dominated by genuine expertise and provide a useful benchmark to compare others against. They are also hierarchical. Social sciences have little in common with this.
  • Even when we have great examples of learning and progress, and we can see the principles behind them are relatively simple and do not require high intelligence to understand, they are so psychologically hard and run so counter to the dynamics of normal big organisations, that almost nobody learns from them. Extreme success is ‘easy to learn from’ in one sense and ‘the hardest thing in the world to learn from’ in another sense.

It is fascinating how remarkably little interest there is in the world of politics/government, and social sciences analysing politics/government, about all this evidence. This is partly because politics/government is an anti-learning and anti-expertise field, partly because the social sciences are swamped by what Feynman called ‘cargo cult science’ with very noisy predictions, little good feedback and learning, and a lot of chippiness at criticism whether it’s from statistics experts or the ‘ignorant masses’. Fields like ‘education research’ and ‘political science’ are particularly dreadful and packed with charlatans but much of economics is not much better (much pro- and anti-Brexit mainstream economics is classic ‘cargo cult’).

I have found there is overwhelmingly more interest in high technology circles than in government circles, but in high technology circles there is also a lot of incredulity and naivety about how government works — many assume politicians are trying and failing to achieve high performance and don’t realise that in fact nobody is actually trying. This illusion extends to many well-connected businessmen who just can’t internalise the reality of the apex of power. I find that uneducated people on 20k living hundreds of miles from SW1 generally have a more accurate picture of daily No10 work than extremely well-connected billionaires.

This is all sobering and is another reason to be pessimistic about the chances of changing government from ‘normal’ to ‘high performance’ — but, pessimism of the intellect, optimism of the will…

If you are in Whitehall now watching the Brexit farce or abroad looking at similar, you will see from page 26 HERE a checklist for how to manage complex government projects at world class levels (if you find this interesting then read the whole paper). I will elaborate on this. I am also thinking about a project to look at the intersection of (roughly) five fields in order to make large improvements in the quality of people, ideas, tools, and institutions that determine political/government decisions and performance:

  • the science of prediction across different fields (e.g early warning systems, the Tetlock/IARPA project showing dramatic performance improvements),
  • what we know about high performance (individual/team/organisation) in different fields (e.g China’s application of ‘systems management’ to government),
  • technology and tools (e.g Bret Victor’s work, Michael Nielsen’s work on cognitive technologies, work on human-AI ‘minotaur’ teams),
  • political/government decision making affecting millions of people and trillions of dollars (e.g WMD, health), and
  • communication (e.g crisis management, applied psychology).

Progress requires attacking the ‘system of systems’ problem at the right ‘level’. Attacking the problems directly — let’s improve policy X and Y, let’s swap ‘incompetent’ A for ‘competent’ B — cannot touch the core problems, particularly the hardest meta-problem that government systems bitterly fight improvement. Solving the explicit surface problems of politics and government is best approached by a more general focus on applying abstract principles of effective action. We need to surround relatively specific problems with a more general approach. Attack at the right level will see specific solutions automatically ‘pop out’ of the system. One of the most powerful simplicities in all conflict (almost always unrecognised) is: ‘winning without fighting is the highest form of war’. If we approach the problem of government performance at the right level of generality then we have a chance to solve specific problems ‘without fighting’ — or, rather, without fighting nearly so much and the fighting will be more fruitful.

This is not a theoretical argument. If you look carefully at ancient texts and modern case studies, you see that applying a small number of very simple, powerful, but largely unrecognised principles (that are very hard for organisations to operationalise) can produce extremely surprising results.

How to jump from the Idea to Reality? More soon…


Ps. Just as I was about to hit publish on this, the DCMS Select Committee released their report on me. The sentence about the Singapore golf club at the top comes to mind.