The unrecognised simplicities of effective action #3: lessons on ‘capturing the heavens’ from the ARPA/PARC project that created the internet & PC

Below is a short summary of some basic principles of the ARPA/PARC project that created the internet and the personal computer. I wrote it originally as part of an anniversary blog on the referendum but it is also really part of this series on effective action.

One of the most interesting aspects of this project, like Mueller’s reforms of NASA, is the contrast between 1) extreme effectiveness, changing the world in a profound way, and 2) the general reaction to the methods was not only a failure to learn but a widespread hostility inside established bureaucracies (public and private) to the successful approach: NASA dropped Mueller’s approach when he left and has never been the same, and XEROX closed PARC and fired Bob Taylor. Changing the world in a profound and beneficial way is not enough to put a dint in bureaucracies which operate on their own dynamics.

Warren Buffet explained decades ago how institutions actively fight against learning and fight to stay in a closed and vicious feedback loop:

‘My most surprising discovery: the overwhelming importance in business of an unseen force that we might call “the institutional imperative”. In business school, I was given no hint of the imperative’s existence and I did not intuitively understand it when I entered the business world. I thought then that decent, intelligence, and experienced managers would automatically make rational business decisions. But I learned the hard way that isn’t so. Instead rationality frequently wilts when the institutional imperative comes into play.

‘For example, 1) As if governed by Newton’s First Law, any institution will resist any change in its current direction. 2) … Corporate projects will materialise to soak up available funds. 3) Any business craving of the leader, however foolish, will quickly be supported by … his troops. 4) The behaviour of peer companies … will be mindlessly imitated.’

Many of the principles behind ARPA/PARC could be applied to politics and government but they will not be learned from ‘naturally’ inside the system. Dramatic improvements will only happen if a group of people force ‘system’ changes on how government works so it is open to learning.

I have modified the below very slightly and added some references.

*

ARPA/PARC and ‘capturing the heavens’: The best way to predict the future is to invent it

The panic over Sputnik brought many good things such as a huge increase in science funding. America also created the Advanced Research Projects Agency (ARPA, which later added ‘Defense’ and became DARPA). Its job was to fund high risk / high payoff technology development. In the 1960s and 1970s, a combination of unusual people and unusually wise funding from ARPA created a community that in turn invented the internet, or ‘the intergalactic network’ as Licklider originally called it, and the personal computer. One of the elements of this community was PARC, a research centre working for Xerox. As Bill Gates said, he and Steve Jobs essentially broke into PARC, stole their ideas, and created Microsoft and Apple.

The ARPA/PARC project is an example of how if something is set up properly then a tiny number of people can do extraordinary things.

  • PARC had about 25 people and about $12 million per year in today’s money.
  • The breakthroughs from the ARPA/PARC project  created over 35 TRILLION DOLLARS of value for society and counting.
  • The internet architecture they built, based on decentralisation and distributed control, has scaled up over ten orders of magnitude (1010) without ever breaking and without ever being taken down for maintenance since 1969.

The whole story is fascinating in many ways. I won’t go into the technological aspects. I just want to say something about the process.

What does a process that produces ideas that change the world look like?

One of the central figures was Alan Kay. One of the most interesting things about the project is that not only has almost nobody tried to repeat this sort of research but the business world has even gone out of its way to spread mis-information about it because it was seen as so threatening to business-as-usual.

I will sketch a few lessons from one of Kay’s pieces but I urge you to read the whole thing.

‘This is what I call “The power of the context” or “Point of view is worth 80 IQ points”. Science and engineering themselves are famous examples, but there are even more striking processes within these large disciplines. One of the greatest works of art from that fruitful period of ARPA/PARC research in the 60s and 70s was the almost invisible context and community that catalysed so many researchers to be incredibly better dreamers and thinkers. That it was a great work of art is confirmed by the world-changing results that appeared so swiftly, and almost easily. That it was almost invisible, in spite of its tremendous success, is revealed by the disheartening fact today that, as far as I’m aware, no governments and no companies do edge-of-the-art research using these principles.’

‘[W]hen I think of ARPA/PARC, I think first of good will, even before brilliant people… Good will and great interest in graduate students as “world-class researchers who didn’t have PhDs yet” was the general rule across the ARPA community.

‘[I]t is no exaggeration to say that ARPA/PARC had “visions rather than goals” and “funded people, not projects”. The vision was “interactive computing as a complementary intellectual partner for people pervasively networked world-wide”. By not trying to derive specific goals from this at the funding side, ARPA/PARC was able to fund rather different and sometimes opposing points of view.

‘The pursuit of Art always sets off plans and goals, but plans and goals don’t always give rise to Art. If “visions not goals” opens the heavens, it is important to find artistic people to conceive the projects.

‘Thus the “people not projects” principle was the other cornerstone of ARPA/PARC’s success. Because of the normal distribution of talents and drive in the world, a depressingly large percentage of organizational processes have been designed to deal with people of moderate ability, motivation, and trust. We can easily see this in most walks of life today, but also astoundingly in corporate, university, and government research. ARPA/PARC had two main thresholds: self-motivation and ability. They cultivated people who “had to do, paid or not” and “whose doings were likely to be highly interesting and important”. Thus conventional oversight was not only not needed, but was not really possible. “Peer review” wasn’t easily done even with actual peers. The situation was “out of control”, yet extremely productive and not at all anarchic.

‘”Out of control” because artists have to do what they have to do. “Extremely productive” because a great vision acts like a magnetic field from the future that aligns all the little iron particle artists to point to “North” without having to see it. They then make their own paths to the future. Xerox often was shocked at the PARC process and declared it out of control, but they didn’t understand that the context was so powerful and compelling and the good will so abundant, that the artists worked happily at their version of the vision. The results were an enormous collection of breakthroughs.

‘Our game is more like art and sports than accounting, in that high percentages of failure are quite OK as long as enough larger processes succeed… [I]n most processes today — and sadly in most important areas of technology research — the administrators seem to prefer to be completely in control of mediocre processes to being “out of control” with superproductive processes. They are trying to “avoid failure” rather than trying to “capture the heavens”.

‘All of these principles came together a little over 30 years ago to eventually give rise to 1500 Altos, Ethernetworked to: each other, Laserprinters, file servers and the ARPAnet, distributed to many kinds of end-users to be heavily used in real situations. This anticipated the commercial availability of this genre by 10-15 years. The best way to predict the future is to invent it.

‘[W]e should realize that many of the most important ARPA/PARC ideas haven’t yet been adopted by the mainstream. For example, it is amazing to me that most of Doug Engelbart’s big ideas about “augmenting the collective intelligence of groups working together” have still not taken hold in commercial systems. What looked like a real revolution twice for end-users, first with spreadsheets and then with Hypercard, didn’t evolve into what will be commonplace 25 years from now, even though it could have. Most things done by most people today are still “automating paper, records and film” rather than “simulating the future”. More discouraging is that most computing is still aimed at adults in business, and that aimed at nonbusiness and children is mainly for entertainment and apes the worst of television. We see almost no use in education of what is great and unique about computer modeling and computer thinking. These are not technological problems but a lack of perspective. Must we hope that the open-source software movements will put things right?

‘The ARPA/PARC history shows that a combination of vision, a modest amount of funding, with a felicitous context and process can almost magically give rise to new technologies that not only amplify civilization, but also produce tremendous wealth for the society. Isn’t it time to do this again by Reason, even with no Cold War to use as an excuse? How about helping children of the world grow up to think much better than most adults do today? This would truly create “The Power of the Context”.’

Note how this story runs contrary to how free market think tanks and pundits describe technological development. The impetus for most of this development came from government funding, not markets.

Also note that every attempt since the 1950s to copy ARPA and JASON (the semi-classified group that partly gave ARPA its direction) in the UK has been blocked by Whitehall. The latest attempt was in 2014 when the Cabinet Office swatted aside the idea. Hilariously its argument was ‘DARPA has had a lot of failures’ thus demonstrating extreme ignorance about the basic idea — the whole point is you must have failures and if you don’t have lots of failures then you are failing!

People later claimed that while PARC may have changed the world it never made any money for XEROX. This is ‘absolute bullshit’ (Kay). It made billions from the laser printer alone and overall Xerox made 250 times what they invested in PARC before they went bust. In 1983 they fired Bob Taylor, the manager of PARC and the guy who made it all happen.

‘They hated [Taylor] for the very reason that most companies hate people who are doing something different, because it makes middle and upper management extremely uncomfortable. The last thing they want to do is make trillions, they want to make a few millions in a comfortable way’ (Kay).

Someone finally listened to Kay recently. ‘YC Research’, the research arm of the world’s most successful (by far) technology incubator, is starting to fund people in this way. I am not aware of any similar UK projects though I know that a small network of people are thinking again about how something like this could be done here. If you can help them, take a risk and help them! Someone talk to science minister Jo Johnson but be prepared for the Treasury’s usual ignorant bullshit — ‘what are we buying for our money, and how can we put in place appropriate oversight and compliance?’ they will say!

*

As we ponder the future of the UK-EU relationship shaped amid the farce of modern Whitehall, we should think hard about the ARPA/PARC example: how a small group of people can make a huge breakthrough with little money but the right structure, the right ways of thinking, and the right motives.

Those of us outside the political system thinking ‘we know we can do so much better than this but HOW can we break through the bullshit?’ need to change our perspective and gain 80 IQ points.

This real picture is a metaphor for the political culture: ad hoc solutions that are either bad or don’t scale.

Screenshot 2017-06-14 16.58.14.png

ARPA said ‘Let’s get rid of all the wires’. How do we ‘get rid of all the wires’ and build something different that breaks open the closed and failing political cultures? Winning the referendum was just one step that helps clear away dead wood but we now need to build new things.

The ARPA vision that aligned the artists ‘like little iron filings’ was:

‘Computers are destined to become interactive intellectual amplifiers for everyone in the world universally networked worldwide’ (Licklider).

We need a motivating vision aimed not at tomorrow but at changing the basic wiring of  the whole system, a vision that can align ‘the little iron filings’, and then start building for the long-term.

I will go into what I think this vision could be and how to do it another day. I think it is possible to create something new that could scale very fast and enable us to do politics and government extremely differently, as different to today as the internet and PC were to the post-war mainframes. This would enable us to build huge long-term value for humanity in a relatively short time (less than 20 years). To create it we need a process as well suited to the goal as the ARPA/PARC project was and incorporating many of its principles.

We must try to escape the current system with its periodic meltdowns and international crises. These crises move 500-1,000 times faster than that of summer 1914. Our destructive potential is at least a million-fold greater than it was in 1914. Yet we have essentially the same hierarchical command-and-control decision-making systems in place now that could not even cope with 1914 technology and pace. We have dodged nuclear wars by fluke because individuals made snap judgements in minutes. Nobody who reads the history of these episodes can think that this is viable long-term, and we will soon have another wave of innovation to worry about with autonomous robots and genetic engineering. Technology gives us no option but to try to overcome evolved instincts like destroying out-group competitors.

Watch Alan Kay explain how to invent the future HERE and HERE.

This link has these seminal papers:

  • Man-Computer Symbiosis, Licklider (1960)
  • The computer as a communications device, Licklider & Taylor (1968)

Part I of this series is HERE.

Part II on the emergence of ‘systems management’, how George Mueller used it to put man on the moon, and a checklist of how successful management of complex projects is systematically different to how Whitehall (and other state bureaucracies) work HERE.


Ps. Kay also points out that the real computer revolution won’t happen until people fulfil the original vision of enabling children to use this powerful way of thinking:

‘The real printing revolution was a qualitative change in thought and argument that lagged the hardware inventions by almost two centuries. The special quality of computers is their ability to rapidly simulate arbitrary descriptions, and the real computer revolution won’t happen until children can learn to read, write, argue and think in this powerful new way. We should all try to make this happen much sooner than 200 or even 20 more years!’

Almost nobody in education policy is aware of the educational context for the ARPA/PARC project which also speaks volumes about the abysmal field of ‘education research/policy’. People rightly say ‘education tech has largely failed’ but very few are aware that many of the original ideas from Licklider, Engelbart et al have never been tried and the Apple and MS versions are not the original vision.

 

On the referendum #23, a year after victory: ‘a change of perspective is worth 80 IQ points’ & ‘how to capture the heavens’

‘Just like all British governments, they will act more or less in a hand to mouth way on the spur of the moment, but they will not think out and adopt a steady policy.’ Earl Cromer, 1896.

Fascinating that the same problems recur time after time, in almost every program, and that the management of the program, whether it happened to be government or industry, continues to avoid reality.’ George Mueller, pioneer of systems management and head of the Apollo programme to put man on the moon.

Traditional cultures, those that all humans lived in until quite recently and which still survive in pockets, don’t realise that they are living inside a particular perspective. They think that what they see is ‘reality’. It is, obviously, not their fault. It is not because they are stupid. It is a historical accident that they did/do not have access to mental models that help more accurate thinking about reality.

Westminster and the other political cultures dotted around the world are similar to these traditional cultures. They think they they are living in ‘reality’. The MPs and pundits get up, read each other, tweet at each other, give speeches, send press releases, have dinner, attack, fuck or fight each other, do the same tomorrow and think ‘this is reality’. Like traditional cultures they are wrong. They are living inside a particular perspective that enormously distorts reality. 

They are trapped in thinking about today and their careers. They are trapped in thinking about incremental improvements. Almost nobody has ever been part of a high performance team responsible for a complex project. The speciality is a hot take to explain post facto what one cannot predict. They mostly don’t know what they don’t know. They don’t understand the decentralised information processing that allows markets to enable complex coordination. They don’t understand how scientific research works and they don’t value it. Their daily activity is massively constrained by the party and state bureaucracies that incentivise behaviour very different to what humanity needs to create long-term value. As Michael Nielsen (author of Reinventing Science) writes:

‘[M]uch of our intellectual elite who think they have “the solutions” have actually cut themselves off from understanding the basis for much of the most important human progress.’

Unlike traditional cultures, our modern political cultures don’t have the excuse of our hunter-gatherer ancestors. We could do better. But it is very very hard to escape the core imperatives that make big bureaucracies — public companies as well as state bureaucracies — so bad at learning. Warren Buffet explained decades ago how institutions actively fight against learning and fight to stay in a closed and vicious feedback loop:

‘My most surprising discovery: the overwhelming importance in business of an unseen force that we might call “the institutional imperative”. In business school, I was given no hint of the imperative’s existence and I did not intuitively understand it when I entered the business world. I thought then that decent, intelligence, and experienced managers would automatically make rational business decisions. But I learned the hard way that isn’t so. Instead rationality frequently wilts when the institutional imperative comes into play.

‘For example, 1) As if governed by Newton’s First Law, any institution will resist any change in its current direction. 2) … Corporate projects will materialise to soak up available funds. 3) Any business craving of the leader, however foolish, will quickly be supported by … his troops. 4) The behaviour of peer companies … will be mindlessly imitated.’

Almost nobody really learns from the world’s most successful investor about investing and how to run a successful business with good corporate governance. (People read what he writes but almost no investors choose to operate long-term like him, I think it is still true that not a single public company has copied his innovations with corporate governance like ‘no pay for company directors’, and governments have consistently rejected his and Munger’s advice about controlling the looting of public companies by management.) Almost nobody really learns how to do things better from the experience of dealing with this ‘institutional imperative’. We fail over and over again in the same way, trusting in institutions that are programmed to fail.

It is very very hard for humans to lift our eyes from today and to go out into the future and think about what could be done to bring the future back to the present. Like ants crawling around on the leaf, we political people only know our leaf.

Science has shown us a different way. Newton looked up from his leaf, looked far away from today, and created a new perspective — a new model of reality. It took an extreme genius to discover something like calculus but once discovered billions of people who are far from being geniuses can use this new perspective. Science advances by turning new ideas into standard ideas so each generation builds on the last.

Politics does the equivalent of constantly trying to reinvent children’s arithmetic and botching it. It does not build reliable foundations of knowledge. Archimedes is no longer cutting edge. Thucydides and Sun Tzu are still cutting edge. Even though Tetlock and others have shown how to start making similar progress with politics, our political cultures fiercely resist learning and fight ferociously to stay in closed and failing feedback loops.

In many ways our political culture has regressed as it has become more and more audio-visual and less and less literate. (Only 31% of US college graduates can read at a basic level. I’d guess it’s similar here. See end.) I’ve experimented with the way Jeff Bezos runs meetings at Amazon: i.e start the meeting with giving people a 5-10 page memo to read. Impossible in Westminster, nobody will sit and read like that! Officials have tried and failed for a year to get senior ministers to engage with complex written material about the EU negotiations. TV news dominates politics and is extremely low-bandwidth: it contains a few hundred words and rarely uses graphics properly. Evan Davis illustrates a comment about ‘going down the plughole’ with a picture of water down a plughole and Nick Robinson illustrates a comment about ‘the economy taking off’ with a picture of a plane taking off. The constant flow of bullshit from the likes of Robert Peston and Jon Snow dominates the medium because competition has been impossible until recently. BUT, although technology is making these charlatans less relevant (good) it also creates new problems and will not necessarily improve the culture.

Watching political news makes you dumber — switch it off and read books! If you work in it, either QUIT or go on holiday and come back determined to subvert it. How? Start with a previous blog which has some ideas, like tracking properly which people have a record of getting things right and wrong. Every editor I’ve suggested this to winces and says ‘impossible’. Insiders fear accountability and competition.

Today, the anniversary of the referendum, is a good day to forget the babble in the bubble and think about lessons from another project that changed the world, the famous ARPA/PARC team of the 1960s and 1970s.

*

ARPA/PARC and ‘capturing the heavens’: The best way to predict the future is to invent it

The panic over Sputnik brought many good things such as a huge increase in science funding. America also created the Advanced Research Projects Agency (ARPA, which later added ‘Defense’ and became DARPA). Its job was to fund high risk / high payoff technology development. In the 1960s and 1970s, a combination of unusual people and unusually wise funding from ARPA created a community that in turn invented the internet, or ‘the intergalactic network’ as Licklider originally called it, and the personal computer. One of the elements of this community was PARC, a research centre working for Xerox. As Bill Gates said, he and Steve Jobs essentially broke into PARC, stole their ideas, and created Microsoft and Apple.

The ARPA/PARC project has created over 35 TRILLION DOLLARS of value for society and counting.

The whole story is fascinating in many ways. I won’t go into the technological aspects. I just want to say something about the process.

What does a process that produces ideas that change the world look like?

One of the central figures was Alan Kay. One of the most interesting things about the project is that not only has almost nobody tried to repeat this sort of research but the business world has even gone out of its way to spread mis-information about it because it was seen as so threatening to business-as-usual.

I will sketch a few lessons from one of Kay’s pieces but I urge you to read the whole thing.

‘This is what I call “The power of the context” or “Point of view is worth 80 IQ points”. Science and engineering themselves are famous examples, but there are even more striking processes within these large disciplines. One of the greatest works of art from that fruitful period of ARPA/PARC research in the 60s and 70s was the almost invisible context and community that catalysed so many researchers to be incredibly better dreamers and thinkers. That it was a great work of art is confirmed by the world-changing results that appeared so swiftly, and almost easily. That it was almost invisible, in spite of its tremendous success, is revealed by the disheartening fact today that, as far as I’m aware, no governments and no companies do edge-of-the-art research using these principles.’

‘[W]hen I think of ARPA/PARC, I think first of good will, even before brilliant people… Good will and great interest in graduate students as “world-class researchers who didn’t have PhDs yet” was the general rule across the ARPA community.

‘[I]t is no exaggeration to say that ARPA/PARC had “visions rather than goals” and “funded people, not projects”. The vision was “interactive computing as a complementary intellectual partner for people pervasively networked world-wide”. By not trying to derive specific goals from this at the funding side, ARPA/PARC was able to fund rather different and sometimes opposing points of view.

‘The pursuit of Art always sets off plans and goals, but plans and goals don’t always give rise to Art. If “visions not goals” opens the heavens, it is important to find artistic people to conceive the projects.

‘Thus the “people not projects” principle was the other cornerstone of ARPA/PARC’s success. Because of the normal distribution of talents and drive in the world, a depressingly large percentage of organizational processes have been designed to deal with people of moderate ability, motivation, and trust. We can easily see this in most walks of life today, but also astoundingly in corporate, university, and government research. ARPA/PARC had two main thresholds: self-motivation and ability. They cultivated people who “had to do, paid or not” and “whose doings were likely to be highly interesting and important”. Thus conventional oversight was not only not needed, but was not really possible. “Peer review” wasn’t easily done even with actual peers. The situation was “out of control”, yet extremely productive and not at all anarchic.

‘”Out of control” because artists have to do what they have to do. “Extremely productive” because a great vision acts like a magnetic field from the future that aligns all the little iron particle artists to point to “North” without having to see it. They then make their own paths to the future. Xerox often was shocked at the PARC process and declared it out of control, but they didn’t understand that the context was so powerful and compelling and the good will so abundant, that the artists worked happily at their version of the vision. The results were an enormous collection of breakthroughs.

‘Our game is more like art and sports than accounting, in that high percentages of failure are quite OK as long as enough larger processes succeed… [I]n most processes today — and sadly in most important areas of technology research — the administrators seem to prefer to be completely in control of mediocre processes to being “out of control” with superproductive processes. They are trying to “avoid failure” rather than trying to “capture the heavens”.

‘All of these principles came together a little over 30 years ago to eventually give rise to 1500 Altos, Ethernetworked to: each other, Laserprinters, file servers and the ARPAnet, distributed to many kinds of end-users to be heavily used in real situations. This anticipated the commercial availability of this genre by 10-15 years. The best way to predict the future is to invent it.

‘[W]e should realize that many of the most important ARPA/PARC ideas haven’t yet been adopted by the mainstream. For example, it is amazing to me that most of Doug Engelbart’s big ideas about “augmenting the collective intelligence of groups working together” have still not taken hold in commercial systems. What looked like a real revolution twice for end-users, first with spreadsheets and then with Hypercard, didn’t evolve into what will be commonplace 25 years from now, even though it could have. Most things done by most people today are still “automating paper, records and film” rather than “simulating the future”. More discouraging is that most computing is still aimed at adults in business, and that aimed at nonbusiness and children is mainly for entertainment and apes the worst of television. We see almost no use in education of what is great and unique about computer modeling and computer thinking. These are not technological problems but a lack of perspective. Must we hope that the open-source software movements will put things right?

‘The ARPA/PARC history shows that a combination of vision, a modest amount of funding, with a felicitous context and process can almost magically give rise to new technologies that not only amplify civilization, but also produce tremendous wealth for the society. Isn’t it time to do this again by Reason, even with no Cold War to use as an excuse? How about helping children of the world grow up to think much better than most adults do today? This would truly create “The Power of the Context”.’

Note how this story runs contrary to how free market think tanks and pundits describe technological development. The impetus for most of this development came from government funding, not markets.

Also note that every attempt since the 1950s to copy ARPA and JASON (the semi-classified group that partly gave ARPA its direction) in the UK has been blocked by Whitehall. The latest attempt was in 2014 when the Cabinet Office swatted aside the idea. Hilariously its argument was ‘DARPA has had a lot of failures’ thus demonstrating extreme ignorance about the basic idea — the whole point is you must have failures and if you don’t have lots of failures then you are failing!

People later claimed that while PARC may have changed the world it never made any money for XEROX. This is ‘absolute bullshit’ (Kay). It made billions from the laser printer alone and overall Xerox made 250 times what they invested in PARC before they went bust. In 1983 they fired Bob Taylor, the manager of PARC and the guy who made it all happen.

‘They hated [Taylor] for the very reason that most companies hate people who are doing something different, because it makes middle and upper management extremely uncomfortable. The last thing they want to do is make trillions, they want to make a few millions in a comfortable way’ (Kay).

Someone finally listened to Kay recently. ‘YC Research’, the research arm of the world’s most successful (by far) technology incubator, is starting to fund people in this way. I am not aware of any similar UK projects though I know that a small network of people are thinking again about how something like this could be done here. If you can help them, take a risk and help them! Someone talk to science minister Jo Johnson but be prepared for the Treasury’s usual ignorant bullshit — ‘what are we buying for our money, and how can we put in place appropriate oversight and compliance?’ they will say!

Why is this relevant to the referendum?

As we ponder the future of the UK-EU relationship shaped amid the farce of modern Whitehall, we should think hard about the ARPA/PARC example: how a small group of people can make a huge breakthrough with little money but the right structure, the right ways of thinking, and the right motives.

Those of us outside the political system thinking ‘we know we can do so much better than this but HOW can we break through the bullshit?’ need to change our perspective and gain 80 IQ points.

This real picture is a metaphor for the political culture: ad hoc solutions that are either bad or don’t scale.

Screenshot 2017-06-14 16.58.14.png

ARPA said ‘Let’s get rid of all the wires’. How do we ‘get rid of all the wires’ and build something different that breaks open the closed and failing political cultures? Winning the referendum was just one step that helps clear away dead wood but we now need to build new things.

The ARPA vision that aligned the artists ‘like little iron filings’ was:

‘Computers are destined to become interactive intellectual amplifiers for everyone in the world universally networked worldwide’ (Licklider).

We need a motivating vision aimed not at tomorrow but at changing the basic wiring of  the whole system, a vision that can align ‘the little iron filings’, and then start building for the long-term.

I will go into what I think this vision could be and how to do it another day. I think it is possible to create something new that could scale very fast and enable us to do politics and government extremely differently, as different to today as the internet and PC were to the post-war mainframes. This would enable us to build huge long-term value for humanity in a relatively short time (less than 20 years). To create it we need a process as well suited to the goal as the ARPA/PARC project was and incorporating many of its principles.

We must try to escape the current system with its periodic meltdowns and international crises. These crises move 500-1,000 times faster than that of summer 1914. Our destructive potential is at least a million-fold greater than it was in 1914. Yet we have essentially the same hierarchical command-and-control decision-making systems in place now that could not even cope with 1914 technology and pace. We have dodged nuclear wars by fluke because individuals made snap judgements in minutes. Nobody who reads the history of these episodes can think that this is viable long-term, and we will soon have another wave of innovation to worry about with autonomous robots and genetic engineering. Technology gives us no option but to try to overcome evolved instincts like destroying out-group competitors.

In a previous blog I outlined how the ‘systems management’ approach used to put man on the moon provides principles for a new approach.

*

Ironically, one of the very few people in politics who understood the sort of thinking needed was … Jean Monnet, the architect of the EEC/EU! Monnet understood how to step back from today and build institutions. He worked operationally to prepare the future:

‘If there was stiff competition round the centres of power, there was practically none in the area where I wanted to work – preparing the future.’

Monnet was one of the few people in modern politics who really deserve the label ‘genius’. The story of how he wangled the creation of his institutions through the daily chaos of post-war politics is a lesson to anybody who wants to get things done.

But the institutions he created are in many ways the opposite of what the world needs. Their core operating principle is perpetual centralisation of power in the hands of an all powerful bureaucracy (Commission) and Court (ECJ). Nothing that works well in the world works like this!

Thanks to the prominence of Farage the dominant story among educated people is that those who got us out of the EU want to take us back to the pre-1914 era of hostile competing nation states. Nothing could be further from the truth. The key people in Vote Leave wanted and want not just what is best for Britain but what is best for all humanity. We want more international cooperation, not less. The problem with the EU is not that it is about international cooperation but that it is so bad at it and actually undermines it.

Britain leaving forces those with power to ask: how can all European countries trade freely and cooperate without subscribing to Monnet’s bureaucratic centralism? This will help Europe in the long-term. To those who favour this bureaucratic centralism and uniformity, reflect on the different trajectories of Europe and China post-Renaissance. In Europe, regulatory competition (so Columbus could chase funding in Spain after rejection in Portugal) brought immense gains. In China, centrally directed uniformity led to centuries of stagnation. America’s model of competitive federalism created by the founding fathers has been a far more effective engine of civilisation, growth, and new knowledge than the Monnet-Delors Single Market model.

If Britain were to focus on science and education with huge resources and a new-found seriousness, then this regulatory diversity would help not just Britain but all Europe and the global science community. We could make Britain the best place in the world to be for those who can invent the future. Like Alan Kay and his colleagues, we could create whole new industries. We could call Jeff Bezos and say, ‘Ok Jeff, you want a permanent international manned moon base, let’s talk about who does what, but not with that old rocket technology.’ No country on earth funds science as well as we already know how it could be done — that is something for Britain to do that would create real long-term value for humanity, instead of the ‘punching above our weight’ and ‘special relationship’ bullshit that passes for strategy in London. How we change our domestic institutions is within our power and will have much much greater influence on our long-term future than whatever deal is botched together with Brussels. We have the resources. But can we break the system open? If we don’t then we’re likely to go down the path we were already going down inside the EU, like the deluded Norma Desmond in Sunset Boulevard claiming ‘I am big, it’s the pictures that got small.’

*

Vote Leave and ‘good will’

Although Vote Leave was enmeshed in a sort of collective lunacy we managed, barely, to fend it off from the inner working of the campaign. Much of my job (sadly) was just trying to maintain a cordon around the core team so they could deliver the campaign with as little disruption as possible. We managed this because among the core people we had great good will. The stories of the campaign focus on the lunacy, but the people who really made it work remember the goodwill.

A year ago tonight I was sitting alone in a room thinking ‘we’ve won, now…’ when the walls started rumbling. At first I couldn’t make it out then, as Tim Shipman tells the story in his definitive book on the campaign, I heard ‘Dom, Dom, DOM’ — the team had declared victory. I went next door…

Thanks to everybody who sacrificed something. As I said that night and as I said in my long blog on the campaign, I’ve been given credit I don’t deserve and which rightly belongs to others — Cleo Watson, Richard ‘Ricardo’ Howell, Brother Starkie, Oliver Lewis, Lord Suart et al. Now, let’s think about what should come next…

 

Watch Alan Kay explain how to invent the future HERE and HERE.


Ps. Kay also points out that the real computer revolution won’t happen until people fulfil the original vision of enabling children to use this powerful way of thinking:

‘The real printing revolution was a qualitative change in thought and argument that lagged the hardware inventions by almost two centuries. The special quality of computers is their ability to rapidly simulate arbitrary descriptions, and the real computer revolution won’t happen until children can learn to read, write, argue and think in this powerful new way. We should all try to make this happen much sooner than 200 or even 20 more years!’

Almost nobody in education policy is aware of the educational context for the ARPA/PARC project which also speaks volumes about the abysmal field of ‘education research/policy’.

* Re the US literacy statistic, cf. A First Look at the Literacy of America’s Adults in the 21st Century, National Assessment of Adult Literacy, U.S. Dept of Education, NCES 2006.

 

 

 

Unrecognised simplicities of effective action #2(b): the Apollo programme, the Tory train wreck, and advice to spads starting work today

A few months ago I put a paper on my blog: The unrecognised simplicities of effective action #2: ‘Systems engineering’ and ‘systems management’ — ideas from the Apollo programme for a ‘systems politics’.

It examined the history of the classified programme to build ICBMs and the way in which George Mueller turned the failing NASA bureaucracy into an organisation that could put man on the moon. The heart of the paper is about the principles behind effective management of complex projects. These principles are relevant to Government, politics, and campaigns.

The paper is long as I thought it worthwhile to tell some of the detailed story. At the suggestion of various spads, ministers, hacks and so on I have cut and pasted the conclusion below particularly for those starting new jobs today. This is in the form of a crude checklist that compares a) the principles of Mueller’s systems management and b) how Whitehall actually works.

You will see that Whitehall operates on exactly opposite principles to those organisations where high performance creates real value. You will also soon see that you are now in a culture in which almost nobody is aware of this and anybody who suggests it sinks their career. In your new department, failure is so normal it is not defined as ‘failure’. Officials lose millions and get a gong. There is little spirit of public service or culture of responsibility. The most political people are promoted and the most competent people, like Victoria Woodcock, leave. The very worst officials are often put in charge of training the next generation. For most powerful officials, the most important thing is preserving the system, closed and impregnable. Unlike for ministers, the TV blaring with DISASTER is of no concern – provided it is the Minister in the firing line not them – and the responsible officials will happily amble to the tube at 4pm while political careers hang in the balance and you draft statements taking ‘full responsibility’ for things you knew nothing about and would have been prohibited from fixing if you had.

For all those spads in particular who are moving into new jobs, it is worth reflecting on the deep principles that actually determine why things work and do not work. Nobody will explain these to you or talk to you about them. Sadly, few MPs these days understand the crucial role of management – they tend to think of it like science as a rather lowly skill beneath their Olympian status – so you will also probably have to cope with the fact that your minister is more interested in keeping one step ahead of Simon Walters (they won’t). The thing that officials will try hardest to do is convey to you that you have no role in personnel decisions and/or management.

If you accept that, you are accepting at the start that you will achieve very little. The reason why Gove’s team got much more done than ANY insider thought was possible – including Cameron and the Perm Sec – was because we bent or broke the rules and focused very hard on a) replacing rubbish officials and bringing in people from outside and b) project management.

You cannot reform the way the civil service works. Only a PM can do that and there is no chance of May doing it – she blew her chance and her reward is to be pushed around by Heywood and Sue Gray until her colleagues pull the plug and start the leadership campaign. You should assume that won’t be long so focus, manage a few priorities with daily and weekly timetables, and use embarrassing errors to negotiate secret deals with the Perm Sec to move rubbish officials out of your priority areas – trust me, Perm Secs understand this game and will do deals with alacrity to make their lives easier. Officials are less politically biased than you probably have been told – they are much more concerned with avoiding hard work and protecting the system than in resisting specific policies, and you can exploit this. Make alliances with the good officials who still have hope and have not been broken by the system, there are surprisingly many who will pop up if they think you actually care about the public rather than party interests.

You will also notice that fundamental issues of organisational culture described below explain the shambles of CCHQ over the past 8 weeks: the lack of information sharing, the lack of orientation, the culture of blaming juniors for the failures of overpaid senior people, bottlenecks preventing fast decisions, endless small errors compounding into a broken organisation because nobody knows who is responsible for what and so on. Every failing organisation has the same stories, people find it very hard to learn from the most successful organisations and people.

To the extent Vote Leave was successful, it was partly because I consciously tried to copy Mueller in various ways, though given my own severe limitations this was patchy. If you ever get the chance to exercise leadership, try to copy people like Mueller who tried to make the world better and build an organisation that people were proud to serve.

Finally, consider the basic condition that allows Westminster and Whitehall to be so rubbish and get away with it: they are not just monopolies, they set the rules of the game, and both the civil service and the parties make it almost impossible for outsiders to influence anything. But a) the combination of the 2008 crisis, Brexit, and extreme unhappiness about politics as usual provides a potentially powerful fuel for an insurgency, and b) technology provides opportunities for startups to catch public imagination and scale extremely fast. I’ve always been sceptical of the idea of a new UK party of any sort but I increasingly think there is a chance that a handful of entrepreneurs could start a sort of anti-party to exploit the broken system and create something which confounds the right/centre/left broken mental model that dominates SW1 and which combines Mueller’s principles with Silicon Valley technology.

If the Tory Party does not make some profound changes fast, then it faces being blamed for the disintegration of Brexit talks and the election of Corbyn after which it is possible that, rather than attempting a coup to take them over, entrepreneurs may decide it is more rational to build something that ploughs them into the earth next to Corbyn.

I said since last summer that if the Tory Party tried to carry on with Brexit and government using the same broken Downing Street operation, which spends its time on crap spin and has almost no capacity for serious management, and the same broken political operation, dominated by people who have failed to persuade the country convincingly for many years, then they would blow up. They failed to change Downing Street and they ran yet another fundamentally misconceived campaign that blew massive structural advantages. Kaboom.

[[Within minutes of publishing this blog I got the following email from someone I haven’t met but who I know was inside CCHQ with the para above highlighted and these words: ‘This is exactly my depressing experience – shit show run by people who don’t care about anything other than their jobs.’]]

MPs of all parties need to realise that the referendum makes it impossible to carry on with your usual bullshit – it forces changes upon you even though you want to carry on with the old games. The first set of MPs that realise this and change their operating principles will quickly overwhelm the others: there is a huge first-mover advantage especially in a field characterised by institutional incompetence that is susceptible to external shocks (terror, financial collapse) and which is opening up to technological disruption. And you will only get on top of Brexit if you realise that leaving the EU is a systems problem requiring a systems response and this means a radically different organisation of the UK negotiating team. The challenge is not far short of the political equivalent of the Apollo program and it needs similarly imaginative management.

For those who do want to do something better, the below will be useful. I encourage you to read the whole history HERE but for those rushing through a sandwich on Day 1 this summary will help you think of the big picture. If you want a detailed tutorial on how the civil service works then read The Hollow Men HERE

[Added later… It is also very instructive that despite the triumph of Mueller’s methods, NASA itself abandoned them after he left and has never recovered. Even spectacular success on a world-changing project is not enough to beat bureaucratic inertia. Also, the US Government passed so many laws that Mueller himself said in later life it would be impossible to repeat Apollo without making it a classified ‘black’ project to evade the regulations. JSOC, US classified special forces, has to run a lot of its standard procurement via ‘black’ procurement processes just to get anything done. The abysmal procurement rules imposed under the Single Market are just one of the good reasons for us to get out of the SM as well as the EU. I had to deal with them a lot in the DfE and had to find ways to cheat them a lot to get things done faster and cheaper. They add billions to costs every year and Whitehall refused for years even to assess this huge area to avoid undermining support for the EU.]

*

Excerpt from The unrecognised simplicities of effective action #2 (p.28ff) 

Core lessons [of Mueller’s systems management] for politics?

Finally, I will summarise some of the core lessons of systems management that could be applied to re-engineering political institutions such as Downing Street.  Mueller’s approach meant an extreme focus on some core principles:

  • Organisation-wide orientation. Everybody in a large organisation must understand as much about the goals and plans as possible. Whitehall now works on opposite principles: I doubt a single department has proper orientation across most of the organisation (few will have it even across the top 10 people), never mind a whole government. This is partly because most ministers fail at the first hurdle — developing coherent goals — so effective orientation is inherently impossible.
  • Integration. There must be an overall approach in which the most important elements fit together, including in policy, management, and communications. Failures in complex projects, from renovating your house to designing a new welfare system, often occur at interfaces between parts. Whitehall now works on opposite principles: for example, Cameron and Osborne approached important policy on immigration/welfare in the opposite way by 1) promising to reduce immigration to less than 100,000 while simultaneously 2) having no legal tools to do this (and even worse promising to change this then failing in the EU renegotiation) and 3) having welfare policies that incentivised more immigration then 4) announcing a new living wage thus increasing incentives further for immigration. They emphasised each element as part of short-term political games and got themselves into a long-term inescapable mess.
  • Extreme transparency and communication, horizontally as well as hierarchically. More, richer, deeper communication so that ‘all of us understand what was going on throughout the program… [C]ommunications on a level that is free and easy and not constrained by the fact that you’re the boss… [This was] the secret of the success of the program, because so many programs fail because everybody doesn’t know what it is they are supposed to do’ (Mueller). Break information and management silos — a denser network of information and commands is necessary and much of it must be decentralised and distributed between different teams, but with leadership having fast and clear information flow at the centre so problems are seen and tackled fast (a virtuous circle). There is very little that needs to be kept secret in government and different processes can easily be developed for that very small number of things. As McChrystal says of special forces operations generally the advantages of communication hugely outweigh the dangers of leaks. Whitehall now works on opposite principles: it keeps information secret that does not need to be secret in order to hide its own internal processes from scrutiny, thus adding to its own management failures and distrust (a vicious circle).
  • ‘Configuration management’. There must be a process whereby huge efforts go into the initial design of a complex system then there is a process whereby changes are made in a disciplined way such that a) interdependencies are tested where possible by relevant people before a change is agreed and b) then everybody relevant knows about the change. This ties together design, engineering, management, scheduling, cost, contracts, and allows the coordination of interdisciplinary teams. Test, learn, communicate results, change where needed, communicate… Whitehall now works on opposite principles: it does not put enough effort into the initial design then makes haphazard changes then fails to communicate changes effectively.
  • Physical and information structures should reinforce open communication. From Mueller’s NASA to JSOC, organisations that have coped well with complexity have built novel control centres to reinforce extreme communication. Spend money and time on new technologies and processes to help spread orientation and learning through the organisation. Whitehall now works on opposite principles: e.g. its antiquated committee structure and ‘red box’ system are ludicrously inefficient regarding management but are kept because they give officials huge control over ministers.
  • Long-term budgets. Long-term budgets save money. Whitehall now works on opposite principles: normal government budget processes do not value speed and savings from doing things fast. They are focused on what Parliament thinks this year. This makes it very hard to plan wisely and wastes money in the long-term (see below).
  • You need a complex mix of centralisation and decentralisation. While overall vision, goals, and strategy usually comes from the top, it is vital that extreme decentralisation dominates operationally so that decisions are fast and unbureaucratic. Information must be shared centrally and horizontally across the organisation — it is not either/or. Big complex projects must empower people throughout the network and cannot rely on issuing orders through a hierarchy. Whitehall now works on opposite principles: it is a centralising ratchet. E.g. Budgets and spending reviews are the exact opposite of Mueller’s approach. 1) They are short-term with almost no long-term elements. 2) They do not balance off priorities in any serious way. 3) They involve totally fake numbers — every department lies to the Treasury and provides fake numbers. Treasury officials dig into these. There are rounds of these games. Officials never stop lying. To maintain the charade the Chancellor never says to the SoS ‘stop your officials lying to us’ — candour would break the system. 4) The Treasury does not have the expertise to evaluate most of what they are looking at. The idea it is a department staffed by brilliant whiz kids is a joke. I saw DfE officials with very modest abilities routinely cheat the Treasury.
  • Extreme focus on errors. Schriever had ‘Black Saturdays’ and Mueller had similar meetings focused not on ‘reporting progress’ but making clear the problems. Simple as it sounds this is very unusual. Whitehall now works on opposite principles: routinely nobody is held responsible for errors and most management works on the basis of ‘give me good news not bad news’. Neither the culture nor incentives focus effort on eliminating errors. Most don’t care and you see those responsible for disaster ambling to the tube at 4pm or going on holiday amid meltdown.
  • Spending on redundancy to improve resilience. Whitehall now works on opposite principles: it tends to treat redundancy as ‘waste’ and its short-term budget processes reinforce decisions that mean out-of-control long-term budgets. By the time the long-term happens, the responsible people have all moved on to better paid jobs and nobody is accountable.
  • Important knowledge is discovered but then the innovation is standardised and codified so it can be easily learned and used by others. Whitehall now works on opposite principles: for example, in the Department for Education officials systematically destroyed its own library. The DfE operated with almost no institutional memory. By the time I left in 2014, after David Cameron banned me from entering any department officials would ask to meet me outside to find out why decisions had been taken in 2011 because three years later almost everybody had moved on to other things. The Foreign Office similarly destroyed its own library.
  • Systems management means lots of process and documentation but at its best it is fluid and purposeful — it is not process for ass-covering. The crucial ‘Gillette Procedures’ swept away red tape and Schriever battled the system to maintain freedom from normal government processes. When asked how he would do a similar programme to Apollo now (1990s) Mueller responded that the only way to do it would be as a classified ‘black’ project to escape the law on issues like procurement. Whitehall now works on opposite principles: its obsession is bullshit process for buck-passing and it fights with all its might against simplification and focus.
  • Saving time saves money. Schriever and Mueller focused on speed and saving time. Whitehall now works on opposite principles: its default mode is to go slower and those who advocate speed are denounced as reckless. Repeatedly in the DfE I was told it was ‘impossible’ to do things in the period I demanded — often less than half what senior officials wanted — yet we often achieved this and there was practically no example of failure that came because my time demands were inherently unreasonable. The system naturally pushes for the longest periods they can get away with to give themselves what they think of as a chance to beat ‘expectations’ but then they often fail on absurdly long timetables. In the DfE we often had a better record of hitting timetables that were ‘impossibly’ short than on those that were traditionally long. Also in many areas there is no downside to pushing fast — the worst that happens is minor and irrelevant embarrassment while the cumulative gains from trying to go fast are huge.
  • The ‘systems’ approach is inherently interdisciplinary ‘because its function is to integrate the specialized separate pieces of a complex of apparatus and people — the system — into a harmonious ensemble that optimally achieves the desired end’ (Ramo). Whitehall now works on opposite principles: it is hopeless at assembling interdisciplinary teams and elevates legal advice over everything in relation to practically any problem, causing huge delays and cost overruns.
  • The ‘matrix management’ system allowed coordination across different departments and different projects.  Whitehall now works on opposite principles. It is stuck with antiquated departments, an antiquated Cabinet Office system, and antiquated project management. Anything ‘cross-government’ is an immediate clue to the savvy that it is doomed and rarely worth wasting time on. A ‘matrix’ approach could and should be applied to break existing hierarchies and speed everything up.
  • People and ideas were more important than technology. Computers and other technologies can help but the main ideas came in the 1950s before personal computers. JSOC applied all sorts of technologies but Colonel Boyd’s dictum holds: people, ideas, technology — in that order. Whitehall now works on opposite principles: for example, the former Cabinet Secretary, Gus O’Donnell, recently blamed a ‘lack of investment’ in IT and a shortage of staff for a huge range of Whitehall blunders. This is really deluded. The central problem is known to all experts and is shown in almost every inquiry: IT projects fail repeatedly in the same ways because of failures of management, not ‘lack of investment’, and adding people to flawed projects is not a solution.

Ministers have little grip of departments and little power to change their direction. They can’t hire or fire and they can’t set incentives. They are almost never in a job long enough to acquire much useful knowledge and they almost never have the sort of management skills that provide alternative value to specific knowledge. They have little chance to change anything and officials ensure this little chance becomes almost no chance.

This story shows how to do things much better than normal. It shows that the principles underlying Mueller’s success are naturally in extreme competition with the principles of management that dominate all normal bureaucracies, public or private. People have been able to read about these principles for decades yet today in Whitehall almost everything runs on exactly the opposite principles: incentives operate to suppress learning. The institutional and policy changes inherent in leaving the EU are a systems problem requiring a systems response. Implementing Mueller’s principles would mean changes to most of the antiquated and failing foundations of Whitehall and bring big improvements and cost savings. Such changes are likely to be resisted by most MPs as well as Whitehall given few of them understand or have experience in high performance teams and would regard Mueller’s approach as a threat to their career prospects.

Because Whitehall is a system failure in which different failures are entangled, its inhabitants tend to potter around in an uncomprehending fog of confusion without understanding why things fail every day and therefore they do not support changes that could improve things even though these changes would be personally advantageous particularly for the first mover.

What is the minimum needed to break bureaucratic resistance and spark a virtuous circle?

How can people outside the system affect mission critical political institutions protected from market competition and resistant to major reforms?

How can we replace many traditional centralised bureaucracies with institutions that mimic successful biological systems such as the immune system that a) use distributed information processing to identify useful structure in the environment, b) find ‘good enough’ solutions in a vast search space of possibilities, and c) move at least ten times faster than existing systems?

[If you find this interesting and/or useful, then the PDF of the whole story is here. It involves some of the cleverest people of the 20th Century, such as John von Neumann.]

Unrecognised simplicities of effective action #2: ‘Systems’ thinking — ideas from the Apollo programme for a ‘systems politics’

This is the second in a series: click this link 201702-effective-action-2-systems-engineering-to-systems-politics. The first is HERE.

This paper concerns a very interesting story combining politics, management, institutions, science and technology. When high technology projects passed a threshold of complexity post-1945 amid the extreme pressure of the early Cold War, new management ideas emerged. These ideas were known as ‘systems engineering’ and ‘systems management’. These ideas were particularly connected to the classified program to build the first Intercontinental Ballistic Missiles (ICBMs) in the 1950s and successful ideas were transplanted into a failing NASA by George Mueller and others from 1963 leading to the successful moon landing in 1969.

These ideas were then applied in other mission critical teams and could be used to improve government performance. Urgently needed projects to lower the probability of catastrophes for humanity will benefit from considering why Mueller’s approach was 1) so successful and 2) so un-influential in politics. Could we develop a ‘systems politics’ that applies the unrecognised simplicities of effective action?

For those interested, it also looks briefly at an interesting element of the story – the role of John von Neumann, the brilliant mathematician who was deeply involved in the Manhattan Project, the project to build ICBMs, the first digital computers, and subjects like artificial intelligence, artificial life, possibilities for self-replicating machines made from unreliable components, and the basic problem that technological progress ‘gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we have known them, cannot continue.’

An obvious project with huge inherent advantages for humanity is the development of an international manned lunar base as part of developing space for commerce and science. It is the sort of thing that might change political dynamics on earth and could generate enormous support across international boundaries. After 23 June 2016, the UK has to reorient national policy on many dimensions. Developing basic science is one of the most important dimensions (for example, as I have long argued we urgently need a civilian version of DARPA similarly operating outside normal government bureaucratic systems including procurement and HR). Supporting such an international project would be a great focus for UK efforts and far more productive than our largely wasted decades of focus on the dysfunctional bureaucracy in Brussels that is dominated by institutions that fail the most important test – the capacity for error-correction the importance of which has been demonstrated over long periods and through many problems by the Anglo-American political system and its common law.

Please leave comments or email dmc2.cummings at gmail.com

 

Please help: how to make a big improvement in the alignment of political parties’ incentives with the public interest?

I am interested in these questions:

1) What incentives drive good/bad behaviour for UK political parties?

2) How could they be changed (legal and non-legal) to align interests of existing parties better with the public interest?

3) If one were setting up a new party from scratch what principles could be established in order to align the party’s interests with the public interest much more effectively than is now the case anywhere in the world, and how could one attract candidates very different to those who now dominate Parliament (cleverer, quantitative problem-solving skills, experience in managing complex organisations etc)?

4) Is there a good case for banning political parties (as sometimes was attempted in ancient Greece), how to do it, what would replace them, why would this be better etc (I assume this is a bad and/or impractical idea but it’s worth asking why)?

5) In what ways do existing or plausible technologies affect these old questions?

What are the best things written on these problems?

What are the best examples around the world of how people have made big improvements?

Assume that financial resources are effectively unlimited for the entity trying to make these changes, let me worry about things like ‘would the public buy it’ etc – focus on policy not communication/PR advice.

The more specific the better: an ideal bit of help would be detailed draft legislation. I don’t expect anybody to produce this, but just to show what I mean…

The overall problem is: how to make government performance dramatically, quantifiably, and sustainably better?

Please leave ideas in comments or email dmc2.cummings@gmail.com

Thanks

D

Unrecognised simplicities of effective action #1: expertise and a quadrillion dollar business

‘The combination of physics and politics could render the surface of the earth uninhabitable.’ John von Neumann.

Introduction

This series of blogs considers:

  • the difference between fields with genuine expertise, such as fighting and physics, and fields dominated by bogus expertise, such as politics and economic forecasting;
  • the big big problem we face – the world is ‘undersized and underorganised’ because of a collision between four forces: 1) our technological civilisation is inherently fragile and vulnerable to shocks, 2) the knowledge it generates is inherently dangerous, 3) our evolved instincts predispose us to aggression and misunderstanding, and 4) there is a profound mismatch between the scale and speed of destruction our knowledge can cause and the quality of individual and institutional decision-making in ‘mission critical’ institutions – our institutions are similar to those that failed so spectacularly in summer 1914 yet they face crises moving at least ~103 times faster and involving ~106 times more destructive power able to kill ~1010 people;
  • what classic texts and case studies suggest about the unrecognised simplicities of effective action to improve the selection, education, training, and management of vital decision-makers to improve dramatically, reliably, and quantifiably the quality of individual and institutional decisions (particularly 1) the ability to make accurate predictions and b) the quality of feedback);
  • how we can change incentives to aim a much bigger fraction of the most able people at the most important problems;
  • what tools and technologies can help decision-makers cope with complexity.

[I’ve tweaked a couple of things in response to this blog by physicist Steve Hsu.]

*

Summary of the big big problem

The investor Peter Thiel (founder of PayPal and Palantir, early investor in Facebook) asks people in job interviews: what billion (109) dollar business is nobody building? The most successful investor in world history, Warren Buffett, illustrated what a quadrillion (1015) dollar business might look like in his 50th anniversary letter to Berkshire Hathaway investors.

‘There is, however, one clear, present and enduring danger to Berkshire against which Charlie and I are powerless. That threat to Berkshire is also the major threat our citizenry faces: a “successful” … cyber, biological, nuclear or chemical attack on the United States… The probability of such mass destruction in any given year is likely very small… Nevertheless, what’s a small probability in a short period approaches certainty in the longer run. (If there is only one chance in thirty of an event occurring in a given year, the likelihood of it occurring at least once in a century is 96.6%.) The added bad news is that there will forever be people and organizations and perhaps even nations that would like to inflict maximum damage on our country. Their means of doing so have increased exponentially during my lifetime. “Innovation” has its dark side.

‘There is no way for American corporations or their investors to shed this risk. If an event occurs in the U.S. that leads to mass devastation, the value of all equity investments will almost certainly be decimated.

‘No one knows what “the day after” will look like. I think, however, that Einstein’s 1949 appraisal remains apt: “I know not with what weapons World War III will be fought, but World War IV will be fought with sticks and stones.”’

Politics is profoundly nonlinear. (I have written a series of blogs about complexity and prediction HERE which are useful background for those interested.) Changing the course of European history via the referendum only involved about 10 crucial people controlling ~£107  while its effects over ten years could be on the scale of ~108 – 10people and ~£1012: like many episodes in history the resources put into it are extremely nonlinear in relation to the potential branching histories it creates. Errors dealing with Germany in 1914 and 1939 were costly on the scale of ~100,000,000 (108) lives. If we carry on with normal human history – that is, international relations defined as out-groups competing violently – and combine this with modern technology then it is extremely likely that we will have a disaster on the scale of billions (109) or even all humans (~1010). The ultimate disaster would kill about 100 times more people than our failure with Germany. Our destructive power is already much more than 100 times greater than it was then: nuclear weapons increased destructiveness by roughly a factor of a million.

Even if we dodge this particular bullet there are many others lurking. New genetic engineering techniques such as CRISPR allow radical possibilities for re-engineering organisms including humans in ways thought of as science fiction only a decade ago. We will soon be able to remake human nature itself. CRISPR-enabled ‘gene drives’ enable us to make changes to the germ-line of organisms permanent such that changes spread through the entire wild population, including making species extinct on demand. Unlike nuclear weapons such technologies are not complex, expensive, and able to be kept secret for a long time. The world’s leading experts predict that people will be making them cheaply at home soon – perhaps they already are. These developments have been driven by exponential progress much faster than Moore’s Law reducing the cost of DNA sequencing per genome from ~$108 to ~$10in roughly 15 years.

screenshot-2017-01-16-12-24-13

It is already practically possible to deploy a cheap, autonomous, and anonymous drone with facial-recognition software and a one gram shaped-charge to identify a relevant face and blow it up. Military logic is driving autonomy. For example, 1) the explosion in the volume of drone surveillance video (from 71 hours in 2004 to 300,000 hours in 2011 to millions of hours now) requires automated analysis, and 2) jamming and spoofing of drones strongly incentivise a push for autonomy. It is unlikely that promises to ‘keep humans in the loop’ will be kept. It is likely that state and non-state actors will deploy low-cost drone swarms using machine learning to automate the ‘find-fix-finish’ cycle now controlled by humans. (See HERE for a video just released for one such program and imagine the capability when they carry their own communication and logistics network with them.)

In the medium-term, many billions are being spent on finding the secrets of general intelligence. We know this secret is encoded somewhere in the roughly 125 million ‘bits’ of information that is the rough difference between the genome that produces the human brain and the genome that produces the chimp brain. This search space is remarkably small – the equivalent of just 25 million English words or 30 copies of the King James Bible. There is no fundamental barrier to decoding this information and it is possible that the ultimate secret could be described relatively simply (cf. this great essay by physicist Michael Nielsen). One of the world’s leading experts has told me they think a large proportion of this problem could be solved in about a decade with a few tens of billions and something like an Apollo programme level of determination.

Not only is our destructive and disruptive power still getting bigger quickly – it is also getting cheaper and faster every year. The change in speed adds another dimension to the problem. In the period between the Archduke’s murder and the outbreak of World War I a month later it is striking how general failures of individuals and institutions were compounded by the way in which events moved much faster than the ‘mission critical’ institutions could cope with such that soon everyone was behind the pace, telegrams were read in the wrong order and so on. The crisis leading to World War I was about 30 days from the assassination to the start of general war – about 700 hours. The timescale for deciding what to do between receiving a warning of nuclear missile launch and deciding to launch yourself is less than half an hour and the President’s decision time is less than this, maybe just minutes. This is a speedup factor of at least 103.

Economic crises already occur far faster than human brains can cope with. The financial system has made a transition from people shouting at each other to a a system dominated by high frequency ‘algorithmic trading’ (HFT), i.e. machine intelligence applied to robot trading with vast volumes traded on a global spatial scale and a microsecond (10-6) temporal scale far beyond the monitoring, understanding, or control of regulators and politicians. There is even competition for computer trading bases in specific locations based on calculations of Special Relativity as the speed of light becomes a factor in minimising trade delays (cf. Relativistic statistical arbitrage, Wissner-Gross). ‘The Flash Crash’ of 9 May 2010 saw the Dow lose hundreds of points in minutes. Mini ‘flash crashes’ now blow up and die out faster than humans can notice. Given our institutions cannot cope with economic decisions made at ‘human speed’, a fortiori they cannot cope with decisions made at ‘robot speed’. There is scope for worse disasters than 2008 which would further damage the moral credibility of decentralised markets and provide huge chances for extremist political entrepreneurs to exploit. (* See endnote.)

What about the individuals and institutions that are supposed to cope with all this?

Our brains have not evolved much in thousands of years and are subject to all sorts of constraints including evolved heuristics that lead to misunderstanding, delusion, and violence particularly under pressure. There is a terrible mismatch between the sort of people that routinely dominate mission critical political institutions and the sort of people we need: high-ish IQ (we need more people >145 (+3SD) while almost everybody important is between 115-130 (+1 or 2SD)), a robust toolkit for not fooling yourself including quantitative problem-solving (almost totally absent at the apex of relevant institutions), determination, management skills, relevant experience, and ethics. While our ancestor chiefs at least had some intuitive feel for important variables like agriculture and cavalry our contemporary chiefs (and those in the media responsible for scrutiny of decisions) generally do not understand their equivalents, and are often less experienced in managing complex organisations than their predecessors.

The national institutions we have to deal with such crises are pretty similar to those that failed so spectacularly in summer 1914 yet they face crises moving at least ~103 times faster and involving ~106 times more destructive power able to kill ~1010 people. The international institutions developed post-1945 (UN, EU etc) contribute little to solving the biggest problems and in many ways make them worse. These institutions fail constantly and do not  – cannot – learn much.

If we keep having crises like we have experienced over the past century then this combination of problems pushes the probability of catastrophe towards ‘overwhelmingly likely’.

*

What Is To be Done? There’s plenty of room at the top

‘In a knowledge-rich world, progress does not lie in the direction of reading information faster, writing it faster, and storing more of it. Progress lies in the direction of extracting and exploiting the patterns of the world… And that progress will depend on … our ability to devise better and more powerful thinking programs for man and machine.’ Herbert Simon, Designing Organizations for an Information-rich World, 1969.

‘Fascinating that the same problems recur time after time, in almost every program, and that the management of the program, whether it happened to be government or industry, continues to avoid reality.’ George Mueller, pioneer of ‘systems engineering’ and ‘systems management’ and the man most responsible for the success of the 1969 moon landing.

Somehow the world has to make a series of extremely traumatic and dangerous transitions over the next 20 years. The main transition needed is:

Embed reliably the unrecognised simplicities of high performance teams (HPTs), including personnel selection and training, in ‘mission critical’ institutions while simultaneously developing a focused project that radically improves the prospects for international cooperation and new forms of political organisation beyond competing nation states.

Big progress on this problem would automatically and for free bring big progress on other big problems. It could improve (even save) billions of lives and save a quadrillion dollars (~$1015). If we avoid disasters then the error-correcting institutions of markets and science will, patchily, spread peace, prosperity, and learning. We will make big improvements with public services and other aspects of ‘normal’ government. We will have a healthier political culture in which representative institutions, markets serving the public (not looters), and international cooperation are stronger.

Can a big jump in performance – ‘better and more powerful thinking programs for man and machine’ – somehow be systematised?

Feynman once gave a talk titled ‘There’s plenty of room at the bottom’ about the huge performance improvements possible if we could learn to do engineering at the atomic scale – what is now called nanotechnology. There is also ‘plenty of room at the top’ of political structures for huge improvements in performance. As I explained recently, the victory of the Leave campaign owed more to the fundamental dysfunction of the British Establishment than it did to any brilliance from Vote Leave. Despite having the support of practically every force with power and money in the world (including the main broadcasters) and controlling the timing and legal regulation of the referendum, they blew it. This was good if you support Leave but just how easily the whole system could be taken down should be frightening for everybody .

Creating high performance teams is obviously hard but in what ways is it really hard? It is not hard in the same sense that some things are hard like discovering profound new mathematical knowledge. HPTs do not require profound new knowledge. We have been able to read the basic lessons in classics for over two thousand years. We can see relevant examples all around us of individuals and teams showing huge gains in effectiveness.

The real obstacle is not financial. The financial resources needed are remarkably low and the return on small investments could be incalculably vast. We could significantly improve the decisions of the most powerful 100 people in the UK or the world for less than a million dollars (~£106) and a decade-long project on a scale of just ~£107 could have dramatic effects.

The real obstacle is not a huge task of public persuasion – quite the opposite. A government that tried in a disciplined way to do this would attract huge public support. (I’ve polled some ideas and am confident about this.) Political parties are locked in a game that in trying to win in conventional ways leads to the public despising them. Ironically if a party (established or new) forgets this game and makes the public the target of extreme intelligent focus then it would not only make the world better but would trounce their opponents.

The real obstacle is not a need for breakthrough technologies though technology could help. As Colonel Boyd used to shout, ‘People, ideas, machines – in that order!’

The real obstacle is that although we can all learn and study HPTs it is extremely hard to put this learning to practical use and sustain it against all the forces of entropy that constantly operate to degrade high performance once the original people have gone. HPTs are episodic. They seem to come out of nowhere, shock people, then vanish with the rare individuals. People write about them and many talk about learning from them but in fact almost nobody ever learns from them – apart, perhaps, from those very rare people who did not need to learn – and nobody has found a method to embed this learning reliably and systematically in institutions that can maintain it. The Prussian General Staff remained operationally brilliant but in other ways went badly wrong after the death of the elder Moltke. When George Mueller left NASA it reverted to what it had been before he arrived – management chaos. All the best companies quickly go downhill after the departure of people like Bill Gates – even when such very able people have tried very very hard to avoid exactly this problem.

Charlie Munger, half of the most successful investment team in world history, has a great phrase he uses to explain their success that gets to the heart of this problem:

‘There isn’t one novel thought in all of how Berkshire [Hathaway] is run. It’s all about … exploiting unrecognized simplicities… It’s a community of like-minded people, and that makes most decisions into no-brainers. Warren [Buffett] and I aren’t prodigies. We can’t play chess blindfolded or be concert pianists. But the results are prodigious, because we have a temperamental advantage that more than compensates for a lack of IQ points.’

The simplicities that bring high performance in general, not just in investing, are largely unrecognised because they conflict with many evolved instincts and are therefore psychologically very hard to implement. The principles of the Buffett-Munger success are clear – they have even gone to great pains to explain them and what the rest of us should do – and the results are clear yet still almost nobody really listens to them and above average intelligence people instead constantly put their money into active fund management that is proved to destroy wealth every year!

Most people think they are already implementing these lessons and usually strongly reject the idea that they are not. This means that just explaining things is very unlikely to work:

‘I’d say the history that Charlie [Munger] and I have had of persuading decent, intelligent people who we thought were doing unintelligent things to change their course of action has been poor.’ Buffett.

Even more worrying, it is extremely hard to take over organisations that are not run right and make them excellent.

‘We really don’t believe in buying into organisations to change them.’ Buffett.

If people won’t listen to the world’s most successful investor in history on his own subject, and even he finds it too hard to take over failing businesses and turn them around, how likely is it that politicians and officials incentivised to keep things as they are will listen to ideas about how to do things better? How likely is it that a team can take over broken government institutions and make them dramatically better in a way that outlasts the people who do it? Bureaucracies are extraordinarily resistant to learning. Even after the debacles of 9/11 and the Iraq War, costing many lives and trillions of dollars, and even after the 2008 Crash, the security and financial bureaucracies in America and Europe are essentially the same and operate on the same principles.

Buffett’s success is partly due to his discipline in sticking within what he and Munger call their ‘circle of competence’. Within this circle they have proved the wisdom of avoiding trying to persuade people to change their minds and avoiding trying to fix broken institutions.

This option is not available in politics. The Enlightenment and the scientific revolution give us no choice but to try to persuade people and try to fix or replace broken institutions. In general ‘it is better to undertake revolution than undergo it’. How might we go about it? What can people who do not have any significant power inside the system do? What international projects are most likely to spark the sort of big changes in attitude we urgently need?

This is the first of a series. I will keep it separate from the series on the EU referendum though it is connected in the sense that I spent a year on the referendum in the belief that winning it was a necessary though not sufficient condition for Britain to play a part in improving the quality of government dramatically and improving the probability of avoiding the disasters that will happen if politics follows a normal path. I intended to implement some of these ideas in Downing Street if the Boris-Gove team had not blown up. The more I study this issue the more confident I am that dramatic improvements are possible and the more pessimistic I am that they will happen soon enough.

Please leave comments and corrections…

* A new transatlantic cable recently opened for financial trading. Its cost? £300 million. Its advantage? It shaves 2.6 milliseconds off the latency of financial trades. Innovative groups are discussing the application of military laser technology, unmanned drones circling the earth acting as routers, and even the use of neutrino communication (because neutrinos can go straight through the earth just as zillions pass through your body every second without colliding with its atoms) – cf. this recent survey in Nature.