Summary: a short blog on a new paper casting doubt on claims re microtargeting using Facebook.

The audience for conspiracy theories about microtargeting, Facebook and Brexit is large and includes a big subset of SW1 and a wider group (but much smaller than it thinks it is) that wants a rematch against the public. The audience for facts, evidence and research about microtargeting, Facebook and Brexit is tiny. If you are part of this tiny audience…

I wrote a few days ago about good evidence on microtargeting in general and Cambridge Analytica’s claims on ‘psychographics’ in particular (see HERE).

Nutshell: the evidence and science re ‘microtargeting’ does not match the story you read in the media or the conspiracy theories about the referendum, and Vote Leave did not do microtargeting in any normal sense of the term.

Another interesting paper on this subject has been published a few days ago.

Background…

One of the most influential researchers cited by the media since Brexit/Trump is Michal Kosinski who wrote a widely cited 2015 paper on predicting Big 5 personality traits from Facebook ‘likes’: Computer-based personality judgments are more accurate than those made by humans.

Duncan Watts, one of the leading scholars in computational sociology, pointed out:

‘All it shows is that algorithmic predictions of Big 5 traits are about as accurate as human predictions, which is to say only about 50 percent accurate. If all you had to do to change someone’s opinion was guess their openness or political attitude, then even really noisy predictions might be worrying at scale. But predicting attributes is much easier than persuading people.’

Kosinski published another paper recently: Psychological targeting as an effective approach to digital mass persuasion (November 2017). The core claim was:

‘In three field experiments that reached over 3.5 million individuals with psychologically tailored advertising, we find that matching the content of persuasive appeals to individuals’ psychological characteristics significantly altered their behavior as measured by clicks and purchases. Persuasive appeals that were matched to people’s extraversion or openness-to-experience level resulted in up to 40% more clicks and up to 50% more purchases than their mismatching or unpersonalized counterparts. Our findings suggest that the application of psychological targeting makes it possible to influence the behavior of large groups of people by tailoring persuasive appeals to the psychological needs of the target audiences.’

If this claim were true it would be a big deal in the advertising world. Further, Kosinski claimed that ‘The assumption is that the same effects can be observed in political messages.’ That would be an even bigger deal.

I was sceptical when I read the 2017 paper, mainly given the large amount of evidence in books like Hacking the Electorate that I touched on in the previous blog, but I didn’t have the time or expertise to investigate. I did read this Wired piece on that paper in which Watts commented:

‘Watts says that the 2017 paper didn’t convince him the technique could work, either. The results barely improve click-through rates, he says — a far cry from predicting political behavior. And more than that, Kosinski’s mistargeted openness ads — that is, the ads tailored for the opposite personality characteristic — far outperformed the targeted extraversion ads. Watts says that suggests other, uncontrolled factors are having unknown effects. “So again,” he says, “I would question how meaningful these effects are in practice.”‘

Another leading researcher, David Lazer, commented:

‘On the psychographic stuff, I haven’t see any science that really aligns with their [CA/Kosinski] claims.’

Another leading researcher, Alex Pentland at MIT (who also successfully won a DARPA project to solve a geolocation intelligence problem) was also sceptical:

‘Everybody talks about Google and Facebook, but the things that people say online are not nearly as predictive as, say, what your telephone company knows about you. Or your credit card company. Fortunately telephone companies, banks, things like that are very highly regulated companies. So we have a fair amount of time. It may never happen that the data gets loose.’

I’ve just been sent this paper (preprint link): Field studies of psychologically targeted ads face threats to internal validity (2018). It is an analysis of Kosinski’s 2017 experiments. It argues that the Kosinski experiment is NOT RANDOMISED and points out statistical and other flaws that undermine Kosinski’s claims:

‘The paper [Kosinski 2017] uses Facebook’s standard ad platform to compare how different versions of ads perform. However, this process does not create a randomized experiment: users are not randomly assigned to different ads, and individuals may even receive multiple ad types (e.g., both extroverted and introverted ads). Furthermore, ad platforms like Facebook optimize campaign performance by showing ads to users whom the platform expects are more likely to fulfill the campaign’s objective… This optimization generates differences in the set of users exposed to each ad type, so that differences in responses across ads do not by themselves indicate a causal effect.’ (Emphasis added.)

Kosinski et al reply here. They admit that the optimisation of Facebook’s ad algorithms could affect their results though they defend their work. (Campaigns face similar operational problems in figuring out ways to run experiments on Facebook without FB’s algorithms distorting them.)

I am not remotely competent to judge the conflicting claims and haven’t yet asked anybody who is though I have a (mostly worthless) hunch that the criticisms will stack up. I’ll add an update in the future when this is resolved.

Big claims require good evidence and good science — not what Feynman called ‘cargo cult science’ which accounts for a lot of social science research. Most claims you read about psychological manipulation are rubbish. There are interesting possibilities for applying advanced technology, as I wrote in my last blog, but a) almost everything you read about is not in this class and b) I am sceptical in general that ideas in published work on using Big 5 personality traits could add anything more than a very small boost to political campaigns at best and it can also easily blow up in your face, as Hersh’s evidence to the Senate shows. I strongly suspect that usually the ‘gains’ are less than the fees of the consultants flogging the snake oil — i.e a net loss for campaigns.

If you believe, like the Observer, that the US/UK military and/or intelligence services have access to technological methods of psychological manipulation that greatly exceed what is done commercially, you misunderstand their real capabilities. For example, look at how the commander of US classified special forces (JSOC), Stanley McChrystal, recruited civilians for his propaganda operations in Afghanistan because the military did not know what to do. The evidence since 9/11 is of general failure in the UK/USA viz propaganda / ‘information war’ / ‘hybrid war’ etc. Further, if you want expertise on things like Facebook and Google, the place to look is Silicon Valley, not the Pentagon. Look at how recent UK Prime Ministers have behaved. Look at how Cameron tweeted about rushing back from Chequers in the middle of the night to deal with ISIS beheadings. Look at how Blair, Brown and Cameron foolishly read out the names of people killed in the Commons. Of course it is impossible from the outside to know how much of this is because Downing Street mangles advice and operations and how much is failure elsewhere. I assume there are lots of good people in the system but, like elsewhere in modern Whitehall, expertise is suppressed by centralised hierarchies (as with Brexit).

On campaigns and in government, figuring out the answers to a few deep questions is much more important than practically anything you read about technology issues like microtargeting. But focus and priorities are very hard for big organisations including parties and governments, because they are mostly dominated by seniority, groupthink, signalling, distorted incentives and so on. A lack of focus means they spread intelligent effort too widely and don’t think enough about deep questions that overwhelmingly determine their fate.

Of course, it is possible to use technology to enhance campaigns and it is possible to devise messages that have game-changing effects but the media focus on microtargeting is almost completely misguided and the Select Committee’s inquiry into fake news has mostly spread fake news. There has been zero scrutiny, as far as I have seen, on the evidence from reputable scholars like Duncan Watts or Eitan Hersh on the facts and evidence about microtargeting and fake news in relation to Trump/Brexit. Sadly they are more interested in grandstanding than truth-seeking, which is why the Committee turned down my offer to arrange a time to give evidence and instead tried to grab headlines. I offered friendly cooperation, as the government should have done with Brexit, but the Committee went for empty threats, as per May and Hammond, and this approach will be as successful as this government’s negotiating strategy.