Over the past century, more than a few great writers have expressed concern about humanity’s future. In The Iron Heel
(1908), the American writer Jack London pictured a world in which a
handful of wealthy corporate titans – the ‘oligarchs’ – kept the masses
at bay with a brutal combination of rewards and punishments. Much of
humanity lived in virtual slavery, while the fortunate ones were bought
off with decent wages that allowed them to live comfortably – but
without any real control over their lives.
In We (1924),
the brilliant Russian writer Yevgeny Zamyatin, anticipating the
excesses of the emerging Soviet Union, envisioned a world in which
people were kept in check through pervasive monitoring. The walls of
their homes were made of clear glass, so everything they did could be
observed. They were allowed to lower their shades an hour a day to have
sex, but both the rendezvous time and the lover had to be registered
first with the state.
In Brave New World
(1932), the British author Aldous Huxley pictured a near-perfect
society in which unhappiness and aggression had been engineered out of
humanity through a combination of genetic engineering and psychological
conditioning. And in the much darker novel 1984 (1949),
Huxley’s compatriot George Orwell described a society in which thought
itself was controlled; in Orwell’s world, children were taught to use a
simplified form of English called Newspeak in order to assure that they
could never express ideas that were dangerous to society.
These
are all fictional tales, to be sure, and in each the leaders who held
the power used conspicuous forms of control that at least a few people
actively resisted and occasionally overcame. But in the non-fiction
bestseller The Hidden Persuaders (1957) – recently released in a
50th-anniversary edition – the American journalist Vance Packard
described a ‘strange and rather exotic’ type of influence that was
rapidly emerging in the United States and that was, in a way, more
threatening than the fictional types of control pictured in the novels.
According to Packard, US corporate executives and politicians were
beginning to use subtle and, in many cases, completely undetectable methods to change people’s thinking, emotions and behaviour based on insights from psychiatry and the social sciences.
Most of us have heard of at least one of these methods: subliminal stimulation, or
what Packard called ‘subthreshold effects’ – the presentation of short
messages that tell us what to do but that are flashed so briefly we
aren’t aware we have seen them. In 1958, propelled by public concern
about a theatre in New Jersey that had supposedly hidden messages in a
movie to increase ice cream sales, the National Association of
Broadcasters – the association that set standards for US television –
amended its code to prohibit the use of subliminal messages in
broadcasting. In 1974, the Federal Communications Commission opined that
the use of such messages was ‘contrary to the public interest’.
Legislation to prohibit subliminal messaging was also introduced in the
US Congress but never enacted. Both the UK and Australia have strict
laws prohibiting it.
Subliminal
stimulation is probably still in wide use in the US – it’s hard to
detect, after all, and no one is keeping track of it – but it’s probably
not worth worrying about. Research suggests that it has only a small
impact, and that it mainly influences people who are already motivated
to follow its dictates; subliminal directives to drink affect people
only if they’re already thirsty.
Packard
had uncovered a much bigger problem, however – namely that powerful
corporations were constantly looking for, and in many cases already
applying, a wide variety of techniques for controlling people without
their knowledge. He described a kind of cabal in which marketers worked
closely with social scientists to determine, among other things, how to
get people to buy things they didn’t need and how to condition young
children to be good consumers – inclinations that were explicitly
nurtured and trained in Huxley’s Brave New World. Guided by
social science, marketers were quickly learning how to play upon
people’s insecurities, frailties, unconscious fears, aggressive feelings
and sexual desires to alter their thinking, emotions and behaviour
without any awareness that they were being manipulated.
By
the early 1950s, Packard said, politicians had got the message and were
beginning to merchandise themselves using the same subtle forces being
used to sell soap. Packard prefaced his chapter on politics with an
unsettling quote from the British economist Kenneth Boulding: ‘A world
of unseen dictatorship is conceivable, still using the forms of
democratic government.’ Could this really happen, and, if so, how would
it work?
The
forces that Packard described have become more pervasive over the
decades. The soothing music we all hear overhead in supermarkets causes
us to walk more slowly and buy more food, whether we need it or not.
Most of the vacuous thoughts and intense feelings our teenagers
experience from morning till night are carefully orchestrated by highly
skilled marketing professionals working in our fashion and entertainment
industries. Politicians work with a wide range of consultants who test
every aspect of what the politicians do in order to sway voters:
clothing, intonations, facial expressions, makeup, hairstyles and
speeches are all optimised, just like the packaging of a breakfast
cereal.
Fortunately,
all of these sources of influence operate competitively. Some of the
persuaders want us to buy or believe one thing, others to buy or believe
something else. It is the competitive nature of our society that keeps
us, on balance, relatively free.
But
what would happen if new sources of control began to emerge that had
little or no competition? And what if new means of control were
developed that were far more powerful – and far more invisible –
than any that have existed in the past? And what if new types of
control allowed a handful of people to exert enormous influence not just
over the citizens of the US but over most of the people on Earth?
It might surprise you to hear this, but these things have already happened.
Google decides which web pages to include in search results, and how to rank them. How it does so is one of the best-kept secrets in the world, like the formula for Coca-Cola
To
understand how the new forms of mind control work, we need to start by
looking at the search engine – one in particular: the biggest and best
of them all, namely Google. The Google search engine is so good and so
popular that the company’s name is now a commonly used verb in languages
around the world. To ‘Google’ something is to look it up on the Google
search engine, and that, in fact, is how most computer users worldwide
get most of their information about just about everything these days.
They Google it. Google has become the main gateway to virtually
all knowledge, mainly because the search engine is so good at giving us
exactly the information we are looking for, almost instantly and almost
always in the first position of the list it shows us after we launch
our search – the list of ‘search results’.
That
ordered list is so good, in fact, that about 50 per cent of our clicks
go to the top two items, and more than 90 per cent of our clicks go to
the 10 items listed on the first page of results; few people look at
other results pages, even though they often number in the thousands,
which means they probably contain lots of good information. Google
decides which of the billions of web pages it is going to include in our
search results, and it also decides how to rank them. How it decides
these things is a deep, dark secret – one of the best-kept secrets in
the world, like the formula for Coca-Cola.
Because
people are far more likely to read and click on higher-ranked items,
companies now spend billions of dollars every year trying to trick
Google’s search algorithm – the computer program that does the selecting
and ranking – into boosting them another notch or two. Moving up a
notch can mean the difference between success and failure for a
business, and moving into the top slots can be the key to fat profits.
Late
in 2012, I began to wonder whether highly ranked search results could
be impacting more than consumer choices. Perhaps, I speculated, a top
search result could have a small impact on people’s opinions about
things. Early in 2013, with my associate Ronald E Robertson of the
American Institute for Behavioral Research and Technology
in Vista, California, I put this idea to a test by conducting an
experiment in which 102 people from the San Diego area were randomly
assigned to one of three groups. In one group, people saw search results
that favoured one political candidate – that is, results that linked to
web pages that made this candidate look better than his or her
opponent. In a second group, people saw search rankings that favoured
the opposing candidate, and in the third group – the control group –
people saw a mix of rankings that favoured neither candidate. The same
search results and web pages were used in each group; the only thing
that differed for the three groups was the ordering of the search
results.
To make
our experiment realistic, we used real search results that linked to
real web pages. We also used a real election – the 2010 election for the
prime minister of Australia. We used a foreign election to make sure
that our participants were ‘undecided’. Their lack of familiarity with
the candidates assured this. Through advertisements, we also recruited
an ethnically diverse group of registered voters over a wide age range
in order to match key demographic characteristics of the US voting
population.
All
participants were first given brief descriptions of the candidates and
then asked to rate them in various ways, as well as to indicate which
candidate they would vote for; as you might expect, participants
initially favoured neither candidate on any of the five measures we
used, and the vote was evenly split in all three groups. Then the
participants were given up to 15 minutes in which to conduct an online
search using ‘Kadoodle’, our mock search engine, which gave them access
to five pages of search results that linked to web pages. People could
move freely between search results and web pages, just as we do when
using Google. When participants completed their search, we asked them to
rate the candidates again, and we also asked them again who they would
vote for.
We
predicted that the opinions and voting preferences of 2 or 3 per cent of
the people in the two bias groups – the groups in which people were
seeing rankings favouring one candidate – would shift toward that
candidate. What we actually found was astonishing. The proportion of
people favouring the search engine’s top-ranked candidate increased by 48.4 per cent,
and all five of our measures shifted toward that candidate. What’s
more, 75 per cent of the people in the bias groups seemed to have been
completely unaware that they were viewing biased search rankings. In the
control group, opinions did not shift significantly.
This
seemed to be a major discovery. The shift we had produced, which we
called the Search Engine Manipulation Effect (or SEME, pronounced
‘seem’), appeared to be one of the largest behavioural effects ever
discovered. We did not immediately uncork the Champagne bottle, however.
For one thing, we had tested only a small number of people, and they
were all from the San Diego area.
Over
the next year or so, we replicated our findings three more times, and
the third time was with a sample of more than 2,000 people from all 50
US states. In that experiment, the shift in voting preferences was 37.1
per cent and even higher in some demographic groups – as high as 80 per
cent, in fact.
We
also learned in this series of experiments that by reducing the bias
just slightly on the first page of search results – specifically, by
including one search item that favoured the other candidate in the third or fourth position of the results – we could mask our manipulation so that few or even no
people were aware that they were seeing biased rankings. We could still
produce dramatic shifts in voting preferences, but we could do so invisibly.
Still
no Champagne, though. Our results were strong and consistent, but our
experiments all involved a foreign election – that 2010 election in
Australia. Could voting preferences be shifted with real voters in the
middle of a real campaign? We were skeptical. In real elections, people
are bombarded with multiple sources of information, and they also know a
lot about the candidates. It seemed unlikely that a single experience
on a search engine would have much impact on their voting preferences.
To
find out, in early 2014, we went to India just before voting began in
the largest democratic election in the world – the Lok Sabha election
for prime minister. The three main candidates were Rahul Gandhi, Arvind
Kejriwal, and Narendra Modi. Making use of online subject pools and both
online and print advertisements, we recruited 2,150 people from 27 of
India’s 35 states and territories to participate in our experiment. To
take part, they had to be registered voters who had not yet voted and
who were still undecided about how they would vote.
unlike subliminal stimuli, SEME has an enormous impact – like Casper the ghost pushing you down a flight of stairs
Participants
were randomly assigned to three search-engine groups, favouring,
respectively, Gandhi, Kejriwal or Modi. As one might expect, familiarity
levels with the candidates was high – between 7.7 and 8.5 on a scale of
10. We predicted that our manipulation would produce a very small
effect, if any, but that’s not what we found. On average, we were able
to shift the proportion of people favouring any given candidate by more
than 20 per cent overall and more than 60 per cent in some demographic
groups. Even more disturbing, 99.5 per cent of our participants showed
no awareness that they were viewing biased search rankings – in other
words, that they were being manipulated.
SEME’s
near-invisibility is curious indeed. It means that when people –
including you and me – are looking at biased search rankings, they look just fine. So if right now you Google ‘US presidential candidates’, the search results you see will probably look fairly random, even if they happen to favour one candidate. Even I have trouble detecting bias in search rankings that I know
to be biased (because they were prepared by my staff). Yet our
randomised, controlled experiments tell us over and over again that when
higher-ranked items connect with web pages that favour one candidate,
this has a dramatic impact on the opinions of undecided voters, in large
part for the simple reason that people tend to click only on
higher-ranked items. This is truly scary: like subliminal stimuli, SEME
is a force you can’t see; but unlike subliminal stimuli, it has an
enormous impact – like Casper the ghost pushing you down a flight of
stairs.
We published a detailed
report about our first five experiments on SEME in the prestigious
Proceedings of the National Academy of Sciences
(PNAS) in August 2015. We had indeed found something important,
especially given Google’s dominance over search. Google has a
near-monopoly on internet searches in the US, with 83 per cent of
Americans specifying Google as the search engine they use most often,
according to the
Pew Research Center. So if Google favours one candidate in an election, its impact on undecided voters could easily decide the election’s outcome.
Keep
in mind that we had had only one shot at our participants. What would
be the impact of favouring one candidate in searches people are
conducting over a period of weeks or months before an election? It would
almost certainly be much larger than what we were seeing in our
experiments.
Other
types of influence during an election campaign are balanced by
competing sources of influence – a wide variety of newspapers, radio
shows and television networks, for example – but Google, for all intents
and purposes, has no competition, and people trust its search results
implicitly, assuming that the company’s mysterious search algorithm is
entirely objective and unbiased. This high level of trust, combined with
the lack of competition, puts Google in a unique position to impact
elections. Even more disturbing, the search-ranking business is entirely
unregulated, so Google could favour any candidate it likes without
violating any laws.
Some courts have even ruled that Google’s right to rank-order search results as it pleases is protected as a form of free speech.
Does
the company ever favour particular candidates? In the 2012 US
presidential election, Google and its top executives donated more than
$800,000 to President Barack Obama and just $37,000 to his opponent,
Mitt Romney. And in 2015, a team of researchers from the University of
Maryland and elsewhere
showed that Google’s search results routinely favoured Democratic candidates. Are Google’s search rankings really biased? An
internal report
issued by the US Federal Trade Commission in 2012 concluded that
Google’s search rankings routinely put Google’s financial interests
ahead of those of their competitors, and anti-trust actions currently
under way against Google in both the
European Union and
India are based on similar findings.
In
most countries, 90 per cent of online search is conducted on Google,
which gives the company even more power to flip elections than it has in
the US and, with internet penetration increasing rapidly worldwide,
this power is growing. In our PNAS article, Robertson and I calculated that Google now has the power to flip upwards of 25 per cent of the national elections in the world
with no one knowing this is occurring. In fact, we estimate that, with
or without deliberate planning on the part of company executives,
Google’s search rankings have been impacting elections for years, with
growing impact each year. And because search rankings are ephemeral,
they leave no paper trail, which gives the company complete deniability.
Power
on this scale and with this level of invisibility is unprecedented in
human history. But it turns out that our discovery about SEME was just
the tip of a very large iceberg.
Recent
reports
suggest that the Democratic presidential candidate Hillary Clinton is
making heavy use of social media to try to generate support – Twitter,
Instagram, Pinterest, Snapchat and Facebook, for starters. At this
writing, she has 5.4 million followers on Twitter, and her staff is
tweeting several times an hour during waking hours. The Republican
frontrunner, Donald Trump, has 5.9 million Twitter followers and is
tweeting just as frequently.
Is
social media as big a threat to democracy as search rankings appear to
be? Not necessarily. When new technologies are used competitively, they
present no threat. Even through the platforms are new, they are
generally being used the same way as billboards and television
commercials have been used for decades: you put a billboard on one side
of the street; I put one on the other. I might have the money to erect
more billboards than you, but the process is still competitive.
What happens, though, if such technologies are misused by the companies that own them? A
study by Robert M Bond, now a political science professor at Ohio State University, and others published in
Nature
in 2012 described an ethically questionable experiment in which, on
election day in 2010, Facebook sent ‘go out and vote’ reminders to more
than 60 million of its users. The reminders caused about 340,000 people
to vote who otherwise would not have. Writing in the
New Republic
in 2014, Jonathan Zittrain, professor of international law at Harvard
University, pointed out that, given the massive amount of information it
has collected about its users, Facebook could easily send such messages
only to people who support one particular party or candidate, and that
doing so could easily flip a close election –
with no one knowing that this has occurred.
And because advertisements, like search rankings, are ephemeral,
manipulating an election in this way would leave no paper trail.
Are
there laws prohibiting Facebook from sending out ads selectively to
certain users? Absolutely not; in fact, targeted advertising is how
Facebook makes its money. Is Facebook currently manipulating elections
in this way? No one knows, but in my view it would be foolish and
possibly even improper for Facebook not to do so. Some
candidates are better for a company than others, and Facebook’s
executives have a fiduciary responsibility to the company’s stockholders
to promote the company’s interests.
The Bond study was largely ignored, but
another Facebook experiment, published in 2014 in
PNAS,
prompted protests around the world. In this study, for a period of a
week, 689,000 Facebook users were sent news feeds that contained either
an excess of positive terms, an excess of negative terms, or neither.
Those in the first group subsequently used slightly more positive terms
in their communications, while those in the second group used slightly
more negative terms in their communications. This was said to show that
people’s ‘emotional states’ could be deliberately manipulated on a
massive scale by a social media company, an idea that many people found
disturbing. People were also upset that a large-scale experiment on
emotion had been conducted without the explicit consent of any of the
participants.
Facebook’s
consumer profiles are undoubtedly massive, but they pale in comparison
with those maintained by Google, which is collecting information about
people 24/7, using
more than 60 different observation platforms
– the search engine, of course, but also Google Wallet, Google Maps,
Google Adwords, Google Analytics, Chrome, Google Docs, Android, YouTube,
and on and on. Gmail users are generally oblivious to the fact that
Google stores and analyses every email they write, even the drafts they
never send – as well as all the
incoming email they receive from both Gmail and non-Gmail users.
if
Google set about to fix an election, it could identify just those
voters who are undecided. Then it could send customised rankings
favouring one candidate to just those people
According to Google’s
privacy policy
– to which one assents whenever one uses a Google product, even when
one has not been informed that he or she is using a Google product –
Google can share the information it collects about you with almost
anyone, including government agencies. But never with
you. Google’s privacy is sacrosanct; yours is nonexistent.
Could
Google and ‘those we work with’ (language from the privacy policy) use
the information they are amassing about you for nefarious purposes – to
manipulate or coerce, for example? Could inaccurate information in
people’s profiles (which people have no way to correct) limit their
opportunities or ruin their reputations?
Certainly,
if Google set about to fix an election, it could first dip into its
massive database of personal information to identify just those voters
who are undecided. Then it could, day after day, send customised
rankings favouring one candidate to just those people. One advantage of this approach is that it would make Google’s manipulation extremely difficult for investigators to detect.
Extreme forms of monitoring, whether by the KGB in the Soviet Union, the Stasi in East Germany, or Big Brother in
1984,
are essential elements of all tyrannies, and technology is making both
monitoring and the consolidation of surveillance data easier than ever.
By 2020, China will have put in place the most ambitious government
monitoring system ever created – a single database called the
Social Credit System,
in which multiple ratings and records for all of its 1.3 billion
citizens are recorded for easy access by officials and bureaucrats. At a
glance, they will know whether someone has plagiarised schoolwork, was
tardy in paying bills, urinated in public, or blogged inappropriately
online.
As Edward
Snowden’s revelations made clear, we are rapidly moving toward a world
in which both governments and corporations – sometimes working together –
are collecting massive amounts of data about every one of us every day,
with few or no laws in place that restrict how those data can be used.
When you combine the data collection with the desire to control or
manipulate, the possibilities are endless, but perhaps the most
frightening possibility is the one expressed in Boulding’s assertion
that an ‘unseen dictatorship’ was possible ‘using the forms of
democratic government’.
Since Robertson and I submitted our initial report on SEME to PNAS
early in 2015, we have completed a sophisticated series of experiments
that have greatly enhanced our understanding of this phenomenon, and
other experiments will be completed in the coming months. We have a much
better sense now of why SEME is so powerful and how, to some extent, it
can be suppressed.
We
have also learned something very disturbing – that search engines are
influencing far more than what people buy and whom they vote for. We now
have evidence suggesting that on virtually all issues where people are
initially undecided, search rankings are impacting almost every decision
that people make. They are having an impact on the opinions, beliefs,
attitudes and behaviours of internet users worldwide – entirely without
people’s knowledge that this is occurring. This is happening with or
without deliberate intervention by company officials; even so-called
‘organic’ search processes regularly generate search results that favour
one point of view, and that in turn has the potential to tip the
opinions of millions of people who are undecided on an issue. In one of
our recent experiments, biased search results shifted people’s opinions
about the value of fracking by 33.9 per cent.
Perhaps
even more disturbing is that the handful of people who do show
awareness that they are viewing biased search rankings shift even further in the predicted direction; simply knowing that a list is biased doesn’t necessarily protect you from SEME’s power.
Remember what the search algorithm is doing: in response to your query, it is selecting a handful of webpages from among the billions that are available, and it is ordering
those webpages using secret criteria. Seconds later, the decision you
make or the opinion you form – about the best toothpaste to use, whether
fracking is safe, where you should go on your next vacation, who would
make the best president, or whether global warming is real – is
determined by that short list you are shown, even though you have no
idea how the list was generated.
The
technology has made possible undetectable and untraceable manipulations
of entire populations that are beyond the scope of existing regulations
and laws
Meanwhile,
behind the scenes, a consolidation of search engines has been quietly
taking place, so that more people are using the dominant search engine
even when they think they are not. Because Google is the best search
engine, and because crawling the rapidly expanding internet has become
prohibitively expensive, more and more search engines are drawing their
information from the leader rather than generating it themselves. The
most recent deal, revealed in a
Securities and Exchange Commission filing in October 2015, was between Google and Yahoo! Inc.
Looking
ahead to the November 2016 US presidential election, I see clear signs
that Google is backing Hillary Clinton. In April 2015, Clinton hired
Stephanie Hannon
away from Google to be her chief technology officer and, a few months
ago, Eric Schmidt, chairman of the holding company that controls Google,
set up a semi-secret company
– The Groundwork – for the specific purpose of putting Clinton in
office. The formation of The Groundwork prompted Julian Assange, founder
of Wikileaks, to dub Google Clinton’s ‘
secret weapon’ in her quest for the US presidency.
We
now estimate that Hannon’s old friends have the power to drive between
2.6 and 10.4 million votes to Clinton on election day with no one
knowing that this is occurring and without leaving a paper trail. They
can also help her win the nomination, of course, by influencing
undecided voters during the primaries. Swing voters have always been the
key to winning elections, and there has never been a more powerful,
efficient or inexpensive way to sway them than SEME.
We
are living in a world in which a handful of high-tech companies,
sometimes working hand-in-hand with governments, are not only monitoring
much of our activity, but are also invisibly controlling more and more
of what we think, feel, do and say. The technology that now surrounds us
is not just a harmless toy; it has also made possible undetectable and
untraceable manipulations of entire populations – manipulations that
have no precedent in human history and that are currently well beyond
the scope of existing regulations and laws. The new hidden persuaders
are bigger, bolder and badder than anything Vance Packard ever
envisioned. If we choose to ignore this, we do so at our peril.
Aeon
