Thieves, Troublemakers & Useful Idiots
"We  live  in  a  society  in  which  spurious  realities  are manufactured  by  the  media,  by governments,  by  big  corporations,  by  religious  groups,  political  groups.  I  ask,  in  my  writing, 'What is real?' Because unceasingly we are bombarded with pseudo realities manufactured by very sophisticated people using very sophisticated electronic mechanisms."   Philip K Dick
All misinformation is bad and potentially dangerous. It doesn’t matter whether it is circulated innocently by people wanting to inform, educate, amuse, impress or comfort others, or whether it is spread deliberately by cynical individuals, criminals, theocrats or hostile foreign powers seeking to control, distract, deceive, embarrass, frighten, injure or undermine; either way the consequences are profound and far-reaching, and all too often, socially-divisive. [1]

This page deals with:
1     Information Spun & Shared on an Industrial Scale
1    Information Spun & Shared on an Industrial Scale
"Je participe. Tu participes. Il participe. Nous participons. Vous participez. Ils profitent" [2]
One of many slogans from the May 1968 student uprising in Paris

Misinformation is today being generated and propagated on an industrial scale. This has been made possible by advances in IT and communication technology, and the proliferation of social networking services like Facebook, video sharing websites like YouTube, and open source blogging sites like WordPress. This in turn has enabled information and opinion to be shared, spun and re-posted; and where the material is sufficiently eye-catching, saucy or scurrilous, to go viral and within minutes reach a global audience, potentially of millions. But very little of this phenomenal output is edited or peer reviewed, and much of it is partial, biased or plain wrong.
This torrent of often questionable information and opinion is drowning out accurate news and considered comment, and leaving everyone confused and struggling to tell fact from fiction. Indeed, in this respect it is worrying that an increasing number of us are relying on social media for much or all of our news because we no longer want to pay for news and comment. And one consequence of this is that  traditional newspapers and mainstream news channels are going to the wall, and those whose job it is to winkle out the truth and contextualize it for the rest of us, are being put out of business.
Whilst effective methods for assessing information on the internet are now available most are labour-intensive and cannot keep pace with the speed at which fake news is evolving. So while the Internet is a fantastic tool for intellectual, social and economic development, it has also become a conduit for rumour, bile and human detritus. It has been described as "an electronic asylum filled with babbling loonies" (Mike Royko), "the biggest lavatory wall in history" (AC Grayling), and the place where "no gatekeepers insist you pay the price of accuracy before publishing and lies are given the same status as truth."  (Nick Cohen)
2    Who's Generating The Stuff?
At one end of the 'lavatory wall' we have the immature, ill-informed, dishonest, disturbed or sick individuals locked away in their bedrooms with their smart phones, tablets or laptops — and we can include here the output from wise-boys and fraudsters who use click-bait[3] to attract punters to their dubious promotions, and the sick, sad or malevolent individuals who create or spread fake news and conspiracy theories.

At the other end of the 'wall' there are hostile foreign powers spreading fake news, disinformation and confusion, visiting all sorts of problems on other nations, and undermining public confidence in democracy and the open society. And then somewhere in between are major corporations, media moguls, and government bodies that carefully select their ‘facts’ or spin stories, and twist or distort the truth in the process. We also have the powerful lobbyists and pressure groups with their own special interests (anti-vaxers, climate change deniers, alternative therapists and the like) — and religious bodies and cults that seek to present belief as ‘fact’ — who collectively reject science and have little or no respect for reasoned argument.

We have provided a summary of these various groups in the table below (and in this file).
3    False Accounts & Fake 'Likes'
Some people are making a lot of money out of selling fake followers. In Jan 2018 New York's Chief Prosecutor announced that the state was opening an investigation into one operation, Devumi, that is alleged to have sold millions of fake followers to social media users. The Prosecutor said he was concerned that such ‘opaque’ operations were undermining democracy.
The action follows publication, by the New York Times, of an explosive report, ‘The Follower Factory’, which accuses Devumi of stealing people's identities  (a claim it denies). The report also contains details of interviews with people who allege that their account details and profile pictures had been copied to create bots. It claims that actors, entrepreneurs and political commentators who wanted to increase their follower count were paying to be followed by these (often highly realistic) bots. On social media, high follower accounts boost influence, which can impact public opinion, or bring advantages, such as job offers or sponsorship deals, to account holders. [4]
How Fakers cover their tracks
Someone’s profile and background images are stolen; their portrait is then colour-shifted and recompressed (to evade automated detection). One or more letters in the account name is changed e.g. from a lowercase 'i' to a lowercase 'l', or on '0' for an 'O' (which is easy to miss). Two give-aways of fake accounts are: unusual ratios (bots typically following thousands of accounts but with very few followers themselves), and bot accounts retweeting content on a dizzying assortment of topics and in a range of different languages.
4    Digital Warfare
"The point of modern propaganda isn’t only to misinform or push an agenda. It is to exhaust your critical thinking, to annihilate truth."    Garry Kasparov
We don’t know very much about what dirty tricks different governments are up to (including our own). There is much criticism in the West about psychological warfare being waged by Russia (and China & North Korea), and much of the criticism is focused on the work of Sputnik (formerly The Voice of Russia). This news service, established by a Russian government-controlled news agency, is geared towards non-Russian audiences, and has been widely accused of bias and spreading disinformation. Sputnik currently operates news websites in over 30 languages and runs newswire services around the clock; it also produces photo essays, live streaming, infographics and public opinion surveys. Shortly after it was launched (in 2014) Foreign Policy magazine described Sputnik as a "slick and internet-savvy outlet of Kremlin propaganda" which "remixes President Vladimir Putin's brand of revanchist nationalism for an international audience... beating a predictable drum of anti-Western rhetoric." [5]
 How It's Done
A study published in an authoritative Russian military source (see Giles) lists "the primary methods of manipulating information used by the mass media in the interests of information-psychological confrontation objectives as:
●   Direct lies for the purpose of disinformation both of the domestic population and foreign societies;
●   Concealing critically important information;
●   Burying valuable information in a mass of information dross;
●   Simplification, confirmation & repetition (inculcation);
●   Terminological substitution: use of concepts and terms whose meaning is unclear or has undergone qualitative change, which makes it harder to form a true picture of events;
●   Introducing taboos on specific forms of information or categories of news;
●   Image recognition: known politicians or celebrities can take part in political actions to order, thus exerting influence on the world view of their followers;
●   Providing negative information, which is more readily accepted by the audience than positive.”
Global Voices (virtual non-profit newsroom) has reported that Sputnik "acts as a spoiler to try and disrupt or blur information unfriendly to Russia, such as Russian troops' widely alleged involvement in the war in Ukraine." The European Parliament has accused Sputnik (and RT, Russia Today) of ‘information warfare’, and placed Russian media organisations alongside terrorist organisations such as the Islamic State.
Quite how the CIA, GCHQ and other Western intelligence agencies are responding to such threats is anybody’s guess. We learned (in Feb 2014) that GCHQ had a ‘Joint Threat Research Intelligence Group’ whose mission is cyber operations based on ‘dirty tricks’ to "shut down enemy communications, discredit, and plant misinformation on enemies." These operations are said to account for 5% of all GCHQ operations...[6]
“Russia’s ability to successfully conduct hybrid warfare is predicated on the creation of a fog of ambiguity between the Kremlin’s actions and the Kremlin itself. By conducting operations through an ad hoc network of proxies — whose connection to the Kremlin is difficult to definitively establish — the Russian government is able to maintain plausible deniability and thus lower the diplomatic and military costs of its actions.[7] This presents an obvious challenge: How can the West respond to an adversary who denies even being present on the battlefield?”

The spread of disinformation has been likened to the laundering of illicit funds. Just as ill-gotten money needs to be moved from an illegitimate source into an established financial institution, disinformation is most powerful when a façade of legitimacy is created through ‘information laundering’.
Effective disinformation campaigns rely on social media accounts that can disseminate information in a manner that masks both its intent and its source. In the money laundering context, this process is often achieved through the use of shell companies — firms with no physical location and little-to-no assets — because they are cheap to establish, provide anonymity, and can be jettisoned if their true purpose is revealed. And one approach is ‘layering’. This involves either the use of middle-men (who seemingly have no relation to the originator of the information) or indirect citations (known as ‘cascading citations’) from unsubstantiated social media posts to seemingly legitimate news sources. The use of such intermediaries provided the information with a greater sense of legitimacy. “Once a misleading rumour enters the ‘mainstream,’ it is almost impossible to combat, even if it is subsequently debunked." [Condensed extracts from a Policy Brief by Alliance for Securing Democract]
5    The Anti-Brigade
“Not all viewpoints are created equal. Demonstrably racist, sexist, homophobic views are not worthy of debate because there's literally nothing about them with any truthful or factual substance.”
              Channel Awesome
Another cause for concern is the tendency of the media to 'play fair' and give equal airtime / column inches to the different sides of some important arguments, as if the two sides were equally valid — like teaching creationism in school alongside evolution, with the latter talked of as 'just a theory'. Often, 'false equivalence' like this would not matter, but when it comes to such topics as vaccination, genetic modification, stem-cell research and climate change, it most certainly does! Much of the argument used by denialists might be categorised as 'misleading' or 'manipulated content'.
6    Is AI Neutral?
Putting out misleading information is one concern; machine bias (and devious programming) is another.  With Artificial intelligence (AI) playing an increasingly important role, there is growing concern about the way algorithms are being used by the big tech companies.  Whilst algorithmsy can in principle detect fake news, hate speech, violent videos and the like, smart software is also making decisions about the selection and routing of videos and other products on line, and the way they do this can favour some clients or interests over others. Following the 2016 US Presidential Election it has been claimed that YouTube’s AI was partisan towards the candidates.  On the eve of the Election, one group of researchers gathered recommendation data on the two main candidates and found that "more than 80% of recommended videos were favorable to Trump, whether the initial query was 'Trump' or 'Clinton'." Moreover, "a large proportion of these recommendations were fake news.” [8] Whether this influenced voting patterns is yet to be established.
7    Worse to Come?
And there are signs that things are about to get worse:
Expect to see fake video being used increasingly, with footage and voices replicated and manipulated, and new words put into people’s mouths, hastening the time when simulation is indistinguishable from reality. A number of software packages have recently become available that are blazing a trail to this new future. [9]

In this video the engineer demonstrating the software is asked about  the technology getting into the wrong hands, and he replies 'we're thought about that' and 'we're working on it' This is hardly reassuring...
In 2018, Apps became available that enable people to superimpose someones face, perhaps a famous actress or a victim of revenge porn, onto an adult video. This has grotesque potential implications and consequences. Now add into the mix ‘fake news farms’ and we're in even more trouble — one recent study found that a single farm had been responsible for over 78,000 similarly named, keyword-packed YouTube videos across 19 channels with a new video uploaded every three to four minutes on some.[10]
We have provided a discussion of the different types of fake news on a separate page, and briefly described some recent examples. There are also more examples of video manipulation here.
8    The Internet Untamed
In summary then, the Internet  — which started wholly innocently as a way for scientists to share research findings — has developed into an extraordinary tool for research, education, commerce, entertainment and social intercourse, but it is today being used increasingly to disseminate tittle-tattle, rumour, uninformed opinion, negative advertising, conspiracy theories, and worse, hate mail, revenge porn and extremist propaganda — and there's even worse on the Dark / Deep Web. [11]
This development has serious potential consequences for the quality of public discourse, and for social cohesion and democratic government. Indeed, the stakes could not be higher. This explains why so much effort is now going into tackling fake news and malign propaganda. We discuss this on a separate page and have provided information on what concerned citizens and groups can themselves do here.

If you would like to comment on this or any other page, you'll find a feedback form here.
Notes

1   The term ‘useful idiots’ was born of the Cold War; it describes people or institutions that unknowingly assisted Soviet Union propaganda efforts. We are using it here to also include people who reject reasoned argument or scientific facts in favour of their own interpretation of events.

2      "I take part. You take part. He takes part. We take part. You all take part. They profit."

3   'Clickbait' is  web content that is aimed at generating online advertising revenue, especially at the expense of quality or accuracy (see Glossary) . There is a growing backlash against clickbait, which started around 2014 when the satirical digital organ The Onion launched a new website, ClickHole, that parodied clickbait websites.

4     On its website, Devumi offers customers the chance to order up to 250,000 Twitter followers, with prices starting at $12 (£8.50). Clients can also buy ‘likes’ and retweets. The company sells followers on a range of other platforms, including Pinterest, LinkedIn, Soundcloud and YouTube. The Devumi website claims to have “helped over 200,000 businesses, celebrities, musicians, YouTubers and other pros gain more exposure and make a big impact to their audience.” The NYT  report claims that “At least 55,000 of the accounts use the names, profile pictures, hometowns and other personal details of real Twitter users” and profiles one young woman, Jessica Rychly, whose social identity was stolen by a Twitter bot when she was in high school.

5     For an excellent description of Russia's disinformation campaign see this article by Ben Nimmo.

6   The existence of the JTRIG was revealed in documents released by Edward Snowden. It is right that people are skeptical about the impact of new technologies or techniques (genetic manipulation, whether to eradicate congenital diseases or improve crop yields/introduce better disease-resistance, etc.) however any criticism should be constructive and have some basis in fact.

7   This is “a strategy with deep roots: maskirovka — a Soviet-era military doctrine that translates as ‘mask’ or ‘masquerade’ — established operational deceit as a core tenet of both conventional and irregular warfare. While modern maskirovka is most commonly associated with the use of “little green men” to occupy Crimea, it is a tactic that is also deeply ingrained in the Kremlin’s ongoing disinformation campaign against the United States and Europe.” One way of detecting information laundering is where multiple accounts with common IP addresses are set up rapidly, sometimes outside of normal daytime hours for the account’s claimed physical location, and routinely retweet or share posts from other accounts with no additional activity. (ibid Policy Brief by AFSD)

8   YouTube's AI system recommends tens of billions of videos every single day, yielding billions of views. It is optimized to maximize time spent online and clicks; the combination of those is called engagement. Hence, recommendations are aligned with engagement. YouTube has been blamed inter alia for a rise in Flat Earthers!

9    In November 2016 Adobe introduced 'Project VoCo,' which it described as ‘Photoshop for audio’. This can ingest 20 minutes of voice data, break it down into phonemes (the distinct sounds that make up a spoken language), and then reassemble this material to create a voice model of the speaker with their particular modulation or inflection, stress patterns, etc. And more recently, Canadian startup Lyrebird [logo right] announced plans for a software programme that can convert supplied text into spoken words. It has already provided audio clips featuring the synthesised voices of Donald Trump, Barack Obama and Hillary Clinton, saying things they never said.
The potential for spoofing is enormous, especially when background noise is added to mask audio imperfections and video face manipulation is applied. Two other packages are also available, one by the French company CandyVoice, another by Carnegie Mellon's speech group, under the name 'FestVox'; and Baidu and Google are also reported to be making advances in speech synthesis. [see Thomas Claburn's piece, 24 Apr 2017]
10   "In recent weeks there has been an explosion in what has become known as deepfakes: pornographic videos manipulated so that the original actress' face is replaced with somebody else's. As these tools have become more powerful and easier to use, it has enabled the transfer of sexual fantasies from people's imaginations to the internet. It flies past not only the boundaries of human decency, but also our sense of believing what we see and hear. Beyond its use for hollow titillation, the sophistication of the technology could bring about serious consequences. The fake news crisis, as we know it today, may only just be the beginning. Several videos have already been made involving President Trump's face, and while they are obvious spoofs it's easy to imagine the effect being produced for propaganda purposes." Dave Lee (BBC)

11   The Dark Web is the World Wide Web content that exists on darknets (overlay networks which use the Internet but require specific software, configurations or authorization to access). It consists of  hundreds of thousands of websites that use anonymity tools like Tor and I2P to hide their IP address. The Dark Web forms a small part of the Deep Web (that part of the Web not indexed by search engines). Users of the Dark Web refer to the regular web as Clearnet due to its unencrypted nature. The Tor Dark Web may be referred to as onionland, a reference to the network's top level domain suffix .onion and the traffic anonymization technique of onion routing. [Wikipedia edited] This short video provides an excellent description.