Rise of the Trolls

_______ |_ _| | | | | ____| | | | | | | | \______/ ournalist Nedim Turfent was reporting on a brutal counterterrorism operation in Turkey’s Kurdish region when he published video of soldiers standing over villagers, who were face down with their hands bound. Soon, odd messages seeking Turfent’s whereabouts began appearing on his Facebook page.

Then, Twitter accounts linked to Turkish counterterrorism units joined in, taunting locals with a single question—“Where is Nedim Turfent?”—as soldiers torched and raided more villages.

The threat was clear: Give him up, or you’re the next target.

That was in the spring of 2016. Within days, Turfent was in the military’s hands, and he was eventually charged with membership in a terrorist organization. An anonymous Twitter account capped off the social media manhunt by tweeting a picture of Turfent in custody, handcuffed and haggard. Then soldiers doused the office of his employer, Dicle News Agency, with gasoline and set it ablaze. Turfent remains behind bars.

Only a few years after Twitter and Facebook were celebrated as the spark for democratic movements worldwide, states and their proxies are hatching new forms of digitally enabled suppression that were unthinkable before the age of the social media giants, according to evidence collected from computer sleuths, researchers and documents across more than a dozen countries.

Combining virtual hate mobs, surveillance, misinformation, anonymous threats, and the invasion of victims’ privacy, states and political parties around the globe have created an increasingly aggressive online playbook that is difficult for the platforms to detect or counter.

Some regimes use techniques like those Russia deployed to influence the 2016 U.S. presidential election, while others are riffing in homegrown ways. And an informal but burgeoning industry of bot brokers and trolls-for-hire has sprung up to assist. The efforts have succeeded in many cases, sending journalists into exile or effectively silencing online expression.

In Venezuela, prospective trolls sign up for Twitter and Instagram accounts at government-sanctioned kiosks in town squares and are rewarded for their participation with access to scarce food coupons, according to Venezuelan researcher Marianne Diaz of the group @DerechosDigitales. A self-described former troll in India says he was given a half-dozen Facebook accounts and eight cell phones after he joined a 300-person team that worked to intimidate opponents of Prime Minister Narendra Modi. And in Ecuador, contracting documents detail government payments to a public relations company that set up and ran a troll farm used to harass political opponents.

Many of those findings are contained in a report released this week by a global group of researchers that uncovered evidence of state-sponsored trolling in seven countries, and Bloomberg reporters documented additional examples in several others. The report is by the Institute for the Future, a non-partisan, foresight research and public policy group based in Palo Alto, California.

“These campaigns can take on the scale and speed of the modern internet,” the report said. “States are using the same tools they once perceived as a threat to deploy information technology as a means for power consolidation and social control, fueling disinformation operations and disseminating government propaganda at a greater scale than ever before.”

Almost two years in the making, the report grew out of an earlier project commissioned by Google but never published. Researchers for the company’s Jigsaw division, its technology incubator, documented vicious harassment campaigns that were intended to appear spontaneous but in fact had links to various governments. These campaigns often operate “under a high degree of centralized coordination and deploy bots and centrally-managed social media accounts designed to overwhelm victims and drown out their dissent,” according to an unpublished copy of the Google report obtained through an outside researcher.

In response to revolutions and social movements launched on Twitter and Facebook, national governments initially censored content, blocked access to social media and used surveillance technology to monitor their citizens. But it turned out to be far more effective to simply inundate the platforms with a torrent of disinformation and anonymized threats—what the researchers dubbed a strategy of “information abundance” made possible by the rapid spread of social media.

Turkey is a prime example, according to Camille Francois, who directed the Jigsaw project as a principal researcher at Google. Since the 2013 protests at Istanbul’s Gezi Park, President Recep Erdogan’s government has used a combination of online and offline repression to turn social media “into a near dead zone for genuine social protest in Turkey,” Francois said. “Five years later, there is very little organically organized activity.”

Social media companies such as Facebook and Twitter are struggling to counter the significant resources and ingenuity that national governments and others are investing to manipulate their platforms. Millions of fake accounts game the companies’ algorithms, manipulating what users see, while technologies such as location spoofing can make centrally controlled accounts appear to be posts by armies of real people.

Twitter reportedly suspended 70 million fake and malicious accounts in May and June. The company says it has taken preventive measures to address trolling and that any form of malicious automation is a violation of Twitter rules. Facebook announced this week that it will begin removing misinformation that serves to incite violence, and a spokesman said it has invested in more effective ways to fight fake accounts. “We enforce these policies whether the responsible party is acting individually, as part of a company, or acting on behalf of a government,” a Facebook spokesman said.

Experts say the companies need to do even more to confront the fact that they’re being used by some of the world’s most repressive regimes.

“People sometimes worry that Azerbaijan will shut down Facebook,” said Katy Pearce, a communications professor at the University of Washington who has studied the platform's use in that country. “Why would it? Facebook is the most effective tool of control the government has.”

II
Trolling for Faith and Modi

Sitting cross-legged on a charpoy, an Indian day bed, Mahaveer Prasad Khileri taps on his laptop, his face lit by the screen and a single bulb hanging from the ceiling of his dirt-walled house. He uses his computer and two smartphones to advocate on social media for an organization dedicated to the well-being of cows, which are sacred to Hindus. Khileri’s work serves as a penance of sorts for a time when his deep faith and social media skills found a more toxic expression. He’s a former troll for India’s ruling Bharatiya Janata Party, or the BJP. “At that time, poison was in my mind,” he said.

Khileri was recruited by two acquaintances into the party’s social media operation in February 2014, just as Modi was racing to become India's next prime minister. He was given eight cell phones and ID’s for six different Facebook identities, he recalled in an interview in his home village of Jogaliya. He worked 18-hour days, toggling between legitimate campaign work and trolling of opponents and journalists, he said. When Modi won, the operation evolved as well, transitioning to a tool supporting Modi’s government.

Khileri worked in what the BJP calls its ‘IT Cell,’ which effectively operated as an ad hoc troll farm, he said. The development of the cell in the world's largest democracy occurred around the same time that American authorities believe Russia began using such techniques to influence the 2016 presidential election. The researchers contributing to the institute and Google reports found similar timing in different countries and under various circumstances.

Mahaveer Prasad Khileri, a former troll for the ruling Bharatiya Janata Party, at his home in Jogaliya, India in May 2018. Source: Bibhudatta Pradhan of Bloomberg News

According to Khileri, the Indian version of the trolling toolkit included strategies meant to inflame sectarian differences, malign the Muslim minority and portray Modi as savior of the Hindus. Supervisors would set themes for the day and specify targets to attack. Khileri and 300 other paid trolls would create memes or cut-and-paste Twitter posts that were sent to WhatsApp groups of tens of thousands of party loyalists. Their reposts sent hashtags viral in minutes.

“Muslims slaughter cows, so we’d tell them, ‘When Modi comes, we will slaughter you,’” Khileri recalled. “We’d tell Hindus: ‘If you don’t vote for Modi, then Muslims will destroy you.’”

The former head of BJP’s IT CELL, Arvind Gupta, tweeted in December 2016 that neither the party nor the cell has ever encouraged trolling and that online support for the party comes from a voluntary, grass-roots movement. The current head of the cell, Amit Malviya, said he would comment only after seeing evidence that Khileri was a member of it. Khileri said he eventually quit the cell—which paid its members in cash and left no paper trail—after he grew increasingly uncomfortable with the power it amassed.

“At any time, they can control the situation of India. The troll army can call a nationwide strike, shut down the country,'' he said. “They are able to push for fights between communities, to create communal tension or destroy communal harmony.”

III
From Jingles to Death Threats

Trolling efforts take different forms in different countries. Party youth groups were commandeered in some. Others developed highly structured volunteer armies. Some simply paid contractors to do the work. Always, though, organizers took pains to make the activity appear spontaneous.

In Ecuador, former President Rafael Correa engaged a Guayaquil-based public relations firm called Inteligencia Emocional to promote his 2012 reelection, complete with videos, jingles, Facebook fan pages and a unified social media front under the brand Yo Revolución.

But additional work by the firm was more secretive. It set up and maintained a network of pro-Correa social media accounts that harassed political opponents, according to documents released by Ecuador Transparente, a whistleblower website that opposed the Correa administration. In a proposal to the Ecuadorean government, the firm proposed to charge $15,000 per month to staff its network around the clock with “community managers” in a “war room,” enabling it to “totally neutralize” offending content on the internet.

Internal Inteligencia Emocional documents anonymously leaked to Ecuador Transparente described the operation as a “troll center” and said that, due to the sensitivity of the project, it would implement a series of security protocols. Those steps included encrypting data shared among computers, scrubbing metadata from all materials and obscuring the location of the computers.

Correa did not respond to requests for comment; nor did the head of Inteligencia Emocional, Kenneth Godwin.

Another set of leaked documents seen by Bloomberg showed that Ecuador’s trolling operation involved the country's intelligence directorate. The operation eventually became notorious for death threats and trafficking in hacked personal material of journalists and political opponents. Even ordinary citizens who used social media to speak out against the government were targeted, according to Martha Roldós, an investigative journalist and the daughter of a former president of Ecuador who has been a target of the troll operations.

A leaked document shows Venezuela's plans for developing a government-backed “troll army,” including a military-style organizational structure. The country’s Interior Ministry did not return requests for comment about the plan.

IV
Organized Like Drug Cartels

In Mexico, the interference of fake Facebook and Twitter accounts in public discourse has been so pernicious during the tenure of outgoing President Enrique Pena Nieto that his political opponents have dubbed them “Penabots.” Bot networks have also been used against the government’s agenda.

Two years ago, investigative journalist and activist Alberto Escorcia began documenting one of Mexico's unique contributions to trolling: loosely organized youth gangs that former members told Escorcia are paid to stage virtual hate-mob attacks on journalists or activists who confront the government.

A picture of Kurdish journalist Nedim Turfent posted by a Twitter user after his arrest in May 2016, with a message saying, the “bastard was arrested!”. Source: Twitter

With monikers such as the Holk Legion, the gangs are organized like the region’s infamous drug cartels, Escorcia said. In one case documented by Amnesty International, Holk Legion trolls used more than 2,000 accounts to send death threats to 10 journalists and public figures over two days. The blizzard of tweets came on the second anniversary of a mass kidnapping of students in Guerrero State, an event that has plagued Peña Nieto's government amid evidence of police and military involvement. The trolls’ menacing tweets arrived as a counterweight to renewed public outrage sparked by the anniversary.

Eduardo Sanchez, a spokesman for the Pena Nieto administration, said that “there’s no such thing” when asked by Bloomberg about the allegations that some networks of trolls have promoted the government’s agenda.

The Google report that examined government-connected trolling was initially set to be published last year, according to researchers who collaborated with the company. Some of them said congressional and law enforcement investigations into Russia's election meddling may have made the topic too sensitive.

A spokesman for the company’s Jigsaw division, Dan Keyserling, said not all research projects end up being published and that information from the report was used internally to improve tools that detect abusive content. Jigsaw supports the fact that other organizations are continuing research into state-sponsored trolling, he said.

V
How to Build a Botnet

Turkey's experience shows how politicians turned social media platforms into tools of information control—and rapidly perfected their techniques. Erdogan's government was badly shaken by protests that began in May 2013 and grew over weeks to focus on government corruption. Within months, the ruling party was fielding its own bot army, albeit a poorly disguised one. Researchers found a collection of nearly 18,000 pro-Erdogan Twitter accounts that used profile pictures taken from porn sites or public figures as American actress Megan Fox.

By 2017, the country's digital troops had evolved into something more finer-tuned and threatening. A report by AccessNow, a digital rights organization, identified a collection of fake accounts posing as people sympathetic to a protest march scheduled for that summer. The event would focus attention on the crackdown on journalists, teachers and others following an attempted coup in 2016.

One such account, with a picture of a young woman claiming to be a march supporter, posted a link to a website for participants. Visitors to the site had their devices infected with advanced malware that spied on their communications and tracked their movements, the AccessNow report found. The company that created the malware sells only to governments.

Researchers and journalists have documented similar bot armies in Argentina, Thailand, Bahrain and Saudi Arabia.

Saudi trolling was on display after the May 15 arrest of seven prominent Saudi women’s rights activists who had been demanding an end to the ban on female drivers. The Saudi government initially gave no information about the activists’ whereabouts. Concerned Twitter users posted messages followed by an Arabic hashtag “Where are the activists?” That was quickly countered with a hashtag circulated by a state-backed news organization labeling the activists “Agents of the Embassies,” alongside pictures of them with the Arabic word for “Traitor” stamped over their faces

It got uglier. “How’s prison food?” one account asked Loujain Al-Hathloul, a Western-educated 28-year-old who’d previously been detained after live-Tweeting herself driving. “You act like you’re something when you’re nothing,” read another, next to a knife emoji. Another account linked to an archive of Al-Hathloul’s tweets, so opponents could scour the words of an “enemy of the homeland.”

An analysis by Graphika, a social media intelligence firm based in New York, showed that the “Agents” hashtag was “pushed” in a highly coordinated way, said John Kelly, the company's CEO and founder. Marc Owen Jones, a researcher and lecturer on Gulf affairs at Exeter University, found evidence that the hashtag was pushed by automated accounts tied to a vast pro-Saudi government botnet that he'd previously identified.

“It still remains to be seen whether this is a state-sponsored operation, the work of a PR company, or a wealthy individual’s unilateral project,” Owen Jones wrote about that botnet in February.

VI
Misogyny and Rape Threats

Some of the most virulent attacks are aimed at women. The Institute for the Future report found that every identified instance of state-sponsored trolling involving female victims used heavily misogynistic language, including threats of rape and mutilation.

The graphic, relentless posts often prove to be powerful weapons, publicly singling out a regime's opponents and legitimizing those who attack them, while leading some journalists to self-censor their reporting, according to the report.

This meme, replacing the image of Jesus Christ next to Pontius Pilate with that of Daphne Caruana Galizia, was posted a few weeks before the journalist's assassination on Oct. 16, 2017. Source: Private Facebook group

Before she was killed by a car bomb last October 2017, Daphne Caruana Galizia, who gained fame writing about corruption on the tiny Mediterranean island nation of Malta, grew so anxious about the online attacks that greeted her blog posts that she often hesitated before publishing a new one, said her son, Paul Caruana Galizia.

Many of the violent comments against his mother were coordinated inside a handful of private Facebook groups run by government employees and which count among their members ruling Labour Party lawmakers. The secret groups, whose operations were revealed in the months following her death, circulated “thousands of really horrifically abusive comments” against his mother, he said.

One meme, which circulated shortly before her death, substituted her image for that of Jesus Christ next to Pontius Pilate, who asks the crowd: “What do you want me to do with her?” Comments encouraging violence poured in. “The dehumanization by the time she was killed around that period was complete,” her son said.

According to examples uncovered by researchers, trolls also threatened to sexually abuse Turkish journalist Ceydan Karan with a broken bottle, while Mexican scholar Rossana Reguillo was sent pictures of burned bodies with the warning, “This could happen to you.”

The fact that the threats are designed to look like they are coming from crowds of anonymous social media users makes them more ominous in some cases than if they had been issued directly by the government.

“That’s the genius of these types of attacks,” said Carly Nyst, one of the authors of the Institute for the Future report and an expert on the intersection of human rights and technology. “It’s hard to distinguish between what’s being manufactured on purpose and what is a popular uprising of opinion against the target.”

One question left unanswered by the reports is why so many nations developed strikingly similar trolling operations at around the same time. There is some evidence of information-sharing among countries, consultants and government functionaries.

Correa, the former president of Ecuador, now hosts a television show for the Spanish-language version of RT, the Russian news channel that is closely tied to sophisticated government-sponsored information campaigns. And last year, when Philippines President Rodrigo Duterte was in the Kremlin meeting with Russian President Vladimir Putin, a representative of the Philippines state-run media visited the Moscow headquarters of RT and entered into a partnership with the news agency and Russia’s communication ministry.

Duterte has used Facebook as a platform to aggressively target critics, and he has appointed well-known trolls to government posts.

In Azerbaijan, the government maintains a close relationship with Russia, which has exported some of its technological know-how, including bots, according to Pearce, the University of Washington professor.

Still, tracing the fingerprints of state-sponsored trolling remains a difficult task, largely because states go to great lengths to cover tracks. Francois, the former Google researcher, says the architecture of a trolling campaign is often akin to a hand inside a glove that organizes an action and then is removed.

While the glove is left behind for the world to see, the hand simply disappears.