Beyond the Panopticon: Faceborg update

Facebook 1984 INGSOC

I have always found Facebook to be unpleasant and creepy. Now an increasing number of people are coming to the conclusion that it’s positively dangerous. Here’s a powerful tweetstorm by Google AI researcher François Chollet:

The problem with Facebook is not *just* the loss of your privacy and the fact that it can be used as a totalitarian panopticon. The more worrying issue, in my opinion, is its use of digital information consumption as a psychological control vector. Time for a thread

The world is being shaped in large part by two long-time trends: first, our lives are increasingly dematerialized, consisting of consuming and generating information online, both at work and at home. Second, AI is getting ever smarter.

These two trends overlap at the level of the algorithms that shape our digital content consumption. Opaque social media algorithms get to decide, to an ever-increasing extent, which articles we read, who we keep in touch with, whose opinions we read, whose feedback we get

Integrated over many years of exposure, the algorithmic curation of the information we consume gives the systems in charge considerable power over our lives, over who we become. By moving our lives to the digital realm, we become vulnerable to that which rules it — AI algorithms

If Facebook gets to decide, over the span of many years, which news you will see (real or fake), whose political status updates you’ll see, and who will see yours, then Facebook is in effect in control of your political beliefs and your worldview

This is not quite news, as Facebook has been known to run since at least 2013 a series of experiments in which they were able to successfully control the moods and decisions of unwitting users by tuning their newsfeeds’ contents, as well as prediction user’s future decision

In short, Facebook can simultaneously measure everything about us, and control the information we consume. When you have access to both perception and action, you’re looking at an AI problem. You can start establishing an optimization loop for human behavior. A RL loop.

A loop in which you observe the current state of your targets and keep tuning what information you feed them, until you start observing the opinions and behaviors you wanted to see

A good chunk of the field of AI research (especially the bits that Facebook has been investing in) is about developing algorithms to solve such optimization problems as efficiently as possible, to close the loop and achieve full control of the phenomenon at hand. In this case, us

This is made all the easier by the fact that the human mind is highly vulnerable to simple patterns of social manipulation. While thinking about these issues, I have compiled a short list of psychological attack patterns that would be devastatingly effective

Some of them have been used for a long time in advertising (e.g. positive/negative social reinforcement), but in a very weak, un-targeted form. From an information security perspective, you would call these “vulnerabilities”: known exploits that can be used to take over a system.

In the case of the human mind, these vulnerabilities never get patched, they are just the way we work. They’re in our DNA. They’re our psychology. On a personal level, we have no practical way to defend ourselves against them.

The human mind is a static, vulnerable system that will come increasingly under attack from ever-smarter AI algorithms that will simultaneously have a complete view of everything we do and believe, and complete control of the information we consume.

Importantly, mass population control — in particular political control — arising from placing AI algorithms in charge of our information diet does not necessarily require very advanced AI. You don’t need self-aware, superintelligent AI for this to be a dire threat.

So, if mass population control is already possible today — in theory — why hasn’t the world ended yet? In short, I think it’s because we’re really bad at AI. But that may be about to change. You see, our technical capabilities are the bottleneck here.

Until 2015, all ad targeting algorithms across the industry were running on mere logistic regression. In fact, that’s still true to a large extent today — only the biggest players have switched to more advanced models.

It is the reason why so many of the ads you see online seem desperately irrelevant. They aren’t that sophisticated. Likewise, the social media bots used by hostile state actors to sway public opinion have little to no AI in them. They’re all extremely primitive. For now.

AI has been making fast progress in recent years, and that progress is only beginning to get deployed in targeting algorithms and social media bots. Deep learning has only started to make its way into newsfeeds and ad networks around 2016. Facebook has invested massively in it

Who knows what will be next. It is quite striking that Facebook has been investing enormous amounts in AI research and development, with the explicit goal of becoming a leader in the field. What does that tell you? What do you use AI/RL for when your product is a newsfeed?

We’re looking at a powerful entity that builds fine-grained psychological profiles of over two billion humans, that runs large-scale behavior manipulation experiments, and that aims at developing the best AI technology the world has ever seen. Personally, it really scares me

If you work in AI, please don’t help them. Don’t play their game. Don’t participate in their research ecosystem. Please show some conscience

Now might be an opportune time to talk about government regulation of the social media platforms. And maybe even a temporary shutdown, until we can figure out what is going on and how to mitigate the risk of social catastrophe from this new and poorly understand technology.

Anti-politicians

Beppe Grillo

Italy’s Beppe Grillo

The enigmatic Five Star Movement (Movimento 5 Stelle, or M5S) is now Italy’s biggest political party:

One person notably absent from the Five Star Movement’s triumphant celebrations at a plush hotel in Rome in the early hours of Monday morning was Beppe Grillo, the comedian who less than a decade ago founded the party that seized the biggest share of the vote in Sunday’s inconclusive election. […]

Grillo, who was instrumental in turning the movement built by a rabble of rebels into Italy’s strongest political force, said in January that unless it won an outright majority in the election it should remain in opposition. “It would be like saying that a panda can eat raw meat. We only eat bamboo,” he said of the prospect of sharing power.

But Di Maio, said to have been groomed by Grillo for the leadership, has other ideas. On Monday he said he was open to talks with all political parties, and he has already presented his would-be cabinet – a list of what he calls “anti-politicians”.

In light of the news from Italy, I remembered this prescient article by Francesco Sisci — from April 2013:

Italy over the past century was a staging ground for experiments with new political solutions that had global consequences. Fascism was born in Italy in the 1920s, although it also flourished elsewhere and caused the start of World War II. In the 1970s, the Italian pro-Soviet Communist Party supported coalition governments that included pro-American parties, showing that communism could be adapted to a democratic environment. Thus, it inspired reforms in Gorbachev’s USSR some years later, something that led to the collapse of communism in Europe altogether.

One then wonders whether the new Italian political entity the “5 Star Movement”, created by comedian-turned-politician Beppe Grillo, will also lead to something else – and what that could be. The “5 Star Movement” scored a huge success in the recent Italian elections while refusing to reach out to voters through talks and debates on TV, the traditional means of political campaigning for the past five decades.

It canvassed votes by means of old-fashioned public meetings and by modern web chats and Tweets shot through the Internet and mobile phones. He and his followers explained that this is the new web democracy. In fact, there is something extremely modern in Grillo’s political movement. Certainly, US President Barack Obama understood the importance of the web and relied on songs spread on Facebook and Twitter slogans. But he still went on TV and engaged in all the traditional campaign activities.

Grillo, conversely, refused TV appearances, political debates, and even interviews in the Italian press, and this magnified his image, bringing him almost 25% of the vote. The Internet is and was the ground for internal debates. Candidates were selected through mock elections on the web among Grillo’s supporters; policy discussions were held in web chats rather than in smoky rooms. There were no meetings, no cells, and no steering committees.

Actually, this is not the only new element of Grillo’s party. Contrary to all past practices, Grillo and his main partner, Gianroberto Casalegno, chose not to run for parliament. Notwithstanding that, these two extra-parliamentary leaders control all their elected deputies in parliament through a series of binding agreements. Meanwhile, the few top leaders decide the party line in informal gatherings on phone calls. It may not sound good – the party looks more like a private entity than an organization to promote political change and effective popular participation – but it has so far provided an organization that works similar to, if not better than, the old party systems.

Social networking is devouring the political systems of the West, starting in Italy and the US. It’s easy to imagine that in another five or 10 years, online networks will have taken over the machinery of the major parties, turning politicians into puppets for internet movements/mobs. Which may or may not be an improvement over the existing, obsolete party systems.

In any case, China’s prescience in censoring the internet more severely than Saudi Arabia is now clear. China’s rulers are extremely uninterested in dealing with uncontrollable, socially networked movements that could destabilize the country and threaten the Party’s grip on power, so it has opted to wall off China from huge swaths of the global internet.

Under this brutal logic, all major foreign social media platforms are blocked, and the domestic platforms are heavily censored and monitored. Weibo, the closest thing to Twitter, has recently been chastened (again). The closed nature of WeChat, which now has a billion active monthly users, does not lend itself to hashtag activism. There will be no Beppe Grillo on China’s watch.