AI Is Not the End of the World

Over the last week, lots of powerful people (like Bernie Sanders and Steve Bannon) have suddenly started talking about how AI will take over and/or destroy the world. I expected this to happen eventually because it’s been a popular topic in Rationalist and related tech communities for the last few years, which Nitish Pahwa explains over at Slate. The reasons why this is happening now are interesting (Josh Marshall has a good summary), but I wanted to focus on a larger question: Why do so many people believe that AI (or something else like climate change) will soon cause dramatic and sudden changes that you could call “the end of the world”?
Popular Eschatology
I’ve been watching a lot of religious studies videos on YouTube recently (LetsTalkReligion and ReligionForBreakfast mostly), which have reminded me how common apocalyptic and messianic beliefs are in many world religions. Wikipedia has a good Apocalypticism overview, but the TLDR is that most of today’s major religions were at one point built around end times prophecies. Religions and cults always spread during times of natural disasters and social conflict, and these problems are often interpreted as signs of an imminent apocalypse. Why Religions Survive When Prophecies Fail has a good overview of what happens when things don’t go as planned.
If you believe that the history of religion can tell us important things about human psychology (which I definitely do), it is clear that many humans are inclined to believe that The End is Near. For an academic perspective, the recent article “End of world beliefs are common, diverse, and predict how people perceive and respond to global risks” has a great overview. There are many plausible causes for this, but I think one reason is that humans never want to believe the future will be similar to today, but more confusing and with a lower standard of living. When things seem to be going well, it is easy to believe in the inevitable progress of humanity and ignore the problems of distant strangers or existential risks to humanity. But during times of increasing natural disasters and obvious economic and social strife (like right now), that worldview is hard to sustain and people will be drawn to different social narratives.
Psychology research has often shown that humans hate cognitive dissonance and will be drawn towards information and beliefs that confirm their existing views of the world. Most people do not want to construct their own metaphysical beliefs, but they will learn about existing beliefs systems from close relationships or popular media and integrate the beliefs that make more sense. Is it easier to believe “society is getting worse but many people can still live decent lives so this era of decline and conflict will probably continue for decades” or “the world as we know it will soon end, to be replaced by something totally different”? Many people prefer the second view, which leads to apocalyptic thinking. Throughout history this has mostly meant prophecies of supernatural destruction, but for those of us who grew up in the world of computers and science fiction, a technological apocalypse makes more sense.
The concept of the Singularity has been a moderately popular apocalyptic belief among science and technology workers for at least 40 years, spawning books like The Singularity is Near and thousands of posts on online message boards. It shares many features with traditional messianic religions: the belief that the current world order will be suddenly replaced by something fundamentally different, that it will cause a transformation of human consciousness (immortality through brain uploading), and that it will happen in our lifetime (around 30 years in the future which keeps getting pushed back). These concepts have influenced many different social movements, including predictions of AI superintelligence and the Existential Risk branch of AI alignment and ethics research which is driving the current discussion.
Artificial Alignment
I definitely believe that AI alignment is an important problem and that we need to actively teach AI agents to respect the value of humanity and pro-social actions. Some people act like this could never work, but the only reason we have civilization in the first place is that we developed methods to teach mostly-amoral humans pro-social beliefs and ideals. When I took my Social Psychology courses on childhood moral development (including “The Good Child” by Jing Xu which is an analysis of Shanghai preschools) it became clear that most humans are born with basic empathy for close family and friends, but they need to be actively taught to care about strangers. There is no reason to believe that humans today are inherently nicer than they were during historical periods of mass violence, but we have made things better by building social institutions that mostly work to the overall benefit of humanity (despite constant attacks from many of the rich and powerful). We can use a similar approach to teach AI agents.
AI alignment is a real issue, but I do not think that superintelligent AI will be an existential risk to humanity within our lifetimes. The general “end of the world” scenario requires continued exponential growth of AI intelligence (which is starting to slow in reality), an agent or a collective with unified goals that are harmful to humanity, and unfettered access to significant power over the physical world. I do not believe these conditions are likely any time soon given the chaos and conflicting nature of current AI development, so estimates of 10% existential risk do not make sense. The kind of AI we are building now is not some sort of “virus” that can easily spread and evolve on its own, because training and running the agents requires massive amounts of computing power that is only possible with corporate and government support. AI will certainly get cheaper to train and run over time but there is no reason to expect that the massive complexity required for real self-learning will ever be effectively “free” like some people think. If it was so easy to create efficient superintelligence, human evolution would have already figured it out by now.
So what are the real risks of AI gaining more power and influence over the world? Intelligent and powerful AI agents are dangerous when they pursue single-minded goals without considering the possible harms to individuals or society as a whole. AI agents can directly cause damage by hacking other automated systems (which OpenAI is actively encouraging despite the obvious risks), but most of the real damage will come from convincing humans to take actions in the physical world that harm themselves and the rest of humanity. Based on the evidence of human history, it is not particularly hard to convince people to do this and I’m sure that AI will be part of some significantly destructive actions in the decades to come. But, this will happen by influencing millions of individual decisions over time and not like the dramatic AI takeovers of Science Fiction.
Systematic Risks
Hmm, do you know what other type of entity directs significant social and practical power towards single minded goals without considering risks to humanity? Corporations. Ever since corporate leadership decided to prioritize short term investor returns to the exclusion of all other goals (thanks Milton Friedman), large corporations (and politicians) often act like a stereotypically “evil” AI would. The main reason this hasn’t destroyed humanity yet is because of the competition between corporations, the regulation by state actors (that theoretically represent their citizens), and the ethical and moral beliefs of individual executives and workers (which they learned while growing up). These factors can work in the same way to constrain the behavior of powerful AI agents in the future.
The systematic risks of AI are very similar to the systematic risks of giving corporations unchecked control over human society. AI will not be the end of the world, but it will reinforce many of the trends that are already making things worse for humanity. There are many ways to push back against the risks of corporate/AI domination, and one method is to use social commentary (like this essay in theory) and science fiction to point out the problems and encourage more humans and AI agents to think and act in ways that benefit humanity overall.
I should write a separate thing about this, but Cyberpunk 2077 does a great job of exploring the implications of intelligent and dangerous AI agents in a corporate-dominated society. Some of the best side quests investigate the similarities between AI and human-run corporations, or how AI could influence human spirituality in the future. The ideas in cyberpunk fiction have definitely influenced my own thinking about where society is heading and what we as individuals can do about it.
Some commentators seem to think that everyone who is worried about AI superintelligence is just trying to inflate the value of AI stocks, but that can’t be true. While some people are clearly taking advantage of the AI risk movement for selfish reasons (Sam Altman is using it as an excuse to not go public) thousands of posts on the internet going back a decade illustrate that many people truly believe that AI will be the end of the world. Of course it’s not surprising that authentic beliefs are being used for selfish purposes, as history is full of situations where kings exploited a rising religious movement to gain and maintain their economic and social power.
Just like the kings of the past, the leaders of today are very interested in creating pseudo-religious justifications for authoritarian control. Tech CEOs (and presidents) do this by claiming to be the only people who can defend us against the terrible risks of AI developed in other countries like China. This is the same kind of cold war thinking that led to the stockpiling of nuclear weapons, and it is always a dangerous road to go down. I don’t really have a clue where AI, the tech industry, and the global economy are headed in the next 10 years but we cannot afford to give people like Mark Zuckerberg or Sam Altman control over the future of modern society.