We Need to Talk about AI, the ‘Rationalists’ and ‘Longtermism’

As I’ve suggested in recent posts I’m finding it quite challenging to make sense of this roiling debate about AI safety. When I write about American politics, American history and a few other topics I have a body of knowledge that I draw on and try to share with readers — explain how it might illuminate present-day questions. I may be right or wrong. But I’m speaking from a background of some real knowledge and study. When I delve into questions like AI or AI dangers I’m seldom trying to come up with definitive answers. I lack the qualifications or foundational knowledge. What I’m trying to do is first try to educate myself, but in my writing I at least try to frame the basic questions, provide a framework for considering the issue, point to what are at least good faith and knowledgable voices if not necessarily people who are “right.” But while I’m still uncertain about dangers posed by the companies doing the most advanced and often reckless research on artificial intelligence I’ve been increasingly stuck on the prominence in the AI world of people who are in the overlapping communities of “effective altruism,” “rationalism” and “longtermism”.

Just to point to the latest examples, the Journal has a write-up on Jacob Coxon, the AI researcher whose Twitter thread quite literally triggered the current freakout/reckoning. It turns out his formative years were spent in the various parts of the “rationalist” subculture in London before he migrated to the U.S.

I’ve been loosely aware of the first and the second for a while, but didn’t have much pressing need to learn more because they didn’t seem connected to much that I care about. But they’re very big in the AI world and maybe even bigger in the AI will wipe out humanity worlds. So given the global freakout getting a handle on this becomes more pressing. Just as an aside, I will note that I don’t know how the dorks operating under the “rationalism” label secured exclusive branding rights to that term. Like, rough day for Descartes and quite a few other philosophers. But I digress. What I’m going to argue here is that at least the popularized versions of what I’ll call ERL (“effective altruism”, “rationalism” and “longtermism”) is made up of a mix of absurd and nonsensical ideas. The fact that so many AI doomers are part of that world gives me a lot of pause. But we can’t go from ERL is absurd to AI is fine.

So let’s get down to it.