I just finished listening to Life 3.0: Being Human in The Age of Artificial Intelligence, by Max Tegmark. The book argues that we badly underestimate the existential risk posed by the imminent onset of artificial, superhuman intelligence, and calls for a global and collaborative safety research program.
From the book:
“Not only can non-living matter have goals, at least in [the sense that it was designed to accomplish a goal], but it increasingly does. If you’d been observing Earth’s atoms since our planet formed, you’d have noticed three stages of goal-oriented behavior:
-
- All matter seemed focused on dissipation [of energy].
- Some of the matter came alive and instead focused on replication and subgoals of that.
- A rapidly growing fraction of matter was rearranged by living organisms to help accomplish their goals.
(…) not only do we now contain more matter than all other mammals except cows (which are so numerous because they serve our goals of consuming beef and dairy products), but the matter in our machines, roads, buildings and other engineering projects appears on track to soon overtake [in weight] all living matter on Earth.” (pp. 257–258)
So, if we overcome our anthropocentric bias and understand human intelligence as simply one of the many possible instantiations of intelligence in our Universe, then all it takes for superintelligent machines to be a serious concern is for them to have goals misaligned with our own, and we might end up being treated by those machines much like we treat ants today — we largely don’t care about whatever might be their goals
Importantly, those machines need not at all be rebellious — in fact, they don’t even need to be aware of what they’re doing, especially not subjectively so, or how that conflicts with the pursuits of other sentient beings.
That worries me.
How about you — what do you worry about?
___
Featured photo: a pig head at a market in Lviv ( January ’18 )
Wanna get short inspirational reads like this one straight into your inbox?
Sign up for my weekly newsletter!
‘ Trelograms’ is a wordplay between ‘telegram’ and ‘trélos’ (Greek for ‘mad’)