That mythology, in turn, has spurred a reactionary, perpetual spasm
from people who are horrified by what they hear. You'll have a figure
say, "The computers will take over the Earth, but that's a good thing,
because people had their chance and now we should give it to the
machines." Then you'll have other people say, "Oh, that's horrible, we
must stop these computers." Most recently, some of the most beloved and
respected figures in the tech and science world, including Stephen
Hawking and Elon Musk, have taken that position of: "Oh my God, these
things are an existential threat. They must be stopped."
In the past, all kinds of different figures have proposed that this
kind of thing will happen, using different terminology. Some of them
like the idea of the computers taking over, and some of them don't. What
I'd like to do here today is propose that the whole basis of the
conversation is itself askew, and confuses us, and does real harm to
society and to our skills as engineers and scientists.
A good starting point might be the latest round of anxiety about
artificial intelligence, which has been stoked by some figures who I
respect tremendously, including Stephen Hawking and Elon Musk. And the
reason it's an interesting starting point is that it's one entry point
into a knot of issues that can be understood in a lot of different ways,
but it might be the right entry point for the moment, because it's the
one that's resonating with people.
The usual sequence of thoughts you have here is something like:
"so-and-so," who's a well-respected expert, is concerned that the
machines will become smart, they'll take over, they'll destroy us,
something terrible will happen.
They're an existential threat, whatever
scary language there is. My feeling about that is it's a kind of a
non-optimal, silly way of expressing anxiety about where technology is
going. The particular thing about it that isn't optimal is the way it
talks about an end of human agency.
But it's a call for increased human agency, so in that sense maybe
it's functional, but I want to go little deeper in it by proposing that
the biggest threat of AI is probably the one that's due to AI not
actually existing, to the idea being a fraud, or at least such a poorly
constructed idea that it's phony. In other words, what I'm proposing is
that if AI was a real thing, then it probably would be less of a threat
to us than it is as a fake thing.
What do I mean by AI being a fake thing? That it adds a layer of
religious thinking to what otherwise should be a technical field. Now,
if we talk about the particular technical challenges that AI researchers
might be interested in, we end up with something that sounds a little
duller and makes a lot more sense.
For instance, we can talk about pattern classification. Can you get
programs that recognize faces, that sort of thing? And that's a field
where I've been active. I was the chief scientist of the company Google
bought that got them into that particular game some time ago. And I love
that stuff. It's a wonderful field, and it's been wonderfully useful.
But when you add to it this religious narrative that's a version of
the Frankenstein myth, where you say well, but these things are all
leading to a creation of life, and this life will be superior to us and
will be dangerous ... when you do all of that, you create a series of
negative consequences that undermine engineering practice, and also
undermine scientific method, and also undermine the economy.
The problem I see isn't so much with the particular techniques, which
I find fascinating and useful, and am very positive about, and should
be explored more and developed, but the mythology around them which is
destructive. I'm going to go through a couple of layers of how the
mythology does harm.
The most obvious one, which everyone in any related field can
understand, is that it creates this ripple every few years of what have
sometimes been called AI winters, where there's all this overpromising
that AIs will be about to do this or that. It might be to become fully
autonomous driving vehicles instead of only partially autonomous, or it
might be being able to fully have a conversation as opposed to only
having a useful
part of a conversation to help you interface with the device.
This kind of overpromise then leads to disappointment because it was
premature, and then that leads to reduced funding and startups crashing
and careers destroyed, and this happens periodically, and it's a shame.
It hurt a lot of careers. It has helped other careers, but that has been
kind of random; depending on where you fit in the phase of this process
as you're coming up. It's just immature and ridiculous, and I wish that
cycle could be shut down. And that's a widely shared criticism. I'm not
saying anything at all unusual.
Let's go to another layer of how it's dysfunctional. And this has to
do with just clarity of user interface, and then that turns into an
economic effect. People are social creatures. We want to be pleasant, we
want to get along. We've all spent many years as children learning how
to adjust ourselves so that we can get along in the world. If a program
tells you, well, this is how things are, this is who you are, this is
what you like, or this is what you should do, we have a tendency to
accept that.
Since our economy has shifted to what I call a surveillance economy,
but let's say an economy where algorithms guide people a lot, we have
this very odd situation where you have these algorithms that rely on big
data in order to figure out who you should date, who you should sleep
with, what music you should listen to, what books you should read, and
on and on and on. And people often accept that because there's no
empirical alternative to compare it to, there's no baseline. It's bad
personal science. It's bad self-understanding.
I'll give you a few examples of what I mean by that. Maybe I'll start
with Netflix. The thing about Netflix is that there isn't much on it.
There's a paucity of content on it. If you think of any particular movie
you might want to see, the chances are it's not available for
streaming, that is; that's what I'm talking about. And yet there's this
recommendation engine, and the recommendation engine has the effect of
serving as a cover to distract you from the fact that there's very
little available from it. And yet people accept it as being intelligent,
because a lot of what's available is perfectly fine.
The one thing I want to say about this is I'm not blaming Netflix for
doing anything bad, because the whole point of Netflix is to deliver
theatrical illusions to you, so this is just another layer of theatrical
illusion—more power to them. That's them being a good presenter. What's
a theater without a barker on the street? That's what it is, and that's
fine. But it does contribute, at a macro level, to this overall
atmosphere of accepting the algorithms as doing a lot more than they do.
In the case of Netflix, the recommendation engine is serving to
distract you from the fact that there's not much choice anyway.
There are other cases where the recommendation engine is not serving
that function, because there is a lot of choice, and yet there's still
no evidence that the recommendations are particularly good. There's no
way to compare them to an alternative, so you don't know what might have
been. If you want to put the work into it, you can play with that; you
can try to erase your history, or have multiple personas on a site to
compare them. That's the sort of thing I do, just to get a sense. I've
also had a chance to work on the algorithms themselves, on the back
side, and they're interesting, but they're vastly, vastly overrated.
I want to get to an even deeper problem, which is that there's no way
to tell where the border is between measurement and manipulation in
these systems. For instance, if the theory is that you're getting big
data by observing a lot of people who make choices, and then you're
doing correlations to make suggestions to yet more people, if the
preponderance of those people have grown up in the system and are
responding to whatever choices it gave them, there's not enough new data
coming into it for even the most ideal or intelligent recommendation
engine to do anything meaningful.
In other words, the only way for such a system to be legitimate would
be for it to have an observatory that could observe in peace, not being
sullied by its own recommendations. Otherwise, it simply turns into a
system that measures which manipulations work, as opposed to which ones
don't work, which is very different from a virginal and empirically
careful system that's trying to tell what recommendations would work had
it not intervened. That's a pretty clear thing. What's not clear is
where the boundary is.
If you ask: is a recommendation engine like Amazon more manipulative,
or more of a legitimate measurement device? There's no way to know. At
this point there's no way to know, because it's too universal. The same
thing can be said for any other big data system that recommends courses
of action to people, whether it's the Google ad business, or social
networks like Facebook deciding what you see, or any of the myriad of
dating apps. All of these things, there's no baseline, so we don't know
to what degree they're measurement versus manipulation.
Dating always has an element of manipulation; shopping always has an
element of manipulation; in a sense, a lot of the things that people use
these things for have always been a little manipulative. There's always
been a little bit of nonsense. And that's not necessarily a terrible
thing, or the end of the world.
But it's important to understand it if this is becoming the basis of
the whole economy and the whole civilization. If people are deciding
what books to read based on a momentum within the recommendation engine
that isn't going back to a virgin population, that hasn't been
manipulated, then the whole thing is spun out of control and doesn't
mean anything anymore. It's not so much a rise of evil as a rise of
nonsense. It's a mass incompetence, as opposed to Skynet from the
Terminator movies. That's what this type of AI turns into. But I'm going
to get back to that in a second.
To go yet another rung deeper, I'll revive an argument I've made
previously, which is that it turns into an economic problem. The easiest
entry point for understanding the link between the religious way of
confusing AI with an economic problem is through automatic language
translation. If somebody has heard me talk about that before, my
apologies for repeating myself, but it has been the most readily clear
example.
For three decades, the AI world was trying to create an ideal,
little, crystalline algorithm that could take two dictionaries for two
languages and turn out translations between them. Intellectually, this
had its origins particularly around MIT and Stanford. Back in the 50s,
because of Chomsky's work, there had been a notion of a very compact and
elegant core to language. It wasn't a bad hypothesis, it was a
legitimate, perfectly reasonable hypothesis to test. But over time, the
hypothesis failed because nobody could do it.
Finally, in the 1990s, researchers at IBM and elsewhere figured out
that the way to do it was with what we now call big data, where you get a
very large example set, which interestingly, we call a corpus—call it a
dead person. That's the term of art for these things. If you have
enough examples, you can correlate examples of real translations phrase
by phrase with new documents that need to be translated. You mash them
all up, and you end up with something that's readable. It's not perfect,
is not artful, it's not necessarily correct, but suddenly it's usable.
And you know what? It's fantastic. I love the idea that you can take
some memo, and instead of having to find a translator and wait for them
to do the work, you can just have something approximate right away,
because that's often all you need. That's a benefit to the world. I'm
happy it's been done. It's a great thing.
The thing that we have to notice though is that, because of the
mythology about AI, the services are presented as though they are these
mystical, magical personas. IBM makes a dramatic case that they've
created this entity that they call different things at different
times—Deep Blue and so forth. The consumer tech companies, we tend to
put a face in front of them, like a Cortana or a Siri. The problem with
that is that these are not freestanding services.
In other words, if you go back to some of the thought experiments
from philosophical debates about AI from the old days, there are lots of
experiments, like if you have some black box that can do something—it
can understand language—why wouldn't you call that a person? There are
many, many variations on these kinds of thought experiments, starting
with the Turing test, of course, through Mary the color scientist, and a
zillion other ones that have come up.
This is not one of those. What this is, is behind the curtain, is
literally millions of human translators who have to provide the
examples. The thing is, they didn't just provide one corpus once way
back. Instead, they're providing a new corpus every day, because the
world of references, current events, and slang does change every day. We
have to go and scrape examples from literally millions of translators,
unbeknownst to them, every single day, to help keep those services
working.
The problem here should be clear, but just let me state it
explicitly: we're not paying the people who are providing the examples
to the corpora—which is the plural of corpus—that we need in order to
make AI algorithms work. In order to create this illusion of a
freestanding autonomous artificial intelligent creature, we have to
ignore the contributions from all the people whose data we're grabbing
in order to make it work. That has a negative economic consequence.
This, to me, is where it becomes serious. Everything up to now, you
can say, "Well, look, if people want to have an algorithm tell them who
to date, is that any stupider than how we decided who to sleep with when
we were young, before the Internet was working?" Doubtful, because we
were pretty stupid back then. I doubt it could have that much negative
consequence.
This is all of a sudden a pretty big deal. If you talk to
translators, they're facing a predicament, which is very similar to some
of the other early victim populations, due to the particular way we
digitize things. It's similar to what's happened with recording
musicians, or investigative journalists—which is the one that bothers me
the most—or photographers. What they're seeing is a severe decline in
how much they're paid, what opportunities they have, their long-term
prospects. They're seeing certain opportunities for continuing,
particularly in real-time translation… but I should point out that's
going away soon too. We're going to have real-time translation on Skype
soon.
The thing is, they're still needed. There's an impulse, a correct
impulse, to be skeptical when somebody bemoans what's been lost because
of new technology. For the usual thought experiments that come up, a
common point of reference is the buggy whip: You might say, "Well, you
wouldn't want to preserve the buggy whip industry."
But translators are not buggy whips, because they're still needed for
the big data scheme to work. They're the opposite of a buggy whip.
What's happened here is that translators
haven't been made
obsolete. What's happened instead is that the structure through which we
receive the efforts of real people in order to make translations happen
has been optimized, but those people are still needed.
This pattern—of AI only working when there's what we call big data,
but then using big data in order to not pay large numbers of people who
are contributing—is a rising trend in our civilization, which is totally
non-sustainable. Big data systems are useful. There should be more and
more of them. If that's going to mean more and more people not being
paid for their actual contributions, then we have a problem.
The usual counterargument to that is that they are being paid in the
sense that they too benefit from all the free stuff and reduced-cost
stuff that comes out of the system. I don't buy that argument, because
you need formal economic benefit to have a civilization, not just
informal economic benefit. The difference between a slum and the city is
whether everybody gets by on day-to-day informal benefits or real
formal benefits.
The difference between formal and informal has to do with whether
it's strictly real-time or not. If you're living on informal benefits
and you're a musician, you have to play a gig every day. If you get
sick, or if you have a sick kid, or whatever and you can't do it,
suddenly you don't get paid that day. Everything's real-time. If we were
all perfect, immortal robots, that would be fine. As real people, we
can't do it, so informal benefits aren't enough. And that's precisely
why things, like employment, savings, real estate, and ownership of
property and all these things were invented—to acknowledge the truth of
the fragility of the human condition, and that's what made civilization.
If you talk about AI as a set of techniques, as a field of study in
mathematics or engineering, it brings benefits. If we talk about AI as a
mythology of creating a post-human species, it creates a series of
problems that I've just gone over, which include acceptance of bad user
interfaces, where you can't tell if you're being manipulated or not, and
everything is ambiguous. It creates incompetence, because you don't
know whether recommendations are coming from anything real or just
self-fulfilling prophecies from a manipulative system that spun off on
its own, and economic negativity, because you're gradually pulling
formal economic benefits away from the people who supply the data that
makes the scheme work.
For all those reasons, the mythology is the problem, not the
algorithms. To back up again, I've given two reasons why the mythology
of AI is stupid, even if the actual stuff is great. The first one is
that it results in periodic disappointments that cause damage to careers
and startups, and it's a ridiculous, seasonal disappointment and
devastation that we shouldn't be randomly imposing on people according
to when they happen to hit the cycle. That's the AI winter problem. The
second one is that it causes unnecessary negative benefits to society
for technologies that are useful and good. The mythology brings the
problems, not the technology.
Having said all that, let's address directly this problem of whether
AI is going to destroy civilization and people, and take over the planet
and everything. Here I want to suggest a simple thought experiment of
my own. There are so many technologies I could use for this, but just
for a random one, let's suppose somebody comes up with a way to 3-D
print a little assassination drone that can go buzz around and kill
somebody. Let's suppose that these are cheap to make.
I'm going to give you two scenarios. In one scenario, there's
suddenly a bunch of these, and some disaffected teenagers, or
terrorists, or whoever start making a bunch of them, and they go out and
start killing people randomly. There's so many of them that it's hard
to find all of them to shut it down, and there keep on being more and
more of them. That's one scenario; it's a pretty ugly scenario.
There's another one where there's so-called artificial intelligence,
some kind of big data scheme, that's doing exactly the same thing, that
is self-directed and taking over 3-D printers, and sending these things
off to kill people. The question is, does it make any difference which
it is?
The truth is that the part that causes the problem is the actuator.
It's the interface to physicality. It's the fact that there's this
little killer drone thing that's coming around. It's not so much whether
it's a bunch of teenagers or terrorists behind it or some AI, or even,
for that matter, if there's enough of them, it could just be an utterly
random process. The whole AI thing, in a sense, distracts us from what
the real problem would be. The AI component would be only ambiguously
there and of little importance.
This notion of attacking the problem on the level of some sort of
autonomy algorithm, instead of on the actuator level is totally
misdirected. This is where it becomes a policy issue. The sad fact is
that, as a society, we have to do something to not have little killer
drones proliferate. And maybe that problem will never take place anyway.
What we don't have to worry about is the AI algorithm running them,
because that's speculative. There isn't an AI algorithm that's good
enough to do that for the time being. An equivalent problem can come
about, whether or not the AI algorithm happens. In a sense, it's a
massive misdirection.
This idea that some lab somewhere is making these autonomous
algorithms that can take over the world is a way of avoiding the
profoundly uncomfortable political problem, which is that if there's
some actuator that can do harm, we have to figure out some way that
people don't do harm with it. There are about to be a whole bunch of
those. And that'll involve some kind of new societal structure that
isn't perfect anarchy. Nobody in the tech world wants to face that, so
we lose ourselves in these fantasies of AI. But if you could somehow
prevent AI from ever happening, it would have nothing to do with the
actual problem that we fear, and that's the sad thing, the difficult
thing we have to face.
I haven't gone through a whole litany of reasons that the mythology
of it AI does damage. There's a whole other problem area that has to do
with neuroscience, where if we pretend we understand things before we
do, we do damage to science, not just because we raise expectations and
then fail to meet them repeatedly, but because we confuse generations of
young scientists. Just to be absolutely clear, we don't know how most
kinds of thoughts are represented in the brain. We're starting to
understand a little bit about some narrow things. That doesn't mean we
never will, but we have to be honest about what we understand in the
present.
A retort to that caution is that there's some exponential increase in
our understanding, so we can predict that we'll understand everything
soon. To me, that's crazy, because we don't know what the goal is. We
don't know what the scale of achieving the goal would be... So to say,
"Well, just because I'm accelerating, I know I'll reach my goal soon,"
is absurd if you don't know the basic geography which you're traversing.
As impressive as your acceleration might be, reality can also be
impressive in the obstacles and the challenges it puts up. We just have
no idea.
This is something I've called, in the past, "premature mystery
reduction," and it's a reflection of poor scientific mental discipline.
You have to be able to accept what your ignorances are in order to do
good science. To reject your own ignorance just casts you into a silly
state where you're a lesser scientist. I don't see that so much in the
neuroscience field, but it comes from the computer world so much, and
the computer world is so influential because it has so much money and
influence that it does start to bleed over into all kinds of other
things. A great example is the Human Brain Project in Europe, which is a
lot of public money going into science that's very influenced by this
point of view, and it has upset some in the neuroscience community for
precisely the reason I described.
There is a social and psychological phenomenon that has been going on
for some decades now: A core of technically proficient,
digitally-minded people reject traditional religions and superstitions.
They set out to come up with a better, more scientific framework. But
then they re-create versions of those old religious superstitions! In
the technical world these superstitions are just as confusing and just
as damaging as before, and in similar ways.
To my mind, the mythology around AI is a re-creation of some of the
traditional ideas about religion, but applied to the technical world.
All of the damages are essentially mirror images of old damages that
religion has brought to science in the past.
There's an anticipation of a threshold, an end of days. This thing we
call artificial intelligence, or a new kind of personhood… If it were
to come into existence it would soon gain all power, supreme power, and
exceed people.
The notion of this particular threshold—which is sometimes called the
singularity, or super-intelligence, or all sorts of different terms in
different periods—is similar to divinity. Not all ideas about divinity,
but a certain kind of superstitious idea about divinity, that there's
this entity that will run the world, that maybe you can pray to, maybe
you can influence, but it runs the world, and you should be in terrified
awe of it.
That particular idea has been dysfunctional in human history. It's
dysfunctional now, in distorting our relationship to our technology.
It's been dysfunctional in the past in exactly the same way. Only the
words have changed.
In the history of organized religion, it's often been the case that
people have been disempowered precisely to serve what were perceived to
be the needs of some deity or another, where in fact what they were
doing was supporting an elite class that was the priesthood for that
deity.
That looks an awful lot like the new digital economy to me, where you
have (natural language) translators and everybody else who contributes
to the corpora that allow the data schemes to operate, contributing
mostly to the fortunes of whoever runs the top computers. The new elite
might say, "Well, but they're helping the AI, it's not us, they're
helping the AI." It reminds me of somebody saying, "Oh, build these
pyramids, it's in the service of this deity," but, on the ground, it's
in the service of an elite. It's an economic effect of the new idea. The
effect of the new religious idea of AI is a lot like the economic
effect of the old idea, religion.
There is an incredibly retrograde quality to the mythology of AI. I
know I said it already, but I just have to repeat that this is not a
criticism of the particular algorithms. To me, what would be ridiculous
is for somebody to say, "Oh, you mustn't study deep learning networks,"
or "you mustn't study theorem provers," or whatever technique you're
interested in. Those things are incredibly interesting and incredibly
useful. It's the mythology that we have to become more self-aware of.
This is analogous to saying that in traditional religion there was a
lot of extremely interesting thinking, and a lot of great art. And you
have to be able to kind of tease that apart and say this is the part
that's great, and this is the part that's self-defeating. We have to do
it exactly the same thing with AI now.
This is a hard topic to talk about, because the accepted vocabulary
undermines you at every turn. This is also similar to a problem
traditional religion. If I talk about AI, am I talking about the
particular technical work, or the mythology that influences how we
integrate that into our world, into our society? Well, the vocabulary
that we typically use doesn't give us an easy way to distinguish those
things. And it becomes very confusing.
If AI means this mythology of this new creature we're creating, then
it's just a stupid mess that's confusing everybody, and harming the
future of the economy. If what we're talking about is a set of
algorithms and actuators that we can improve and apply in useful ways,
then I'm very interested, and I'm very much a participant in the
community that's improving those things.
Unfortunately, the standard vocabulary that people use doesn't give
us a great way to distinguish those two entirely different items that
one might reference. I could try to coin some phrases, but for the
moment, I'll just say these are two entirely different things that
deserve to have entirely distinguishing vocabulary. Once again, this
vocabulary problem is entirely retrograde and entirely characteristic of
traditional religions.
Maybe it's worse today, because in the old days, at least we had the
distinction between, say, ethics and morality, where you could talk
about two similar things, where one was a little bit more engaged with
the mythology of religion, and one is a little less engaged. We don't
quite have that yet for our new technical world, and we certainly need
it.
Having said all this, I'll mention one other similarity, which is
that just because a mythology has a ridiculous quality that can
undermine people in many cases doesn't mean that the people who adhere
to it are necessarily unsympathetic or bad people. A lot of them are
great. In the religious world, there are lots of people I love. We have a
cool Pope now, there are a lot of cool rabbis in the world. A lot of
people in the religious world are just great, and I respect and like
them. That goes hand-in-hand with my feeling that some of the mythology
in big religion still leads us into trouble that we impose on ourselves
and don't need.
In the same way, if you think of the people who are the most
successful in the new economy in this digital world—I'm probably one of
them; it's been great to me—they're, in general, great. I like the
people who've done well in the cloud computer economy. They're cool. But
that doesn't detract from all of the things I just said.
That does create yet another layer of potential confusion and
differentiation that becomes tedious to state over and over again, but
it's important to say.
No comments:
Post a Comment