Player is loading...
Compatibility with humans: AI and the problem of control
Stuart russel, Professor of Computer Science and Smith-Zadeh Professor of Engineering, University of California, Honorary Fellow of Berkeley and Wadham College at Oxford
Saturday, Sept. 11, 2021 · 5:45 p.m. · 25m 27s
Embed
Transcriptions
Note: this content has been automatically generated.
00:00:00
okay so i'm i'm gonna talk about artificial intelligence and what it's for
00:00:07
i don and i couldn't begin with a very simple definition um one that's
00:00:15
on the enough in a i haven't i hold it the standard model so um
00:00:21
of course a eyes about making intelligent machines but what that
00:00:24
means it has meant the most to history the field is
00:00:28
um machines whose actions can be expected to achieve their objectives
00:00:33
and so this is what we do we um we developers optimising machinery uh whether it's um
00:00:42
turning algorithms or reinforced with any other than zero supervisory no buttons
00:00:46
and then we um like in the objective and uh said she going to
00:00:52
solve the problem for us there's lots of different branches of a on my
00:00:58
um but uh the standard model applies to you all of these and in fact in many other
00:01:05
disciplines like control theory of statistics operations research
00:01:10
and economics of them will operate on the same standard model
00:01:15
of uh optimising some fixed specified objective
00:01:22
uh and one of the characteristics of their research and
00:01:25
he is is that the the goal no always explicit
00:01:29
but certainly implicit in what we do when um is that we would like to g. general purpose they are
00:01:36
um which would mean roughly a machines are
00:01:40
capable of quickly learning to produce high quality behaviour
00:01:45
in any task environment and any here when means certainly any task environment which humans
00:01:52
and perform well um but terribly many other task environs uh as well
00:01:57
where are a physical or a comedy limitations prevent us from before well
00:02:05
uh and the question that i really want to address today's what if we sixty and
00:02:10
that seems like a reasonable ask given that we are all trying to do this um
00:02:17
and uh you know one one vision of success would be
00:02:21
that that we in general but to say i mean it's abuse
00:02:25
uh use that capability extended with the physical appendages that robotics could use is
00:02:31
um to do what we already know how to do we already know how to build houses and
00:02:38
a line water pipes uh and so on so forth and so
00:02:42
if we can uh simply have a high systems carry out all the complicated process isn't bowl
00:02:49
uh which apparently very expensive we can use that to lift the living standards of everyone on earth
00:02:56
to a respectable level um without yeah additional science fiction inventions of
00:03:03
uh you know eternal life and and possible like travel and so well
00:03:08
and just doing that just giving everyone a respectable middle class a
00:03:14
while but you know in the west the cool respectable middle class
00:03:18
stanton living uh of the foundations a goal of quality of life
00:03:23
would be a tenfold increase in the g. d. p. the well and if you translate
00:03:29
that into the net present value uh would be about thirty and a half or trillion dollars
00:03:36
so that's one sort of ballpark estimate of
00:03:40
uh the size of the price that a a i is working towards
00:03:45
and there were also the additional things we might be able to do
00:03:50
uh i don't currently condom like raise the quality of yeah uh uh
00:03:55
give everybody enough an extremely high quality individualised personalised to to kind of education
00:04:01
um and uh improve the rate of scientific research
00:04:05
progress but oh no if you was to i'm sure
00:04:11
what if we succeed um this is what he said in nineteen fifty one
00:04:16
it seems probable once a machine thinking that that had started it would not take long to outstrip of people cowards
00:04:22
at some stage therefore we should have to expect the machines to take control
00:04:28
um so he says this with the with
00:04:31
gnome solution no mitigation uh just a resignation
00:04:38
um so these are two very different visions and uh obviously the con really coexist
00:04:46
um you know if we move forward in time from during united
00:04:52
but you want to present uh we're seeing some of the capabilities that will once dreamed
00:04:58
of i'm starting to become reality link uh the salt driving car and i will go champion
00:05:06
ah this is the example for my group be uh the monitoring system
00:05:11
what a nuclear test ban treaty is now a large
00:05:15
a probability based a. i. system this is a picture from
00:05:19
uh north korea showing the you know instantaneous and accurate
00:05:24
detection of a new to explosion that took place twenty thirty
00:05:30
um we're of course finding ways to use a i for evil
00:05:34
not rather than a i for good um were the most worrying developments
00:05:38
is the use of a i can kill people um and we're already seeing um
00:05:45
uh events for example libya last year the drone on the left a
00:05:50
cop who from um from uh s. t. m. which is the turkish
00:05:56
weapons company uh was used to uh to attack humans autonomously
00:06:01
so not remotely piloted um but operating which was the um
00:06:07
and uh in geneva uh in the then the the
00:06:12
um the member states of the conventions that commission weapons will be once again
00:06:17
uh to make no progress on the treaty banning these but
00:06:23
so with all those progress people on our uh uh perhaps taking
00:06:29
seriously the possibility of success agreed in general but to say hi
00:06:33
um but as melanie mitchell's tool pointed out uh we're not there yet
00:06:39
um and in fact i i agree with no these point that we have further away than many people think
00:06:45
um but when you look back over the longer time scale progress over
00:06:50
the last seventy years b. c. series a very important breakthroughs a happening
00:06:56
um and get learning is just the latest in a long series
00:07:00
of breakthroughs we need more breakthroughs but i think we have to assume
00:07:06
that those breakthroughs are going to occur and we will have a i systems are what make better decisions
00:07:13
that humans across the broad range of real world scenarios which humans all
00:07:20
and ensuring is asking us uh if that happens um then when creating
00:07:26
machines that are more powerful than human beings 'cause it's our ability to make good decisions intelligent
00:07:32
decisions in the real world gives us how over the manager we have this disease but it
00:07:38
um and so we creation machines that are more powerful than us
00:07:43
how do we retain power over entities remote awfulness rather
00:07:50
um and sure enough he sees no solution to this problem
00:07:54
um i actually think there is a solution but it means we can socialising artificial intelligence
00:08:00
from the ground up so to see an example of how things go wrong we can look at
00:08:06
um what's happening in social media so when you specify injected light
00:08:13
maximising click through the probability that the user clicks on the next station that is recommended
00:08:19
we could also maximising p. h. mm mm of various other kinds of a proxy metrics um
00:08:29
you might think well the learning algorithm is going to learn what
00:08:32
is people want to click on and understand things that they're interested in
00:08:37
um and that seems like a good idea but in fact this is not what happens the
00:08:41
algorithms don't just no no people want because that's not the best way to maximise quite true
00:08:48
if your reinforcement learning algorithm um what you learn is
00:08:52
a sequence of actions a policy that will maximise longterm reward
00:08:59
and uh the policy does that are changing is take a while ago program
00:09:05
it changes the go what by adding pieces to it um with the part with the the goal of winning the game in the long run
00:09:13
if your recommendation over them um then you change the
00:09:17
board namely the brain of human but you're interacting with
00:09:21
uh with the goal of winning in the long run so you stand a sequence of content
00:09:26
that changes the person right into a different person
00:09:29
who's more predictable and if the person will electable
00:09:34
then you can get a higher click through rate from them by sending them of the stuff that you know the good click on
00:09:41
and so this is simply the solution to the optimisation problem on
00:09:45
that was specified by that's so should media platforms to maximise spectrum
00:09:51
oh this is probably not the solution they want and so they're not the
00:09:54
solution we want but this is the solution to the problem that was specified
00:10:00
um and it's creating a uh i would argue a pretty catastrophic situation uh in the well
00:10:07
and this is just one example of what goes wrong with miss specified
00:10:12
objectives that um as they are it gets better the outcome people gets worse
00:10:20
um because their assistant is um we were able to make a mess the rest of 'em through
00:10:27
uh through its ability to make decisions in the world optimise the incorrect objective
00:10:33
that we've given it uh it will also the more able to prevent us from interfering
00:10:39
with its action so certainly the um the learning algorithms that operate in social me deep levels
00:10:46
don't have the ability to prevent interference um but the
00:10:50
corporations that protect them certainly do have the ability to
00:10:54
prevent interference with the operation of the oh so if it's a if it's the case that
00:11:02
a a i systems the ceiling incorrectly specified objectives could lead to catastrophic outcomes
00:11:09
um then that's suggest that there's something fundamentally wrong with the standard model because it
00:11:15
requires us to specify objectives completely incorrectly and
00:11:20
coming back to the purpose of the temptation
00:11:23
uh the quality of life i i'd be willing to bet that the foundation does not think
00:11:30
that it knows exactly how to define quality of
00:11:33
life that we could take any future sequence of
00:11:37
states of the will and write them uh according to which one has a higher quality of life
00:11:43
oh and do so using it explicitly written down definition of quality of life
00:11:48
um i don't think the foundation things that uh something uh governments
00:11:52
think net um to some extent me no but when we see it
00:11:56
um so if it's not if we experience something that we haven't thought of
00:12:01
um either as particularly desirable weekly undesirable but we experience it
00:12:06
be realistically it is um then we can sort of tell
00:12:11
but uh our duty to write down the bonds completely incorrect specification
00:12:17
of the quality of life or any other objective in the real world
00:12:21
uh uh is that that's really an impossible task so the new model that
00:12:28
um i'm going to try to convince use is the right way to think
00:12:32
about a guy is based on replacing the definition that i gave you beginning
00:12:38
that machines are intelligent extent that their actions can be expected to achieve
00:12:42
their objectives you replace that we have a a just a small change
00:12:48
um machines uh beneficial to the extent that their actions can be expected to achieve
00:12:52
our objectives and so this is the key to change from their objectives to our objectives
00:13:00
um and of course is the more difficult problem because our objectives are
00:13:04
in us and in particular were unable to explication exactly what they are
00:13:09
um but this is a much more robust formulation of the problem that we try to
00:13:15
stall because it doesn't require that we extricate our objectives and plug them into the machine
00:13:21
which then assumes that those objectives are exactly correct um so i've formulated
00:13:29
this uh in deference to isaac asimov m. p. three principles unless some overlap
00:13:35
with icicles but also some uh some very what difference
00:13:40
um so the first call uh is that a first principles that the robots call
00:13:46
is to satisfy can move preferences um and trust is
00:13:51
here uh i'm using it in the same sense that
00:13:54
uh economists use it which is which
00:13:57
is really um preferences over lotteries of probability
00:14:01
distribution over entire futures of the world so everything that you could possibly care about
00:14:08
um i am not just o'connor pizza do i like that entire features
00:14:14
um then this one principal and this is the key point is that the robot does not know what
00:14:21
those preferences are and this uncertainty about the objective uh turns out to be central to
00:14:29
uh enabling us to retain control of the machines uh the principal
00:14:36
basically says uh okay so if the rubber doesn't know references are
00:14:40
working than where is the grounding here the grammy yeah it's o.
00:14:45
e. in human behaviour that that um our uh our on line preferences
00:14:52
um generate a behaviour and therefore our behaviour provides evidence of what
00:14:58
goes on my preferences ah it's not it's a complication in project process
00:15:03
uh that produces behaviour and therefore there's
00:15:06
no straightforward mapping from a behaviour fact preferences
00:15:10
um but nonetheless that is the source of evidence of working preferences ah
00:15:16
at least to to have better direct understanding right um so you can take these three principles and formulate
00:15:23
oh oh oh a mathematical definition of the problem that a.
00:15:27
i. system is trying to solve which we call assistance game
00:15:31
so it's a game because there's at least two participants at least one robot at least one him
00:15:36
and uh its insistence came because the robots goal is to satisfy
00:15:42
used basically to maximise containment payoff in the language of game theory
00:15:46
i'll laugh robot does not know what that payoff function it's when you write down those games and soul
00:15:55
um when you can look at this we shall see what's the nash equilibrium
00:15:58
what is the uh the way that we watch a a dissatisfied three principles
00:16:04
and and you see they gave us that don't occur in the castle model where the robot lose the payoff
00:16:11
um for example a robot solving assistance games will be for to humans
00:16:19
um who are the ones you actually have to pay off
00:16:22
and no it um the people will ask permission before turning out
00:16:28
uh an action that would change possible role uh who's down to robot is not sure
00:16:35
so for example if a robot is to is to ask we're
00:16:41
i'm fixing that problem dioxide concentrations so taking it back to korea industrial levels to the atmosphere
00:16:48
um if it comes up with a solution that involves turning the oceans
00:16:51
into soccer aggressive uh and we haven't folded up references about the oceans
00:16:57
um then rather than just caring now the plan of what you would
00:17:02
do if they believed that the objective was to still got lots levels
00:17:06
uh it would ask permission was that is a good idea for me
00:17:10
to change the options into software i guess it a while solving the problem
00:17:15
and we would say no army no let's not do that on something else um and in the string case
00:17:23
ah if um if the robot uh might be doing something that
00:17:28
we really don't like um it will allow itself to be switched off
00:17:33
so this is sort of the extreme case of asking conviction uh it's in a timely
00:17:38
happy for us could just switch it off because it doesn't want to do whatever it is
00:17:43
uh that would cause us to switch it off but it because it doesn't know what that is so it always allows us all
00:17:50
we switched off and that's the offices of the classical machine
00:17:54
that's 'cause you're in an objective it needs to be completed correct
00:17:58
uh which would never allow us all to be switched off because that would guarantee failure achieving dejected um so
00:18:07
oh oh oh at least in civil cases we can even prove that you're
00:18:11
right so that the raising of the level to the decision of the humans
00:18:17
to be all this kind of mushy mushy that's all systems things
00:18:21
we can show that rational for humans to build and deploy she's
00:18:26
uh that's all systems case but this is a sensible thing for
00:18:29
us to do um and the nice property you the small ones that
00:18:36
as you improve it a on a um you're actually getting better rather than worse out
00:18:42
'cause machine is gonna be better at learning and understanding of preferences i think
00:18:48
caring enough actions that help us to cheat us you're thinking of all kinds of
00:18:56
complications and difficulties uh and the problems installing jester
00:19:01
the last two lots of complications so um they include
00:19:04
ooh the fact that there are many humans and machines make
00:19:09
laughter may humans is in the same situation as a as a
00:19:13
as a government or in some some extent uh as a human being
00:19:17
um and that means to many of the problems are more velocity and economic
00:19:22
so um this is not a new problem but perhaps a bus to get
00:19:30
to know us all the a. i. setting uh will help us to make
00:19:34
progress on on figuring out how to trade off the p. r. for instance
00:19:39
but many people um it's only and not suggesting as many people seem
00:19:43
to think that there's one set of human preferences or one set of values
00:19:47
there are a billion humans and that means they're really that's a few
00:19:51
preferences um and of course it or i think ah it's also the case
00:19:58
that they can be many machines are involved in this assistance again uh they
00:20:03
want will necessarily be designed by the same company you wear hats in software
00:20:09
yeah and so you also need to figure out how not to have a strategic
00:20:14
interactions among the machines equipped for princess di
00:20:18
them like a failure mode the their interactions
00:20:23
um i understand preferences of of humans involved
00:20:26
actually he doing with them and that we're emotional
00:20:38
a fact that with my okay uh uh and also the very for me the fact that we are
00:20:44
change your preferences on valuable uh obviously not born with complicated preferences about the
00:20:50
future oh oh mm and the fury that uh that deal satisfactory if human beings
00:21:04
preferences can be changed by the actions of the mushy um and then
00:21:10
if it does approach turns out to be right then that means that because
00:21:14
every every year they i has a at at its
00:21:17
foundation the idea that the objective is fixed and known
00:21:22
uh whether it's in search there's a cost function like goal
00:21:25
it's in the in for some learning but uh that's what function
00:21:29
um so we see areas have x. node objectives uh and therefore since that assumption is not correct
00:21:37
uh we have to be the only into these areas one of what the foundation
00:21:42
um where both technology the objective is just a screen special case it on and then
00:21:48
i think to get the rest of the will to fly into this uh this new approach
00:21:53
uh we need to start of figuring out how to develop real
00:21:58
uh applications that bodies points so just summarise i
00:22:02
think a. i has a enormous potential for good
00:22:07
um enormous economic value and uh and that leads
00:22:10
to i'm say unstoppable momentum um and some people
00:22:16
i think that we can um avoid risks simply by stopping a ah
00:22:22
uh i think that's very unlikely um so for the time being i
00:22:27
would like to try that by the way i away from the standard model
00:22:31
um to once a form in which a i would really beneficial
00:22:36
the humans even though we don't know what that right now there are um
00:22:44
there are those who are uh uh we want to create a
00:22:49
and you feel the a. i. f. x. i don't wanna discourage them
00:22:52
from at um but i would say that the model in which uh
00:22:57
yeah this is who are sort of wagging their fingers yeah researches
00:23:02
a thing bad bad bad is less effective than a model in
00:23:07
which a. i. researchers get up in the morning and and what they
00:23:11
do uh as a i researches is necessarily beneficial to human beings um
00:23:20
and we really talk about changing well what chance as good a okay
00:23:26
as high quality yeah researcher michael t. i. systems development um
00:23:32
so that it's it it will be good for human beings and um you know there
00:23:37
are lots of things that uh that doctors
00:23:40
and medical professionals and pharmaceutical companies could you um
00:23:46
that would be harmful and some we'd seen some pharmaceutical companies do things that are harmful to human beings in recent years
00:23:53
um but i think but i i'm not when when are
00:23:57
medical researcher gets up in the morning and says okay reduces many good medical research
00:24:01
ah it's going to be good for human beings that that was the successful and uh
00:24:07
and that should be the case but they are just what counts as good a hour
00:24:11
is a i that's beneficial e. other other problems that ah i've not
00:24:18
talked about today and one is misuse and we just all previous two examples
00:24:24
of misuse if they are right now to generate it falls uh
00:24:28
information and you got to break a security systems and so on
00:24:33
um and uh i don't have a solution uh that problem and then over use
00:24:40
uh meaning that if we have a if we do have a either pays itself and does everything
00:24:46
um for us uh we have our own um sort of social and cultural problem
00:24:52
how do we retain the the intellectual bigger if human civilised nation
00:24:58
uh when it's possible to simply leave the running it's a playstation machines
00:25:03
um and this is addressed in uh in many works of fiction will me
00:25:09
this picture here being one of them uh the machine stops which i highly recommend
00:25:14
story right impostors another one um and the i who we can
Conference Program
Welcome words
Aurélie Rosemberg, Fondation Dalle Molle
Sept. 11, 2021 · 4 p.m.
Artificial intelligence and quality of life
H. Bourlard, Idiap Research Institute
Sept. 11, 2021 · 4:30 p.m.
Artificial intelligence to think like humans
Melanie Mitchell, Professor at the Santa Fe Institute
Sept. 11, 2021 · 4:45 p.m.
Towards human-centered robotics
Sylvain Calinon, Research Director at the Idiap Research Institute
Sept. 11, 2021 · 5 p.m.
Supporting sustainable transitions around the world through water technology
Eric Valette, Director of AQUA4D
Sept. 11, 2021 · 5:15 p.m.
Biometric security
Sébastien Marcel, Research Director at the Idiap Research Institute
Sept. 11, 2021 · 5:30 p.m.
Compatibility with humans: AI and the problem of control
Stuart russel, Professor of Computer Science and Smith-Zadeh Professor of Engineering, University of California, Honorary Fellow of Berkeley and Wadham College at Oxford
Sept. 11, 2021 · 5:45 p.m.
Model subjectivity at the heart of consciousness to make robots more human
David Rudrauf, Associate professor at the University of Geneva, Director of the laboratory of the multimodal modeling of Emotion and Feeling
Sept. 11, 2021 · 6 p.m.
Round table
Panel
Sept. 11, 2021 · 6:15 p.m.
Recommended talks
Compatibilité avec l’humain : l’IA et le problème du contrôle
Stuart Russel, Professeur en informatique et Professeur de Smith-Zadeh en ingénierie, Université de Californie, membre honoraire de Berkeley et du Wadham Collège à Oxford -
Sept. 11, 2021 · 5:45 p.m.
Les enjeux RH et managériaux de la diversité culturelle pour une entreprise internationale en Suisse (partie 2)
Astrud Steuernagel et Blandine Dupray, Procter & Gamble
Sept. 15, 2011 · 2:09 p.m.
181 views