Player is loading...
Q&A
Andre Freitas, Idiap Research Institute
Friday, March 10, 2023 · 9:45 a.m. · 12m 41s
Embed
Transcriptions
Note: this content has been automatically generated.
00:00:02
i have so i think you you mentioned there cannot come
00:00:08
out though and i will if you have an opinion on why
00:00:13
like this particular model that's a that's a first criticism from the public
00:00:18
even though it has all the study also have
00:00:21
the same kind of hallucination and thing kind of behaviour
00:00:25
so why one was so successful law the other one was suffered so much criticism
00:00:31
so i thank you uh so i think the essence of the problem is that uh
00:00:38
so since i give them a a requires that level for your uh in
00:00:42
the reasoning uh they should have a guarded that but i i think they
00:00:47
this was communicated the market that i'm not sure how it was precisely market but
00:00:51
but i think that there is some indication that as a kind of knowledge assistant
00:00:55
um reach a in this case these models can be very certain
00:01:01
uh with things with some through went but are not
00:01:04
factual in the in the scientific context this is fatal
00:01:08
uh so i think they should have maybe put a lot of warning signs there
00:01:13
and maybe not even commonly taken that this should be used in any form it's highly experimental right
00:01:20
so i think they're probably there was a disconnect between scientific team and the marketing team
00:01:25
uh but i'm not sure exactly how they communicate maybe it was just the public perception as well
00:01:31
uh but there are many domains that they're using as inspiration so by well which are super
00:01:37
sensitive towards this type of being able to write something operative with something that's not fact or
00:01:43
right uh yeah so i i think some of the big need is there are there are high so thanks for the dog
00:01:53
we don't think well like him question proteins um okay so
00:01:58
oh can you talk a bit about uh what do you think
00:02:01
personally on the limitations cancel our teacher and all
00:02:06
we can uh make progress well for the future
00:02:12
um well
00:02:17
i can talk about what i do which improving the
00:02:20
the architecture i think uh so um i think there's still
00:02:25
a lot to be done on this issue of grass
00:02:29
um he he they the input and output is sequences but
00:02:34
the internal latent structure it's some kind of soft graph and we
00:02:40
don't really know yet how to explain that to apply it to
00:02:45
grass like language that knowledge paths or things like that that um
00:02:51
it seems like a natural thing to do but it's it's hard
00:02:55
um there are lots of scale issues and uh so we don't really understand exactly how
00:03:02
to uh get knowledge in in get knowledge out of the transform so i think that's
00:03:09
major limitation particularly applications are lots of applications where you'd like to be able to say
00:03:16
talk about this knowledge base to my customers and and have everything it say
00:03:21
the factual but they can't do that to the i think these are the kinds
00:03:26
of directions that we really going to see some improvement in in the next few
00:03:31
years fronts versus warm up pants off on the remote from version two is also
00:03:42
her charm resort you mentioned term rampant over from swiss comment mark question was exact remote correction
00:03:48
i'm on channel four torture enough from attacks which we
00:03:52
built lucien roth mortals most other mortals improving and improving improving
00:03:59
exactly what concern um how come when you use to make sure remote
00:04:04
this strain of fort attacks can basically you know
00:04:08
regional or wherever normal started users not able to review
00:04:14
arms to switch taken out of context are basically harmful terms
00:04:18
i would like to your fork about your how do we learn form politicians spokespersons were
00:04:24
trained or move to not making statements taken
00:04:28
out of context are really really really bad
00:04:32
uh how do we believe you get from tutoring or your morals order to for more
00:04:38
so thank you uh quite complex question uh i'm not
00:04:41
sure i have a a concrete answer on that side
00:04:45
so what i can say is that uh probably the size of this
00:04:48
whole committee is focusing exactly on this type of uh how to make this
00:04:54
reasoning save for a there are a significant uh bring coming don't save
00:04:59
an l. p. um it's due uh i don't think is proportional to the
00:05:05
to the development of the models that that we have now but there is a growing interest on on this
00:05:11
uh it's so uh i think especially if you're thinking
00:05:15
about a a high and high value a high implications applications
00:05:20
there's more those are not uh yeah probably not so perfect for purpose or you need to guard them
00:05:27
yes i i think you seen chatty pity right i i think
00:05:30
they they have really some safeguards there in terms of live avoiding some
00:05:34
personal questions uh maybe negative topics so
00:05:38
they have safeguard broader safeguard but not grandmother
00:05:42
a safeguard right so i i think it's easier to implement broader uh safeguards
00:05:48
but not really grammar what you what like is that the more do essentially
00:05:52
in its architecture deliver that safety so uh i think it's evolving that direction
00:05:57
uh but uh i don't see anything do you want to broader safeguards in
00:06:02
in this issue designing applications uh in the context of use external to the models i think it's controlling
00:06:08
outside the model is she right so made a final point uh yeah thinking about the retrieval based more dues
00:06:16
where you have some level of yeah user oversight uh instead of purely generative
00:06:22
models so then you get more this sexuality you can control more the fact
00:06:27
relative so there are ways to circumvent this another james if you have anything
00:06:31
to add on that but i don't think it has things from that the talks
00:06:39
just one comment i'm on question one common used uh uh i was also playing with the child to be tedious everyone
00:06:46
um i found that uh uh interaction was more
00:06:51
much more closely difficult baton bronte nearing meaning uh
00:06:57
by along with it so was to go beep and beep
00:07:01
in a specific topic i spoke about a surrealism i wound amanda
00:07:07
and and it was quite nice as a discussion just a comment and a question perhaps
00:07:14
there are elements already in in in some of your presentations is what happened
00:07:18
if we have a sort of a sick of symbolic sequence which is not
00:07:23
human language for human made i mean in my case i work a lot of the genome x. uses
00:07:31
symbolic uses sequential we know there are a lot
00:07:35
of uh implications perhaps not the meaning but implications
00:07:39
but the structure is not responding to our natural way of doing
00:07:44
the question what could also be what happen if
00:07:46
we have a a corpus of uh extraterrestrial language
00:07:52
would be that able to capture all these uh nonhuman uh structures
00:08:00
that's an interesting thought so things are that question maybe i can
00:08:05
start a very briefly uh yes uh so if we need to
00:08:10
address for example specific an all natural language corpora when each of you
00:08:14
quite specific token eyes there's that have a lot of
00:08:16
assumptions on us understanding uh yeah what the symbols mean right
00:08:22
uh i'm just trying to uh to reflect your on this
00:08:26
so uh i think up to a certain point some of the
00:08:29
the characteristics of this more those could be grounded on categories
00:08:35
which are meaningful for does extraterrestrial language right uh but then is
00:08:41
so we're structuring the for example discourse based on very human cognitive assumptions right
00:08:48
do s. but there s. zero it's uh tell stories in the same way light or had arguments in the same way
00:08:55
uh but probably this could be for it if the corpus
00:08:58
use long enough uh and if we know basic assumptions about
00:09:03
about that language right so does it structuring documents in sentences
00:09:08
so maybe maybe that's a tentative often as we yeah yeah i i i mean if i
00:09:15
if i believe the transformers are cognitive model and
00:09:19
that they work so well on language because um
00:09:24
language is really has the same structural characteristics that are built into the interactive
00:09:30
bias it transforms so um i you know i'm i'm genome maybe it's also on
00:09:37
the other hand a very powerful very general model so applying it to gene sequences
00:09:43
maybe it could discover the proteins sequences they can discover how to do protein folding
00:09:49
but that's not a natural language at all but it can still discover
00:09:52
it so it's very general but i think the the biases are are really
00:09:59
the reason it's so successful is because language really is like that
00:10:05
that's my personal opinion we don't we don't see it a lot
00:10:11
more to to get the quest up just to as a follow
00:10:17
up for for the question about extractor extracts a terrorist real languages
00:10:22
uh you could form lies that in a way that if you take a any happy to read during machine
00:10:28
and you make you generate the sequence of the ones based on what the turing machine is when you don't know it
00:10:35
and you you need to huge corpus of such sequences and you train uh
00:10:41
g. p. t. model or late model on that it should be performing a
00:10:47
probabilistic approximation of the two mystic proves is that is in the to the machine
00:10:52
and uh and uh when i have the gap
00:10:57
is i somehow defeating that james has a point saying
00:11:00
that it's not what it which we machine it's probably doing machine that has to do something with language
00:11:06
do we have any hints on that no one should be to all the people try to really generate
00:11:13
deterministic sequences of text three looting the movies these mechanism and see what it was
00:11:20
yeah i i don't think it would work very well if it was just
00:11:23
a randomly generated turing machine because the
00:11:26
parameters of the transformer are not designed to
00:11:32
may make the parameters of the turn turn she they're very
00:11:35
different machines and often the machine in machine learning you change
00:11:40
even if it's the same kind of space of
00:11:42
of of formal uh systems if you change a parameterisation
00:11:47
you change what but the model how the model will general lies and therefore what it's able to learn
00:11:53
so i i really think it's it's not just
00:11:57
then arbitrarily caught a powerful machine which really got inducted bias that specific
00:12:04
and uh it is very very powerful but yeah the reason it generalises so astoundingly well
00:12:11
is because it has inducted biases separately for language and and therefore thought
00:12:19
so i think that that's a very good details work but just emphasising that
00:12:24
a lot of the behaviour this model is given by human feedback right so
00:12:27
i think i opt for their assumptions with with that feedback right so uh yeah
Conference Program
The Evolution of Large Language Models that led to ChatGPT (Andre Freitas, Idiap)
Andre Freitas, Idiap Research Institute
March 10, 2023 · 8:34 a.m.
664 views
Understanding Transformers
James Henderson, Idiap Research Institute
March 10, 2023 · 8:46 a.m.
369 views
Inference using Large Language Models (Andre Freitas, Idiap)
Andre Freitas, Idiap Research Institute
March 10, 2023 · 9:19 a.m.
ChatGPT for Digital Marketing
Floris Keijser, N98 Digital Marketing
March 10, 2023 · 9:58 a.m.
Biomedical Inference & Large Language Models
Oskar Wysocki, University of Manchester
March 10, 2023 · 10:19 a.m.
Abstract Reasoning
Marco Valentino, Idiap Research Institute
March 10, 2023 · 10:38 a.m.
120 views
The Risks Behind Large Language Models (Al Brown, Fujitsu)
Al Brown, Fujitsu
March 10, 2023 · 1:42 p.m.
Q&A: The Risks Behind Large Language Models (Al Brown, Fujitsu)
Al Brown, Fujitsu
March 10, 2023 · 2:01 p.m.
Round Table: Risks & Broader Societal Impact (Legal, Educational and Labor)
Lonneke van der Plas, Idiap Research Institute
March 10, 2023 · 2:07 p.m.
The Infrastructure to build Large Language Models (Vinay Pondenkandath, Cerebras Systems)
Vinay Pondenkandath, Cerebras Systems
March 10, 2023 · 3:12 p.m.
Q&A: The Infrastructure to build Large Language Models (Vinay Pondenkandath, Cerebras Systems)
Vinay Pondenkandath, Cerebras Systems
March 10, 2023 · 3:30 p.m.
Recommended talks
TensorFlow 1
Mihaela Rosca, Google
July 6, 2016 · 10 a.m.
2660 views