Player is loading...
Clinical Natural Language Processing Q&A
Carsten Eickhoff, Assistant professor of medical and computer science at Brown University
Tuesday, Aug. 20, 2019 · noon · 15m 06s
Embed
Transcriptions
Note: this content has been automatically generated.
00:00:02
yeah
00:00:05
yeah
00:00:10
yeah
00:00:12
that's right
00:00:16
yep yep um so so for the record the question was what how much information you need how many
00:00:23
episodes d. in each right in in order to reliably
00:00:25
classified and so it does get easier over time mostly
00:00:30
um we have interesting uh episodes where actually the physicians running completely off
00:00:35
on a on a we're trajectory test for all kinds of weird stuff
00:00:38
uh well actually mislead the system as well just because you have all kinds of strange verbiage that doesn't lead anywhere
00:00:44
any uh would put you should be doing for this patient and but in general i you get better over time
00:00:50
and you have a diminishing returns kind of curve on ever tried so typically
00:00:53
what you want to have is a two to three notes preparation so it
00:00:58
it's hard to translate that into how much time needs to parse just because some people will reach
00:01:03
receive three nodes within a day right because you go to hospital urine test one test tutor straight
00:01:08
and sometimes some weak pulse right so um but it's it's really i think on average something between two and three
00:01:14
nodes will get you who eighty five percent of the way that you'll ever get even if you have forty fifty miles
00:01:20
but it's very much to i mean you you want this um this filtering effect but men having many times that's
00:01:33
yeah
00:01:37
yeah
00:01:54
i
00:02:03
uh_huh
00:02:08
yeah um so we have absolutely true so that intuition so should we and that more medical knowledge into
00:02:14
this process right so right now we're completely agnostic of anything we really only say on his point of text
00:02:19
no we we split up into queries into some matching and now and eventually we have some ranking now
00:02:24
um we clearly see cases where uh we recommend cervical
00:02:29
cancer as a potential cause and the patient is male
00:02:33
so i'm not idea clearly right so have you had ah some
00:02:37
uh engine that could have told you that well certain things only
00:02:39
apply to elderly patients to female patients to male patients to what
00:02:42
not right you could clearly ah i have improved their now um
00:02:47
well we we don't necessarily need the the doctor to do any diagnostics right so if they only describe patient has
00:02:53
the following sometimes that's already enough to start with right so they don't need to come up with hypotheses that we then
00:02:58
evaluate and confirm or deny whatever um because we'll have that in the paper so if
00:03:03
you remember this this pipeline right we start out with whatever description of the patient we have
00:03:07
we go to the papers in the papers will always say oh by the way i'm talking about this disease of these three diseases and what not right
00:03:13
um but clearly and this is something that we uh we're looking
00:03:16
into at the moment so using all these medical ontologies to give you
00:03:20
some form of structured reasoning all the um over these uh these conditions were absolutely yeah
00:03:33
i
00:03:39
oh
00:03:43
oh
00:03:45
that's that's right yeah so we
00:03:51
this is a big problem that we haven't addressed yet so the the way that we
00:03:55
deal with that in the moment at the moment is that we produce a ranked list so
00:03:59
off by merit of that you would sort of get to the idea of all something
00:04:03
is going to bring five and you maybe have the other thing i bring seven and hopefully
00:04:07
you still get to that but clearly you would want to have the product of the tool right
00:04:11
which which makes it extremely hard to do so we don't have a good solution for that right now
00:04:15
yeah yeah so so clearly this is one of the of the really big limitations of this at the moment just because then
00:04:21
the frequency of these things becomes nearly zero for for
00:04:24
virtually anything right so even to separately common conditions um
00:04:29
it's it's often even these two conditions plus some all the t. in the in the patient right that only that to get
00:04:35
the mixed severe right so because you also diabetic which is sort of the third disease in a way that plays into the mix
00:04:40
now everything becomes league lethal it looks very different from you having the common cold at this other thing um
00:04:47
so it's especially so these rare diseases i hit especially hard in patients with
00:04:52
a lot of probabilities right so if you have a lot of stuff already
00:04:55
um so you a bit of a strange patient that we may not as physicians know much about
00:04:59
uh and then you are being diagnosed so you catch some weird rare tropical bargain now anything can happen right sides should problem yeah
00:05:09
yes
00:05:12
uh_huh
00:05:16
oh
00:05:18
that's right
00:05:24
that's right
00:05:29
okay
00:05:33
um so in this case i think this is the main one that
00:05:36
we've been looking at at the moment right um to speed up this this
00:05:40
preparation for the next visit because often people will see many different doctors right so
00:05:45
um on the clinical side we work with the v. a. r. and the us which is the veterans administration
00:05:50
so uh people served in the military uh will often
00:05:54
get free or at least subsides health health care over there
00:05:57
um so that means that they're very loyal because they could go to any doctor and actually pay
00:06:00
or they go to the v. a. is dedicated doctor and they get for cheaper or free so um
00:06:07
for socio economic reasons um many veterans um once
00:06:12
in a while get on those have drug problems
00:06:14
and whatsoever right so the the few of these conditions that are much more problem um to experiencing
00:06:19
so there's a lot of stuff where they're events that may have happened to the patient since i've
00:06:25
seen them last right so in common ones is
00:06:27
the patient had surgery it now got this medication um
00:06:32
it's off this medication and got homeless and an egg in the last six months since i last saw them right so in all of these things are really key
00:06:38
facts that the position once tunnel and otherwise they really have to sift through all
00:06:42
of these records in order to just find out these day these few bits of information
00:06:47
the problem generalises too many things so you ideally you would
00:06:49
want to do summarisation from tables from timelines right from from charts
00:06:54
from ah i. h. o. uh from um from sorry from a radio graffiti output right um
00:07:00
because often even you can't read this right if you see a just c. t.
00:07:04
scan as a as a g. p. e. probably you can't do anything with that
00:07:07
right so i think we are by far not there yet by doing just text
00:07:12
i just because there are many other media that you want want want to
00:07:15
have but this idea of giving you the hopefully article on summary of of
00:07:19
well if we were perfect that is all you need um i think that's that's key and speeding is up
00:07:25
and hopefully does that means that uh we speeded up in such a
00:07:28
way that the physician actually has the time to see the patient right now
00:07:32
too and so this this is the sort of the consuming and so give me all the stuff that we know and make this model just double
00:07:37
for myself and then just clerical and where all the time that i
00:07:40
need to write down there they would have to text it is it's things
00:07:43
like that where um there are bunch of startups and apps that do this
00:07:47
now where you put your phone into the room when you see the doctor
00:07:51
and that thing will record what's being said and they'll write the summary either for the doctor or for the patient
00:07:57
right so the patient could receive a now remember the doctor actually said avoid this food
00:08:04
take this food and take this medication right so and this is something that the there are many studies
00:08:08
that patients in a normal one on one consultation with the doctor they remember a thirty percent of the things that we're
00:08:14
being told them right so they often don't know the name of some condition that's being tested for us alright so often
00:08:20
they really did leave the room and a lot of the information is gone just because it's very technical jargon um so
00:08:26
and then on the other hand you could hopefully make summaries that actually generate the h. or for the for the position right
00:08:38
uh_huh
00:08:41
yeah
00:08:50
that's right yeah
00:08:58
so the construction of the set of diseases that we're considering for the baseline
00:09:05
oh uh_huh so it's basically yeah um mm so the idea is
00:09:11
basically that we have these these ranked lists of conditions right so all
00:09:15
uh our method produces a ranked list of size what forty thousand or something
00:09:20
are we um and these baseline so here so this guy here that we build will
00:09:25
rank any possible disease that the literature ever talks about right so there's really on this
00:09:30
and we measure where in this list is the true eventually by the
00:09:33
physicians confirmed diagnosis right and that computes the score now these guys here
00:09:39
have different levels of advantage right so this guy he uh says i'll you list only needs to be twenty long
00:09:46
i already gave you the eight that out right so right so i i make sure they always in this pool
00:09:52
and the rest are um are the most frequent um diseases right so i would here have the
00:09:58
twelve most frequent things plus the eight things that i know perfectly covered the ground truth my data set
00:10:03
right so let's assume that just by shot by johns these dating site in there right because otherwise if you would only do frequency base
00:10:09
probably this one would be the only one that has a one zero score just because by frequency would never catch that there s. stuff
00:10:16
um so we give this head start right and then we have a list of result length a a list length
00:10:21
twenty or fifty or a hundred so every disease that i tell this method okay now consider the stuff they rank
00:10:27
right and then again somewhere in the in the list i know all the correct diagnosis we will find it um and then we can interesting
00:10:36
yeah
00:10:40
that's right uh_huh
00:10:43
yeah
00:10:46
but as the physician has to do this right so we don't even go there so since is all retrospective
00:10:51
here in this case we would really only say only taking
00:10:54
whatever exists right how good would we be at edge presenting
00:10:58
this information to position and hopefully if it's higher up in this ranking this would catch more of the physicians attention it's
00:11:15
uh so we are we actually have the certified as a medical device or so this is actually something that
00:11:20
can so so basically based on these studies basically right so so
00:11:24
performance uh drinking these uh these things as a diagnostic decision support system
00:11:29
was actually enough to get by the the c. so that your p. and i. c. e. right so they have to his cousin over here
00:11:34
um so swiss medic assisting registered as a product now out so that was enough for them but i think what's
00:11:39
them much more exciting part and this is something that will hopefully soon going to get into is this idea off
00:11:46
could i into feel great feedback into the physicians process start because right
00:11:49
now i really only passively i listen and once in a while i
00:11:52
show something and hopefully they would do something with this but much more
00:11:55
interesting b. if i have such a ranking ah could i suggest the test
00:12:00
so i see there's no obvious my ranking but if you could run this one blood panel for me
00:12:04
i could maximally reduce my uncertainty interesting and much more clearly tell you whether it's
00:12:09
this thing or that thing right so right now we don't do any of that
00:12:12
um but it would be big if the moment you do that you a whole different risk clauses medical device so
00:12:18
the moment you actually want to do that in more than just an experiment setting um uh this this hell to pay
00:12:24
right but i but clearly this is where we need to go with this uh just because
00:12:28
this is sort of only the first step direct but that you are very very good thought
00:12:36
yes
00:12:40
i'm sorry say that again
00:12:48
ah so this is just an l. d. a. model so so we just try and
00:12:51
um a standard topic model a completely outside of the neural network and put that in now
00:12:57
right so so that's pretty trained completely outside and we can measure this for any given text now what we're
00:13:03
doing at the moment is to try and end to end jointly learned the topic model also in there um
00:13:14
i
00:13:18
yes
00:13:20
uh so this goes into the attention models so if you go oh
00:13:25
two or
00:13:30
this guy here right so here we have the topic distributions so this we can compute for any
00:13:34
given text right so if you have any stretch of text i can computers distribution over the topics
00:13:40
right um and this guy then goes into our context vectors so the thing that helps me generate language
00:13:46
right so instead of just saying oh that's just um though the words and
00:13:50
the attention over them now it stops class this topic distribution yeah right so
00:13:59
well that's an additional input exactly into into this context vector i'd wear before
00:14:03
um if you if you look at these architectures here so i'll come
00:14:07
to expect to hear would really only have this stuff here um right
00:14:12
and and now we would have this additional a uh uh i would say oh by the way this is how
00:14:17
strongly and don't just see individual terms but also topics being expressed so now i would know all becomes a lot of
00:14:23
health related information or here's some injection off of east asia or what whatever your topics maybe right
00:14:31
so so so right now the the the big idea i guess is
00:14:33
to also learn the top is wouldn't rather than just injecting this daily
00:14:38
and uh from the outside this topic model to really learn topics such
00:14:41
that they maximally hope summarise has right now we don't yet um and it's
00:14:49
proving hard to do so also i i'm not sure we entirely know how to do that yeah but um so where that
00:14:56
uh_huh
Conference Program
Clinical Natural Language Processing
Carsten Eickhoff, Assistant professor of medical and computer science at Brown University
Aug. 20, 2019 · 11:05 a.m.
299 views
Clinical Natural Language Processing Q&A
Carsten Eickhoff, Assistant professor of medical and computer science at Brown University
Aug. 20, 2019 · noon
110 views
Recommended talks
Presentation of the «Biometrics Security and Privacy» research group
MARCEL, Sébastien, Idiap Senior Researcher
Aug. 29, 2018 · 2:19 p.m.
6512 views
Watson for You
Jérôme de Nomazy, GM & IBM Watson
Aug. 28, 2018 · 4:38 p.m.
565 views