Player is loading...
Food, health and disease: The evidence and reporting the evidence
Dennis Bier, Children's Nutrition Research Center, Baylor College of Medicine, Houston, USA
Friday, Sept. 29, 2017 · 11:11 a.m. · 49m 12s
Embed
Transcriptions
Note: this content has been automatically generated.
00:00:01
okay well it's great fun to be here uh
00:00:04
i shows you know what topic that relates a bit
00:00:09
more to my night job which is the other
00:00:12
of the a. g. c. and i have a feeling that i was initially invited because of my day job and
00:00:19
quite a few years and look good in carbohydrate metabolism but uh as you'll see i've
00:00:25
wanted develop another thing because in order for us to understand the consequences of
00:00:30
but very lookers and carbohydrates and sugars we need to understand have well we know
00:00:37
so disclosures i've consulted in one time or another for just about everyone
00:00:44
a pointer company is bunch of governments a whole bunch of academics aside is
00:00:49
sometimes they paid me and so i'm completely conflict that uh you
00:00:54
can uh you you can you know fine you know uh
00:00:58
diametrically opposed companies that and i think these things sort of cancel each other out of ah
00:01:05
and this event given all of that uh an inhibition by the
00:01:09
way the travel expenses an honorary input the stock right
00:01:13
now i'm not gonna mention any commercial products because academic products can more
00:01:19
then make my case and i say that following what
00:01:23
is a completely informal and and systematic review
00:01:27
of an egg data that i've derived from more than fifteen thousand manuscripts that of course my desk
00:01:34
as the other ray j. c. n. over the last ten years and uh i would say you you know uh
00:01:40
the a. j. c. and uh does not get some of the studies that go off to the big weeklies
00:01:47
and a variety of public health you know type manuscripts but i would say that
00:01:51
we did a pretty good selection of things after that because of our
00:01:56
global position and clinical depression so i think the sample
00:02:00
even though it's an act than mine and data
00:02:04
uh i think it represents what might be considered a very good inflow of
00:02:09
a question related studies that you know that appear in the literature
00:02:15
so my thesis is that if you wanna have confidence in your beliefs about
00:02:21
the impact of liberating carbohydrates on human health you have to have confidence in the measurements
00:02:27
and evidence of a causal pathway now yesterday if you look at the
00:02:31
bottom of the slide you know a bit our son mentioned three
00:02:35
essential components and i think my slide was prepared before you
00:02:39
mention those but i added this to the bottom because
00:02:43
the conference in measurements are david's data and methods can reliability and the evidence
00:02:49
of the causal pathway is the logic that can extant conclusions and
00:02:54
i think all of us have to be aware and keep ourselves always aware of richard fine men's principle
00:03:00
at the first principles that you must not fool yourself and you are the
00:03:04
easiest person to fool and what i mean by that is that uh
00:03:10
most everyone that i know attack academics that i know that publish spend a lot of time
00:03:17
on evidence that doesn't agree with what they think
00:03:22
and they spend very little time on the nature of the evidence that does agree with what they think
00:03:27
because it agrees with what they think so we have to be very careful about fooling ourselves
00:03:34
now the grass tyson said or you know a decade or so ago
00:03:39
uh no science to choose maturity without precision data sciences about measurement
00:03:46
preferably with something that is not your own ah yes which are
00:03:50
inextricably code join with the baggage of your own brain
00:03:54
and i think that's an extraordinary statement and he's talking about
00:03:59
really biases that are introduced by preconceived ideas you know
00:04:04
various you know notions that you come up with your
00:04:07
unchecked imagination and the light now by that definition
00:04:12
nutrition human interaction is a very mature finance so whole decade
00:04:20
and a half ago i'm sitting having my sunday coffee
00:04:25
reading the new york times i just try and relax and enjoy myself and there's an article one airline food
00:04:31
no this is not going to do with nutrition these are airline
00:04:34
executives talking about having to serve food to people on airlines
00:04:39
and the southwest airlines executives uh that southwest airlines doesn't serve food but in surveys
00:04:46
is consistently rated uh serving good meals now people don't even know they ate
00:04:52
how can you expect them to tell you reliably what they eat
00:04:58
i mean you know seriously guys you know okay so when we
00:05:03
have data about dietary intake records that are matched with you
00:05:08
know energy determined by doubly label watered at the same time
00:05:12
and now there are hundreds of subjects i mean it's
00:05:16
probably close to a thousand that with this is done but certainly six or seven hundred energy intake
00:05:21
is routinely under reported by twenty percent and obese people under reported by more than twenty percent
00:05:28
we're not talking small numbers you're in the energy business this is you know this is like
00:05:32
but the malformed and energy twenty percent okay
00:05:37
and then individual nutrient takes are validated
00:05:41
with correlations in the point to the point five range now i want the word
00:05:45
validated removed from wall nutrition manuscripts and they can gonna make this rule
00:05:52
you know for the a. g. c. n. because what it means is newer methods are just as bad as the old ones
00:05:59
it doesn't mean they're right in any way shape or form okay
00:06:04
so if we look at the study that was published you know some years ago
00:06:07
from anonymous phase group and that the u. s. d. a. where they did
00:06:12
a twenty four hour recalls every ten days for people over the course of six months
00:06:19
and to estimate adequate into its you'll notice that it took many many
00:06:27
recalls you know to to estimate what accounted for eighty percent of the variance of
00:06:33
the average intake over that six month period okay now these were comparing
00:06:40
you know a dietary intake records with what the with the average i mean
00:06:46
they didn't have independent measurements of what the intake actually was right
00:06:51
so this could be much worse than it appears to be but the point here is that
00:06:56
a very large fractions that is are done with that anything like this
00:07:00
number of recalls or and or you know or record dietary records
00:07:06
recently published a paper in the ages yeah and actually two papers that were you know you're
00:07:11
really not twins and but maybe mounted to port uh in which my own workers
00:07:19
we're measuring controlled feeding studies this was that the productions enter your wanna land the and a
00:07:27
rose print the son of friday whether people so the user individuals who receive their diets
00:07:34
uh you know over a period of i think it was two weeks or something of that sort
00:07:41
and they were to buy all markers with the independent by markers were matched to
00:07:46
what their intake should be by the fact that they receive the diets
00:07:51
and if you'll notice for all of these you know there's none of them
00:07:56
really that account for more than half of the variations are these are controlled feeding studies so imagine what
00:08:04
is happening out in the real world and for some things that were important here we've talked about saturated fatty acids and
00:08:11
things like this the actual variation explain was you know it's very minor fraction of the total
00:08:18
okay
00:08:21
now the other thing that we don't generally talk about in these kind of in directory records
00:08:27
that uses the changes in food composition the variation in fruit composition so here is
00:08:32
one study were very look at the variation and vitamin c.
00:08:37
uh overseas and so the the yeah the the summer is on the left side and uh
00:08:43
i think it's a summer and winter is on the right side or fall i'm sorry spring and fall
00:08:48
and what you know this is this this was paper was done by experts why
00:08:53
because they change the scale and the second thing to somebody who knew how to change the scale to
00:08:58
make this was not you know this was just not casual right they made it this way
00:09:05
yeah so look at the variation in those concentrations just look at that
00:09:10
the conference and intervals they're they're huge right and this is
00:09:15
not almost never accounted for in dietary intake studies
00:09:21
a similar kind of study was done uh about you know
00:09:25
decade or more ago now looking at individual components
00:09:29
of rustic uh so this is a good core group look assimilate in three different prospectus p. c.'s
00:09:36
and in this case what you see is there is a rough you know distribution of those response
00:09:43
and if you try to relate these responses to the
00:09:47
relative risk for cancer okay and you don't use
00:09:52
variation in the problem in the production chain of concentrations you come up with was a
00:09:58
pretty good dose response curve in the reduction in the relative risk of cancer
00:10:04
but if you also then it back in the variation introduced by the production chain just disappears entirely
00:10:12
right now i'm not saying that this is representative of all studies of this type or any but it's we don't see this
00:10:20
we hardly ever have anybody do with that i know of or at least most of
00:10:24
the papers we get and i think we're potentially you know fooling ourselves sort
00:10:33
some iterations the science of wood we can measure food intake is just bother anybody yeah i mean so i imagine
00:10:41
if you try to understand diabetes which can't accurately and
00:10:45
precisely measure blood sugar or renal disease with that
00:10:49
confidence in your area and create name measurements i mean we have a problem here and it doesn't you know
00:10:54
maybe it's only me you know i guess it doesn't seem to bother a lot of people but
00:10:58
i think this is a serious issue that we have to face and i don't think this field is gonna move
00:11:04
really forward into a much more mature science without someone facing
00:11:09
this uh and even in cases where we have
00:11:13
no one you know by market for frequently not even you just
00:11:19
so the first thing is you know we have a problem with my measurements at least i think we do
00:11:26
uh and the next thing is what is what about evidence so no one you know if
00:11:31
there's no one in this meeting is gonna stand up here and say i'm gonna
00:11:35
tell you something and i have no evidence for at all but you know i'm
00:11:38
gonna put it on okay so so that no one in any committee or
00:11:44
whatever will tell you that they're not making an evidence based decision so talking about
00:11:49
having sound science i love that one 'cause it shows up all the time
00:11:53
everybody things to doing sound science otherwise they don't get crowded right so so i don't think that's very helpful
00:11:59
so why is there so much you know fast now about evidence based depression
00:12:04
'cause everybody thinks they're giving you evidence it's in the definition of evidence
00:12:10
what am i calling it but instead how my waiting okay
00:12:14
no one act kind texting given the conversation just today i'd like to remind you that the dietary reference in
00:12:19
takes for non of the forty essential nutrients has been established by formal systematic review of the evidence
00:12:27
not okay there's been a little evidence light or something is vitamin d. in cancer and
00:12:34
some they're mean it was a little bit but okay the same is true
00:12:37
the dietary guidelines so the dietary guidelines evidence review was looking at what's happened
00:12:45
to the evidence in the last years since the less dietary guidelines
00:12:48
that's not how you doing evidence review you have to go back to the beginning and add the new stuff to all the other stuff
00:12:56
that's just not been that right and then we so every can choose who
00:13:00
was in charge of the dietary guidelines to or so guidelines ago
00:13:05
when we talk about evidence read you know review right so i i mention that i didn't think
00:13:10
it was this was a systematic review when he said well what's the systematic review white
00:13:15
okay i don't really know what that means the other part is that it's there's an inherent conflict
00:13:22
of interest here so the dietary guidelines can maybe you're supposed to give advice to the government
00:13:30
but they're using the government evidence library to establish what the advice should be
00:13:36
now if industry did that there would be an uproar right it would be an absolute uproar
00:13:42
i mean the so to me this there's an inherent conflict here and we need to find a way
00:13:47
around that for the future now ah there's there's a i get i think there's a relatively
00:13:56
prevailing opinion throughout the cushion community that somehow
00:14:00
nutrition evidence these different from other sciences
00:14:04
and i just don't believe that are all high this the principles of the scientific
00:14:10
method are the same whether you're doing physics or what you do we nutrition
00:14:16
you need an explicit question
00:14:20
we need to define an explicit outcome ahead of time you need to intervene in some way so
00:14:27
no causation without manipulation then you have to repeat it so you feel like you're
00:14:33
you you measures something real and if you're lucky and you repeat it enough times then you get predictability
00:14:41
and i don't i say nutrition is absolutely the same as every other science
00:14:45
in this respect what's different it's really hard to do the studies
00:14:51
the principles on any different but it's very hard to do long term diets that is
00:14:57
in control fashion and some of them may be unethical as with her yesterday
00:15:03
so this is the problem right and i also want to remind you that we're all
00:15:08
experts here and the bottom of the evidence based hierarchies expert opinion so that's me
00:15:13
and pretty much everybody in the audience you might disagree with what i just said right i
00:15:17
mean so we're all on the bottom of the evidence hierarchy what we need is
00:15:21
so movement up the evidence hierarchy and what we have is a lot of we
00:15:26
have a massive amount of data in the middle of the evidence or okay
00:15:32
so burner horizon burke who has who to find it but we
00:15:37
usually call the uncertainty principle set it in a different way
00:15:41
which i think applies to essentially every kind of scientific endeavour we're gonna be involved in
00:15:47
the persian or otherwise you said since the measuring devices been constructed by the observer
00:15:53
we have to remember the what we observe is not
00:15:55
nature itself but nature exposed or method of questioning
00:16:00
this is really a critical concept i mean it's an extraordinarily critical kind so
00:16:06
and you know what what what does it matter okay now i picked this example here
00:16:12
because it's a drug example that's that's safety and efficacy the jobs and so
00:16:17
we don't worry about complicated died studies a million people seventy five studies
00:16:24
and what what's right or if one looked at or or if one
00:16:30
looks at the observations bodies okay one sees a very significant hazard
00:16:38
of the axis and the hamster here
00:16:42
have a ratio is very similar to what you see in the question
00:16:45
thirty one point seven six you know that kind of range
00:16:49
and one can reduce that apparent hazard bit by a certain number of corrections right
00:16:58
but it's only when you deal with the r. c. to use the find out there's no
00:17:03
rest at all no increased risk or on your cities
00:17:07
this is been demonstrated in over and over and over again
00:17:13
and we have again numerous examples in an accretion feel so here we have
00:17:19
two different views of nature again by the method of questioning so we a beta
00:17:23
carotene intake and cardiovascular disease mortality if one looks at observation or cohorts
00:17:30
okay one sees a point estimate that's a very significant protective effect
00:17:38
but if one looks at the randomised control trials not only is there
00:17:41
no protective effect is the suggestion that there may actually be oh
00:17:46
yeah
00:17:50
and the if the field is full of the stuff so this is one
00:17:54
of my favourites 'cause i was on the phone look yes it uh
00:17:59
committee for reading probably guess it the grains and at the time there was
00:18:04
a prevailing opinion that well you we're going at it to grain
00:18:08
products because of neural tube defects but just think when we had for
00:18:11
what yes and kind of we put in b. twelve and relaxing
00:18:15
we're gonna lower foremost sistine levels so we're gonna get a totally additional value
00:18:21
added a fat right so that this is in the uh in german
00:18:26
okay if we reduced almost to screen level five to eleven percent
00:18:31
we can decrease my korean far trends and korea a coronary heart disease dats
00:18:38
in humans okay by you know eight to twelve percent
00:18:43
right now in addition to this massive reduction in cardiovascular
00:18:49
complications by doing this the calculation what how much money the country with saving it was immense
00:18:56
alright so mark twain as one of my fear many like they record your t. v. set supposing is good
00:19:05
finding out is better and following these theses i mean of the whole variety
00:19:13
of control trials with on the cardiovascular events in which we have
00:19:18
buffy bow group and then a group like we see you probably have
00:19:21
to beat will be relaxing some different permutations and by and large
00:19:27
nah nothing happened there are few exceptions to that
00:19:32
but by and large nothing happens or this you know money saving
00:19:36
and things just didn't come about no this is also true
00:19:39
provide in many environments say in cardiovascular disease i mean and for
00:19:44
the fact that there are now a several dozen hypotheses
00:19:51
that were derived from observations that either been tested by r. c. t.s in about
00:19:57
fifteen to twenty different publications and none of them have
00:20:02
actually supported the hypotheses generated by the observations studies
00:20:09
now can we get better i would just brought this up because it's another popular topic can
00:20:15
we get better if we have by workers because what we really wanna buy the channels
00:20:21
and that's a long term study so can we you know what can we do with bile market so
00:20:27
they would the mets has a very nice little discussion of that and this article
00:20:31
in the annals of internal medicine any points out that there's really only one
00:20:38
convincing kind of my own marker that you can use and that gives you the true estimate of the clinical outcome
00:20:45
and that's when it's right on the causal pathway any intervention
00:20:50
you know uh is also directed towards that pathway
00:20:55
uh i like a good idea you showed that yet another slide which i together
00:20:59
'cause it's too complicated to explain but he showed another series of permutations
00:21:05
of the bile markers that are not on the causal pathway okay and where the
00:21:10
by market may be affected by not the causal pathway that don't do this
00:21:15
so we have you know many we'll probably by marcus unless we
00:21:19
know they're on the causal pathway and we haven't points
00:21:22
that allow us to assess their validity and as they would
00:21:27
the mets without we have this whole list of drawbacks
00:21:32
that were approved with surrogate markers and which which all of them change the surrogate marker
00:21:40
but none of them had the clinical effect that they were supposed to have
00:21:45
okay things to increase you know lowering l. d. l. increasing h. p. l.
00:21:50
but pressure you know related roads et cetera so the bottom market business is
00:21:55
very iffy unless you have a lot of good information about it
00:22:02
now uh
00:22:06
as i mentioned we have two ways of looking at nature and we have very different is about you
00:22:13
know why differences of opinion about the value of some
00:22:16
of these approaches so these are three quotes
00:22:20
from very famous people whose names i have left off for that
00:22:26
protection of the innocent sort of thing and uh this
00:22:31
you know uh these are people you would all recognise if we if i put the names so in one case as with them
00:22:40
referee for the n. a. j. c. an article who said and it's return
00:22:44
to the water is coming to the authors the weather should be
00:22:48
i should note that in dietary epidemiology prospective cohort studies
00:22:52
are regarded virtually on par with randomised control trials
00:22:56
now as far as i know the only people believe that on a personal epidemiologists right
00:23:01
ah but that's what he said alright ah the other ones
00:23:06
uh you know well constructed prospect of cohort studies
00:23:10
those can be used when for cause out meeting with a high degree
00:23:13
of certainty when randomised trials or of hard endpoint some practical
00:23:19
and then another one there is really very little directory confounding going on here we have almost
00:23:25
everything in the model really not much left that could be plausible can founders now
00:23:32
uh is that true well it may have been crew in the twentieth century but if we look at
00:23:38
what big data have told us about associations of this is
00:23:43
a study that one one you know figure from uh
00:23:47
a provides raj but ellen john you need is that was in gem a couple of two thousand fourteen or so
00:23:54
in which they looked at the associations among variables and and haynes
00:24:00
and only look that associations that the strength of greater than zero point two as i recollect
00:24:07
and you know there's the inter related you know correlations among these variables
00:24:13
and in fact the most of them had associations on the
00:24:18
order of sixty other come variables in an ins
00:24:22
so if you look at beta carotene i think it was like sixty four associations with beta carotene
00:24:28
so the idea that you can you know come to conclusions about beta carotene
00:24:34
seems kind of fanciful to me you know given the level of these different kinds of associations that would set for years
00:24:41
these are lifestyle things okay i can buy that you know they can make one and
00:24:45
all that sort of stuff but we were using tools of the last century
00:24:51
to look at things that i think we need to move on and even more impressive this is from
00:24:57
we were able it's group that appeared in biotechnology the sheer okay the
00:25:03
the in addition you know this or i shoes the endings data
00:25:08
uh the the studies that were done by leeway code were and you know and and or some other
00:25:13
subjects that they study wanted to really over time and in addition to
00:25:20
the sort of physical measurements they did a variety of laboratory measurements and they did you
00:25:24
normally measurements and they measured you know a a whole bunch of series of you
00:25:29
know a blip is and things of this sort of this is a much more
00:25:31
comprehensive want to look at the the related associations here this is the cardiovascular
00:25:39
i think what they called it the little keys you know the the the neighbourhood okay
00:25:43
and what leroy quite did which is almost never done in the traditional
00:25:49
the correctional epidemiological studies the look at associations with the t. f.
00:25:54
zero point o. five after the correction for multiple comparisons
00:25:59
that's almost never done in the traditional reporting of nutritional epidemiologist
00:26:08
so you know i think i don't present all of studies but you know our seat
00:26:12
user repeatedly fail to support implications from observations that is in the christian anyway
00:26:18
and we need to remember first that in those observations that useful to the data or diets or
00:26:25
whatever the nutrients kinda flew it's they're responsible to defect those hypotheses
00:26:31
so was i policies that if they'll know there's lots of reasons why that should be so to do
00:26:37
subjects that are in randomised trials and populations that is that different groups of people right
00:26:44
uh for the r. c. t.s we could have one goes we could
00:26:47
miss the therapeutic window we may not give it long enough
00:26:51
well we maybe using the wrong surrogate endpoints or the one that you
00:26:56
know we generally don't wanna talk about is the nutrients not responsible
00:26:59
for the for the fact he could be something else in non between in the food we could be things to do with the
00:27:07
it individuals genome rabbit genome or not it no matter gene on yet another level of complexity
00:27:13
and they could also be due to just our inability to what town for
00:27:18
this huge explosives all the that i showed from so raj and
00:27:24
and uh the record so it just to be the unknown and
00:27:28
you know that done rooms fills unknown unknowns right we have
00:27:32
the roll out there we just haven't been able to deal with them because of the way we're approaching these data
00:27:37
and and if you tell us the some of my good friends they say but wait cigarette smoking as a
00:27:44
cause of cancer has never been tested with an r. c. t. v. you say well you're right
00:27:50
and the leo point out why yesterday and the reason is that the
00:27:57
relative risks of cancer was smoking on your fifty or sixty
00:28:02
so you didn't feel pretty sure that this is well above
00:28:06
the noise level of your method of finding out
00:28:10
whether someone smokes or not or how much they smoke most nutritional risks are down here below true
00:28:18
and frequently you know we see papers and i unfortunately i publish a bunch of them um it might
00:28:23
better than a bunch of them i don't wanna take single rather for that one point true
00:28:29
rest now i don't know whether those risk around so i don't know it's level at all
00:28:35
we just don't know or we don't talk about it very much
00:28:41
the other issue we have with observations thirty says something that i think
00:28:46
is really insidious like colour making a silk purse for myself here
00:28:52
so one of the conditions that's essential for mad analysis is that all the
00:28:57
primaries studies have to provide an unbiased estimate of the true with that
00:29:03
now this is generally matter if you have good or cities is the only way
00:29:07
to deal with in all the contenders and in that sort of stuff
00:29:13
but the condition is often not met with observation studies first due to a whole
00:29:18
bunch of well appreciated biases and if you haven't seen it there's a
00:29:22
article by neal barnard walker willard and being in the current jammer which they discuss a lot of the use
00:29:29
and uh up a bunch of in it you know other than the ones i
00:29:33
have here they this is a two page article which is worth reading
00:29:38
but what they don't discuss is a what i think is a seriously under appreciated problem
00:29:45
which is the fact that the initial studies themselves often don't
00:29:50
account for the force discovery rate because there are
00:29:54
have multiple observations some hundreds of observations that
00:29:57
are never corrected for the multiple observations
00:30:01
so you have an effect which is really false discovery and that leads into
00:30:08
that analyses when and you know inappropriately because it's not the initial data or by
00:30:16
now we have ways of trying to approach cause albion observation studies and i
00:30:20
think these are being more commonly applied and they're clearly very important so
00:30:25
i picked this not because is the world's greatest study but it's the simplest to look at you know so it's easy to show
00:30:31
and as many of you know there'd been a whole series of observation studies
00:30:37
which suggests that low vitamin d. levels are related to type two
00:30:41
diabetes and in fact it's a pretty impressive you know
00:30:45
rest here one point two relatively asked and very high degree of how you
00:30:51
know how strange and see if you know for false discovery later so
00:30:56
those data are you know with circulating and i recently there been a number of you know approaches using
00:31:03
i mean the only minimisation which allows which in which the
00:31:07
subjects are sorted on the basis of their genes
00:31:11
which presumably can be affected by the environment and in the nature of this
00:31:15
observation was that when we look at that the whole effect disappears
00:31:20
and this doesn't prove that vitamin d. is not somehow on the
00:31:24
causal pathway but it's very strong evidence against it in
00:31:28
and use kind of men billion minimisation studies have now cleared up at least in my mind a whole bunch of
00:31:36
a very weak hypotheses based on traditional observations studies
00:31:42
so when the and you know our final conclusion from a lot of the study should be maybe
00:31:48
right associations are uncovered and observations that is right policies
00:31:54
we may never have for the reasons we discussed earlier people talked about yesterday we
00:32:00
may never have the international studies that allows us to fully test these hypotheses
00:32:06
but the response to that deficit shouldn't be to promote the level
00:32:11
of evidence we do have because i love it which is
00:32:14
frequently done explicitly i'm very often implied
00:32:19
you know what we need to do is to demote the certainty
00:32:23
of our conclusions how strongly do we know something okay
00:32:32
now i i didn't try to get the version in my talk after yesterday's talk about shoulders
00:32:39
'cause i thought i need to throw in some things that was my original charge you know the person carbohydrates
00:32:44
so here's a can of coke and a tablespoon and a half of olive oil which has more calories
00:32:52
the olive oil is more calories right
00:32:55
now if you're doing dietary intake records which is easier to remember
00:33:01
that you have a can of coke yesterday i can't okay
00:33:05
huh what about the void when you're stir fry or in your pay yeah
00:33:13
i mean this this is a serious bias in intake records it
00:33:17
and i would argue i have a thesis that no one is tested for me yet that
00:33:22
because you can measure the cans of coke better then you can measure the hidden fat
00:33:27
the variation around your intake of solid is is now or
00:33:31
then fat and therefore when you try to look at
00:33:35
statistical significance it drops out a significant big not because it
00:33:39
is actually but because you measured more precisely right
00:33:46
the other thing that came up yesterday that surprised me is that uh no
00:33:51
one mentioned the largest most comprehensive evaluation
00:33:55
of dietary core white hydration how
00:33:59
this was the u. k. scientific advisory committee and nutrition report that came out a few years ago
00:34:05
and this report was nine years in the making
00:34:09
it contains the report itself which is like two hundred and seventy five
00:34:13
pages and all the supporting systematic reviews that went into that
00:34:18
which is which are nearly three thousand pages so this is all on the web you can download it but i warn
00:34:24
you if you go to download the supporting material prepared to go away for a while it's a big file
00:34:30
so no one's gonna do this again in not gonna do this at the
00:34:36
dietary guidelines to come nine years to do and publish the stuff right
00:34:41
and this is the biggest most comprehensive davis on you know so that
00:34:47
so the u. k. e. scientific advisory committee report on carbohydrates intel said
00:34:53
they could find no association with or effect on should resign
00:34:57
corner events just delegate biased all that blood pressure
00:35:01
totally l. or h. p. l. testing for eclipse rights fasting blood glucose and insulin
00:35:07
or the risk of type two diabetes those were associations that it's uh
00:35:14
yeah
00:35:16
okay they looked also have randomised control trials
00:35:21
and the data on randomised control trial so there was insufficient evidence
00:35:26
of sure this weekend beverage effect on body weight weight gain
00:35:31
ah energy into it i think i see me or in so many me an an insulin resistance and sensitivity
00:35:37
okay and i senior the aura of uh the parapet glucose tolerance
00:35:44
like oscillator things like like gasoline middlemen and vascular compliance and c. r. p.
00:35:53
all these things are widely believed to occur this is a very
00:35:56
comprehensive report that says the evidence for those things is insufficient
00:36:02
they did damage they the committee said the reason the fact that they
00:36:05
can find an from observation all day it on an association between
00:36:09
should this when beverage consumption and the higher incidence of type two diabetes but there was a problem they couldn't find
00:36:17
any evidence of total sugar intake and type two diabetes or any association between
00:36:23
shoes when beverage intake and body weight and body fat which is
00:36:27
account for ninety something percent of the verification in developing type two diabetes
00:36:33
and it will the arms of a well throughout this report
00:36:37
for every association they look at the the committee stays repeatedly they
00:36:41
can exclude confounding and they don't show cause and effect
00:36:47
now we have to this extensive report that i thought is rather remarkably in you know uh
00:36:53
on the side of the innocence of sugar okay what happened to the recommendations so
00:37:00
like the w. h. o. they recommended that sure was intake be about five percent
00:37:05
of energy now the w. h. o. report okay rip was based largely on
00:37:12
the dental carries issue marion mention that yesterday the orders of those studies
00:37:17
say it's very low quality evidence now the w. h. o. said
00:37:21
well we made a conditional recommendation and i say if you don't have any evan and you should make any recommendation at all
00:37:27
right i mean to suggest that you have some reason to think it's five percent
00:37:35
they have no evidence offer that and gordon guy and others you know ah put them to
00:37:40
tasks and now they're doing some of the grading of evidence and things of this sort
00:37:44
precisely because of the snow the u. k. scientific advisory committee
00:37:48
and aggression which did this nine your extensive data analysis
00:37:53
when it came to make the final recommendation okay they'll use
00:37:57
this energy balance target from a previous can maybe
00:38:02
so they went to a subject or opinion of a previous committee after all
00:38:08
this objective data analysis it makes no sense to me at all right
00:38:12
i i personally see that there was an agenda there okay they wanted five percent they down the reason to come to that
00:38:19
now the other problem that we have is with that we see all the time is
00:38:26
you know oh no more than ninety percent of studies reported and certainly and biomedical literature but in depression
00:38:34
our show positive finding now this is another use a big data years you
00:38:39
know two million but most central abstraction wait million men align abstracts
00:38:45
ninety percent of the papers report accosted a fact finding
00:38:52
and on the face of it this is impossible right we're just not that good
00:38:57
we can't possibly guess right ninety percent of the time we're lucky if it's fifty percent of the time okay
00:39:03
so you know as marion said yesterday that the industry studies okay we're both the sponsor
00:39:10
and what i see in the a. g. c. n. is the academic academics that is
00:39:15
i usually say verbal to the legions biases are the people who are reporting them
00:39:19
because they have from years build on these particularly weaknesses and no matter
00:39:24
what happens no one can say nothing happened in this paper
00:39:31
so we have this vast preponderance of positive studies and uh rubber tables
00:39:39
and said in his book a statistics as a as a
00:39:43
structured arguments something of the you know as a prince principal argument you know
00:39:46
over confidence aborts uncertainty well that's true of all the investigators and
00:39:51
simmons who said this another way when faced with ambiguous
00:39:55
analytical decisions researchers will tend to conclude with convincing
00:39:58
self justification uh you pulling themselves that the appropriate
00:40:02
decisions are those that result in statistical significance
00:40:05
because they can't write a paper that said we did all of this and we work hard and we spend a huge amount of money nothing
00:40:13
okay so uh this is i've summarised human one slide
00:40:19
what i can spend an hour on the various investigator degrees of freedom that would be
00:40:26
to the nature of our positive you know literature and from this if you
00:40:33
haven't seen that any of these these principles come from the broccoli in probability principle
00:40:38
by david and which is a wonderful thing to read if you haven't seen
00:40:44
so to get
00:40:47
the kind of literature we have the first thing you need to do is you need to invoke the wall of
00:40:51
inevitability that is something must happen if i do a
00:40:56
study or you do is study something must happen
00:41:00
this is why we have publication bias in the nature that ninety percent
00:41:05
positive studies white negative studies gonna drawer blah blah blah i
00:41:10
we have to make sure that you count on the wall of cruelly large
00:41:14
numbers if you measure and nothing's likely things will happen just by chance
00:41:22
if you ask more than sixty one question you haven't ninety five percent chance of showing
00:41:27
at least one positive that the p. less than point o. five level and many
00:41:33
you know nutritional observation visual studies have hundreds of observations in the same study
00:41:39
so one expects more than one thing to be single begin at the point of five love
00:41:45
you have to master the last selection which is that you should be our linear room
00:41:49
where land you draw the balls like around right so here's where we have
00:41:55
all the post hoc p. hacking which is as far as i can tell pervasive we have what is
00:42:04
i think equally pervasive when not notice by the community is the baiting and switching of
00:42:08
primary and secondary endpoints which is why you need to read the registration documents first
00:42:14
before you look at the paper and you will discover that
00:42:18
either the secondary endpoints are reported as the primary endpoint
00:42:22
or the primary endpoint you know in the secondary endpoints switched positions during the course of the
00:42:27
study now i don't know about you but i think someone was looking at the results
00:42:32
we figured out which one should be primary right we need to apply
00:42:36
the the laws of near enough which is things like implying causality
00:42:41
or taking numbers that are similar and even though they're not
00:42:46
statistically significant are similar enough that we're gonna say
00:42:50
they're equal word okay exaggerating effects size and on all the time especially with use of relative risk
00:42:57
and then multiple publications from the same study which has a variety
00:43:02
of things one of which is to completely obscure the
00:43:05
primary and secondary endpoints or the number of observations it's actually been done in the study so it's very hard to
00:43:11
you know compute any corrections and okay then we have
00:43:19
you know smaller facts with big stand uh i'm not gonna want then of phi
00:43:24
this except to show you this is a study that's used as a positive
00:43:30
frequently tattered as a positive study in the in in an intervention that effects by way now
00:43:39
you know ah you need your statistician here right i mean just look at those
00:43:44
numbers can you tell me which is the control in which is the
00:43:47
intervention group this is unwilling journal paper by the way is one point out
00:43:52
you know with this is we have a lot of spin and duration
00:43:57
okay what about industry sponsorship it's actually hard to get enough studies and this but there are
00:44:04
true that came out recently from lisa barrows group and one in which appeared in uh the uh
00:44:13
it was jammer drum at it like this
00:44:17
i somehow missed or maybe it's on the bottom so this is looking at group studies
00:44:23
that this side favour is you know let's variable conclusions were industry more fable conclusions
00:44:30
and in this study you can see that of this six
00:44:34
or so things here almost all of them across one
00:44:39
meaning that none of those study shows significant difference right the
00:44:44
point estimate of the uh of the total is also when
00:44:49
significant barely so with the p. l. point o. sex
00:44:53
but what i'm the point here is that we have plenty of evidence and studies
00:44:57
in which there is no difference between the industry sponsored study and the
00:45:01
nine industry sponsored studies for the results now they just put online
00:45:07
several days ago an additional study that deal with industries
00:45:11
body sponsor studies in obesity in public health depression
00:45:15
and i had a little trouble getting this and you know blah blah blah but like
00:45:20
at the conclusion so the uproar apparatuses that would be a statistically significant differences
00:45:26
in the research topics that were funded by the fruit industry in
00:45:29
studies because the the compare with other funders was not confer
00:45:34
okay so one can certainly find studies where there are some differences you know for example this
00:45:39
one here but on the whole it's really in that i don't believe it's you know
00:45:45
completely obvious that there is a big difference
00:45:49
so where do we go in the literature this you know i have to spend the this is my night job that's taken over my day job
00:45:56
uh we need to have all studies registered all studies every study all study
00:46:03
twenty percent of the studies in clinical trials dot gov more than forty thousand
00:46:07
or observation all but only twenty percent of observation studies are reversed
00:46:12
we need to report the primary endpoints together
00:46:16
and we need to make sure people explicitly identify secondary endpoints which is almost never done
00:46:23
we need to have an a priori analytical plan so that we can
00:46:26
deal with post type p. hawking p. hacking and related manipulations
00:46:32
and we need to put some perspective in reporting results are not only relative risk but absolute risks
00:46:38
number needed to treat number need it on a fax sizes and things of this sort
00:46:45
we also need to make sure that everyone uses mandatory reporting violence because that
00:46:51
those allow you to have in the manuscript essential elements
00:46:56
we need to have some way of dealing with alternative analyses no models that are
00:47:00
as complicated as the ones that we see in large studies have a single
00:47:05
uh you know uniquely identifiable solution there are multiple solutions that
00:47:09
fit the models with the same degree of statistical confidence
00:47:14
and then we need to have the original data available scrutiny now some of you know as i said or the out were
00:47:21
lecture you know some of you don't that there's an ad in the us congress switches are already passed the house
00:47:27
and is now in the senate it's called the kindest that that's an acronym for something i don't remember
00:47:32
it's an entirely politically motivated that it's against the e. p. a.
00:47:37
so the motivation for this that is anti u. p. a. no question right
00:47:43
what the act says is
00:47:46
if you are using studies that are gonna
00:47:50
right policy recommendations or things like that you know they need to be
00:47:54
published right and they need to be available for alternative assessment
00:48:00
no i think that should be the case for essentially everything
00:48:05
and then we need to have some penalties for non compliance with the journal editors think
00:48:09
that no clout you know i mean we don't have any way dealing with this
00:48:13
the n. h. frankly doesn't seem to care sort of somebody that we have to have some
00:48:18
way of having penalties so in the end you know don't believe everything you read
00:48:24
especially if it has a picture of a famous scientist a company you know
00:48:28
uh yeah things of that sort and then finally uh
00:48:34
a quote by jonathan swift sort of four hundred years ago or so is
00:48:38
that a man should never be ashamed to own that he's been wrong
00:48:42
which is saying in other words that use wiser today than it was yesterday and
00:48:46
i think we have to use that principle and evaluating the literature so thanks
00:48:56
hi
Conference Program
Nutrition advocacy in action: the politics of sugar vs. fat
Marion Nestle, New York University, USA
Sept. 28, 2017 · 9:27 a.m.
1336 views
Fat and carbohydrate recommendations - Have they changed?
Barbara Schneeman, University of California, Davis, USA
Sept. 28, 2017 · 10:14 a.m.
482 views
Promoting equitable health and wellbeing around the globe
Nancy Brown, American Heart Association, Dallas, USA
Sept. 28, 2017 · 11:30 a.m.
488 views
Gut hormones and nutrition
Sir Stephen Bloom, Imperial College London, England
Sept. 28, 2017 · 1:38 p.m.
1089 views
Nutrition, metabolic health, cancer and NCDs
Elio Riboli, Imperial College London, England
Sept. 28, 2017 · 2:27 p.m.
570 views
Infant and child nutrition and cognitive development
Wendy Oddy, Menzies Institute for Medical Research, University of Tasmania, Hobart, Australia
Sept. 28, 2017 · 3:51 p.m.
993 views
Microbes, metabolism and autoimmunity
Ramnik Xavier, Massachusetts General Hospital, Boston, USA
Sept. 29, 2017 · 9:06 a.m.
1955 views
Lipid metabolism in high fructose fed humans
Luc Tappy, University of Lausanne, Switzerland
Sept. 29, 2017 · 10:03 a.m.
1122 views
Food, health and disease: The evidence and reporting the evidence
Dennis Bier, Children's Nutrition Research Center, Baylor College of Medicine, Houston, USA
Sept. 29, 2017 · 11:11 a.m.
551 views
Deeper down the rabbit hole of data, analysis, and inference errors and suggestions for digging back out
David Allison, Indiana University School of Public Health, Bloomington, USA
Sept. 29, 2017 · 1 p.m.
501 views
Neural circuits of food intake - is it all about calories?
Harvey Grill, University of Pennsylvania, Philadelphia, USA
Sept. 29, 2017 · 1:55 p.m.
510 views
Recommended talks
Finding delphi in medical imaging
Eldad Elnekave, CMO, Zebra MED
June 11, 2015 · 12:24 p.m.
178 views