Player is loading...

Embed

Copy embed code

Transcriptions

Note: this content has been automatically generated.
00:00:00
Japanese panel we're going to be
00:00:01
reflecting on the last decade of all
00:00:04
car the role that it's played on how it
00:00:06
shifted over time and what our ideas
00:00:10
for the future of the of the track will
00:00:13
be my name is Morgan aims Sylvia and I
00:00:16
will be moderating we are two of the
00:00:19
three of the the E C.'s for this year
00:00:22
and also next year. And we have four
00:00:26
wonderful panelists that cover of
00:00:28
brought history of all chi. I will
00:00:31
start by actually all that soviet
00:00:33
takeover disappoint hi everyone thanks
00:00:37
for being here so I just wanna quickly
00:00:39
introduce actually a panel of about
00:00:41
going chronological order to be honest
00:00:43
our panelists to struck cut off with
00:00:46
about five minutes reflecting on their
00:00:48
own experience the assault co chairs
00:00:50
and they'll just briefly introduce that
00:00:52
and and then we'll we'll continue open
00:00:54
up the floor for questions that we
00:00:55
really want to encourage just being
00:00:57
actually discussions that we welcome
00:00:58
everyone to jump in with question
00:01:00
answers microphones there were two
00:01:02
rooms so so please come up and and
00:01:04
that's a really actually discussion
00:01:06
here. So okay so first penalty row
00:01:08
small vertical he's professor human
00:01:10
computer interaction of queen's
00:01:12
university school of computing in
00:01:14
Ontario and he's also the director of
00:01:16
the human media laboratory then we'll
00:01:19
have next speak at Sydney all over and
00:01:21
the fourth here and was professor
00:01:23
department of electrical and computer
00:01:25
engineering at the university of
00:01:27
British Columbia and he's also the
00:01:28
direct of the media and graphics
00:01:31
interdisciplinary centre and we have
00:01:33
very brown ways the professor of fixed
00:01:36
eye at the Stockholm university and
00:01:38
research director also after mobile
00:01:40
life research centre and last but not
00:01:42
least there's the yellow Rawson or as
00:01:44
using assistant professor of human
00:01:46
centre design and engineering at the
00:01:48
university of Washington and all the co
00:01:50
director of the cat tab. So let's start
00:01:52
right away. Um well you wanna come up
00:01:55
okay Okay just take a minute to small
00:02:04
laptops stumbles right Thanks to all
00:02:36
for being here I'm real forty gallon
00:02:39
although I don't think I started all
00:02:42
dark like a Singer who was involved in
00:02:43
that process called french which I ran
00:02:46
the second year I remember out there
00:02:48
she is yeah yeah you shouldn't be here
00:02:51
a yeah but maybe if you have something
00:02:53
to say you know come on come on stage
00:02:55
yeah I'll duck I came the name came
00:03:00
from the the current shut here of the
00:03:03
sick I care kind of your who's in the
00:03:05
room as well. Um complain to me about
00:03:08
that name french it's it's not a name
00:03:11
that people want to have on their
00:03:13
resume and I thought french was I think
00:03:15
number french I thought french was
00:03:17
cool. And you want to be friendly but
00:03:20
that was the motivation for changing
00:03:22
the name so we did that in sometime in
00:03:24
two thousand four and then ran it for
00:03:26
the first time in in two thousand five
00:03:28
so the motivation for all the kind and
00:03:31
it's predecessor the prince was very
00:03:33
much a a response to what we believed
00:03:36
was an increasingly frustrating and
00:03:38
conserve viewing process for cod. And
00:03:41
the important I'll show that in a bit
00:03:44
that was due to an exponential increase
00:03:49
in submissions which is hard to deal
00:03:51
with when you're growing at such a
00:03:53
rapid rate early conferences early chi
00:03:56
conference and not only had a very much
00:03:58
higher acceptance rate but they also
00:03:59
have a rescue ticket. So what happen is
00:04:02
the the problem chairs because like you
00:04:04
would be looking at two three hundred
00:04:05
papers max would go through all the
00:04:08
rejected papers. And the side of the
00:04:10
road papers that were unjustly treated
00:04:12
and then rescue them somehow or or
00:04:14
recruit more reviewers and get them
00:04:16
into the program. And when I became too
00:04:18
big around this time of two thousand
00:04:20
three two thousand four that's not
00:04:23
happening and so I very much saw ultra
00:04:26
high share as having the responsibility
00:04:29
to instead of the programme cheers use
00:04:31
the time to go to the rejected papers.
00:04:34
So the original model for all but I was
00:04:36
a jury process after a shame that off
00:04:39
loaded to capacity to rescue papers to
00:04:43
the older cultures. And in the program
00:04:45
committee meetings we would have a
00:04:46
program committee members or a sees as
00:04:48
we call them. So features. Um to get to
00:04:51
add a table papers that they in one
00:04:54
except for a high first look at so they
00:04:57
said this is the this was the picture
00:04:58
in two thousand four and I'm not gonna
00:05:01
argue but with this exponential or not
00:05:03
but we do agree that is growing rapidly
00:05:04
and not linearly and so as you see as a
00:05:10
consequence I don't have a pointer but
00:05:11
it to the right there you see that that
00:05:14
the number of accepted papers Penn
00:05:15
state table but the acceptances or the
00:05:18
submissions when up so so much that the
00:05:20
what what what the consequence was that
00:05:21
the the except rates went down
00:05:25
dramatically and calm I started in
00:05:28
nineteen eighty two with a forty five
00:05:30
percent acceptance rate. And it went
00:05:32
down to sixteen percent in the year
00:05:34
that we started the fringe and and and
00:05:36
okay. So what I first introduced all
00:05:39
dark I in two thousand five I had this
00:05:41
light on on on on there as well. Um the
00:05:45
reproduction then was that that the
00:05:48
acceptance rate as a function of the
00:05:50
conference age with the factor one
00:05:52
point three as so you take forty five
00:05:54
percent minus that and the production
00:05:57
would be that by two thousand seventeen
00:05:58
only all dockside papers would be
00:06:00
accepted because the acceptance rate
00:06:01
would go down to zero percent. Um I
00:06:04
when I when I told the security
00:06:06
responded and make sure that the
00:06:08
acceptance rate for chi two thousand
00:06:11
five went up to twenty five percent
00:06:13
from sixteen percent the year before
00:06:15
which I thought was very important why
00:06:16
because it means that for conservative
00:06:20
acceptances you can only really take on
00:06:23
work that is non controversial. And
00:06:26
therefore a lot of good work
00:06:28
potentially good work with it has some
00:06:29
flaws gets rejected and this was really
00:06:31
critical so this hasn't happened and
00:06:33
this is really good news and I think
00:06:34
that's one of the one of the results
00:06:36
from from buckeye. So the model was
00:06:38
inspired by this one that it is a which
00:06:41
was the art someone in Paris where the
00:06:43
impression is that there are because
00:06:44
they were rejected by the academy for
00:06:45
not being able to paint these are guys
00:06:47
like Monet and the the process was one
00:06:53
where we selected papers from rejected
00:06:55
work. So the co chairs would go into
00:06:58
the database and get access to all the
00:07:01
reject papers and the the idea was to
00:07:04
send the message the coffers very
00:07:06
strong message that excellent work was
00:07:08
being rejected and I'm very proud say
00:07:11
the first a buckeyes we're standing
00:07:13
room only and these were all rejected
00:07:15
papers and that's and I think a very
00:07:17
powerful message that this wasn't work
00:07:19
that should have been missed in the
00:07:20
program the second goal was to promote
00:07:22
acceptance the next year or two another
00:07:24
journal or or something else so to to
00:07:27
basically make a make make people aware
00:07:29
of this make it less controversial in
00:07:30
the process and make it more acceptable
00:07:32
for for archiving but all buckeye
00:07:35
originally was not archived. And I
00:07:37
still get some complaints every now and
00:07:38
then about some the papers in the early
00:07:39
years that are not archive so we might
00:07:41
actually do something about that. Um
00:07:44
another thing was that the conference
00:07:46
chair in the case of the french
00:07:48
Elizabeth the extra was the
00:07:50
accomplished here and she was so
00:07:51
intimately involved in the process and
00:07:53
so we decided there and then that this
00:07:56
was not gonna be a freak show it had to
00:07:57
somehow we represent that I submissions
00:08:00
and well we wanted a program that was
00:08:03
provocative an interesting there was
00:08:05
space for if it's a paper for example.
00:08:08
Um so some of the results I think I may
00:08:11
be over claiming I really don't know
00:08:13
but I think that abandoning the printer
00:08:16
proceedings and company the acceptance
00:08:18
rate to twenty five percent in the
00:08:19
first year with a very very wise
00:08:21
decision I'm gonna change the
00:08:23
conference because it stayed that way
00:08:25
below with we've had since twenty two
00:08:27
fifty percent of the two thousand five
00:08:30
old archive papers got published in
00:08:31
some way in a journal and a respectable
00:08:34
place or top your conference I I think
00:08:37
all dark I introduce the first at the
00:08:39
sessions a card which is since existed
00:08:41
until I think this conference where
00:08:42
they got property notes. Um and and
00:08:46
this is really wonderful actually I
00:08:47
have a feeling like going to this
00:08:48
conference today that it's very much
00:08:51
like the yearly all dark I it when I
00:08:53
look in the the program here is the
00:08:54
video but I took just the that's the
00:08:57
prior session of someone remote
00:08:59
controlling a person that's walking
00:09:01
using control muscle control using idea
00:09:05
smart phone and so they're actually
00:09:06
moving these people around the park
00:09:08
trying to avoid obstacles I think that
00:09:10
that that would definitely been an old
00:09:11
archive paper back in the day so that's
00:09:16
what it years to name some some
00:09:17
favourites and I I must say my
00:09:19
favourite moment. And all know I was
00:09:22
when I wrestled a tech lecture course
00:09:24
to offstage because you have sixty
00:09:25
slice present and sixty set a seconds.
00:09:28
And use it all don't worry at all times
00:09:30
and I'm like okay and another one was a
00:09:34
paper on edible you wise or you use
00:09:37
presented by MIT media lab in which
00:09:40
they planted questions in the audience
00:09:42
but a a a a clear example is a paper by
00:09:45
a touchy to had a people rejected on
00:09:49
wearable for sensors the martial arts
00:09:50
and actually got to to respectable
00:09:52
publications that that that here. Um
00:09:55
the jury model was not without its the
00:09:57
detractors of the perception was that
00:09:59
too much power was handed to the
00:10:00
chairs. And the perception was that it
00:10:02
was a undemocratic process and within
00:10:04
the company that was really frowned
00:10:05
upon. Um I think the concept of curated
00:10:08
work I was much more comfortable with
00:10:09
that because I have an art background.
00:10:11
But but within this conference it
00:10:13
wasn't really annoying thing. Um some
00:10:16
of the authors reviews to to to attend
00:10:18
or or publish their work as an old
00:10:20
archive paper because they thought it
00:10:21
was they wanted the real thing and so
00:10:23
that was an issue in the back in the
00:10:24
day. Um and as a consequence of these
00:10:28
criticisms now I have a feeling the
00:10:30
older chi has become and and that's I
00:10:32
think one of the because of of the
00:10:34
contention here in in the panel more of
00:10:36
a people's about so that was very much
00:10:40
the opposite of its intent so the
00:10:42
intent was not to put stuff online for
00:10:44
people debate on and then and then
00:10:46
curated but actually take rejected work
00:10:49
so while it still attract provocative
00:10:52
papers it's more of a soapbox now
00:10:54
rather than something that could result
00:10:57
in a prestigious publication venue and
00:11:00
that's one of my I think one of my my
00:11:03
observations for what's going on so
00:11:06
today's picture is not different we're
00:11:09
still experiencing tremendous growth
00:11:12
the acceptance rates have gone up as
00:11:13
you can see so I know that there's a
00:11:15
lot of very hardworking people with the
00:11:17
turtles here the monks a a month's them
00:11:20
working behind the scenes to try to
00:11:21
battle this problem but it's very hard
00:11:23
if you get a lotta content what to do
00:11:25
with it overwhelming amount of
00:11:27
submissions this year we had four
00:11:29
thousand five hundred and eighty six
00:11:31
volunteers for fifteen her in twenty
00:11:34
submissions I'm not sure these does
00:11:35
number is just papers or all the all
00:11:38
the all the submission certainly the
00:11:39
volunteers is is is everything that's a
00:11:42
tremendous amount of volunteering at
00:11:43
thirty and we need to scratch your head
00:11:45
a little bit you know be the the
00:11:47
probably meeting is a hundred ninety
00:11:48
people have to conference in its own
00:11:49
right. Um acceptance rates have only
00:11:52
normally dropped but it is a recurring
00:11:54
problem. And the review process still
00:11:59
yeah has a tendency to reject forward
00:12:01
looking working paper the favour of
00:12:03
safe submissions and moreover chi
00:12:06
reviewing has hi Chris steaks and so
00:12:09
there's always taking bases is much
00:12:11
easier to reject a paper on the basis
00:12:12
of a couple twos and that it is to
00:12:14
basically ballad out and except a paper
00:12:16
for its merits. Um well education
00:12:21
leaves little time for a second look at
00:12:23
difficult but disruptive work and let's
00:12:25
see what that leads to so here's an
00:12:28
interesting experiment that was done in
00:12:29
December this year. Um it was amazing
00:12:32
courageous move by organisers of that
00:12:34
conference to quantify the randomness
00:12:36
in the review process. They split the
00:12:39
program committee down the middle
00:12:40
effectively forming two independent
00:12:42
program committees and most the
00:12:44
submitted papers were assigned to
00:12:47
single side ten percent word you get
00:12:49
about both sides. Um so let's observe
00:12:52
how consistent two committees or on
00:12:54
which papers to accept. Um so well the
00:12:57
Hunter in sixty six papers the two
00:12:59
committees disagreed on the face of
00:13:01
twenty five point nine percent of them
00:13:03
forty three and I did not it's not
00:13:04
that's not that that but if we look at
00:13:06
this more closer. This twenty five
00:13:08
percent numbers misleading because it
00:13:11
actually means that the two committees
00:13:12
disagree more than the agreed on the
00:13:14
papers to accept. And this because
00:13:17
committee one accepted twenty one
00:13:19
papers that to me to reject it and to
00:13:21
be too except at twenty two papers that
00:13:23
maybe one rejected so that actually
00:13:25
brings it up to forty three papers and
00:13:28
then if we look at the the divided by
00:13:31
the total number of papers accepted we
00:13:33
see that it's actually fifty seven what
00:13:36
things fifty seven percent of
00:13:38
disagreement and and and that that is
00:13:42
not a stunning for me it make it means
00:13:45
that maybe the paper product review
00:13:46
process isn't scientific as we'd hoped
00:13:48
it was so we look at this a little bit
00:13:50
closer then we can argue that fifty
00:13:53
seven percent disagreement is closer to
00:13:55
a purely random to BD what you see on
00:13:57
the right in blue in the case of this
00:13:59
twenty two and a half percent
00:14:00
acceptance rate we'd expect seventy
00:14:01
seven and a half percent of papers to
00:14:03
be reject randomly. Um rather than what
00:14:08
we're seeing with the committee
00:14:09
wouldn't mean or thirty percent. Um can
00:14:13
now no one nice thing about this
00:14:14
basically the model the underlying Mall
00:14:16
is that there's a noisy scoring model
00:14:18
where there are say papers that are
00:14:20
except it's a clear rejects. But the
00:14:22
papers in the middle are basically
00:14:23
accepted randomly and we can actually
00:14:26
calculate this like taking the standard
00:14:29
deviation of the between two committees
00:14:32
and dividing it by be standard
00:14:34
deviation of the scores within a
00:14:37
committee and in this particular case
00:14:39
we see that it's two standard
00:14:40
deviations apart so the the red the
00:14:43
centre one describes what what happened
00:14:45
so this is something that you can do
00:14:47
without necessarily having to pull the
00:14:49
resources together. And this is
00:14:51
something that I actually recommend we
00:14:53
do for colour either we do the full
00:14:54
experiment or we start calculating the
00:14:56
standard deviations. And look at a more
00:14:59
scientific statistical way at a process
00:15:02
that is supposed to make or break
00:15:03
scientific careers. Um recommendations
00:15:07
for all but high and we Christine I had
00:15:09
a conversation about making all square
00:15:11
kind all all the colour and a person to
00:15:15
be brave to submit themselves to the
00:15:17
current open common process I think
00:15:19
this is an issue it may reduce risk
00:15:21
taking and encourages of a papers so I
00:15:26
would like to see a return to the
00:15:27
rejected work to remodel for all the
00:15:29
high with or what without tribal skate
00:15:32
statist but I would I would definitely
00:15:34
volunteer to be chair again we would
00:15:36
move to what we the PDF style editing
00:15:38
model where we don't review but we
00:15:40
actually just change the paper you
00:15:41
change the things we don't like in
00:15:42
another people can change back they
00:15:43
don't like it and somebody wins in the
00:15:45
end in everybody gets their author ship
00:15:47
added to the list and I don't think
00:15:49
that's ever been on a conference and if
00:15:50
anyone is brave enough to do that I'll
00:15:52
volunteer to take on that process. And
00:15:55
regulations for chi reviewing well this
00:15:57
is a difficult thing I don't wanna play
00:15:59
knowledge here but there are some
00:16:00
suggestions here. I think observation
00:16:02
number one we just don't have enough
00:16:04
knowledgeable reviewers and how many
00:16:05
reviews do we need you know in the in
00:16:07
the thousands and and and it's just not
00:16:09
keeping up with that I tenants even
00:16:11
there's like three thousand people
00:16:12
here. So who other two thousand
00:16:13
reviewers that's rejection has been
00:16:16
implemented which is yea so that means
00:16:19
that not every paper will be fully
00:16:20
review which means we can reduce the
00:16:22
number of reviewers and have a better
00:16:23
process but I would go go further than
00:16:25
that and suggested we go to an initial
00:16:28
abstract submission process where
00:16:30
September fifteenth you would submit an
00:16:32
abstract and then October fifteenth is
00:16:34
the deadline if you get invited to a
00:16:36
metaphor paper that reduces the burden
00:16:38
on the office a little bit just a
00:16:40
beautiful paper. Um I would like to see
00:16:43
a bigger role for talk I another
00:16:45
journal papers so that we have a longer
00:16:47
process that is more scientific to
00:16:49
getting to chi with the multipath
00:16:51
review. Um I would like to see attract
00:16:54
dedicated hardware and systems there's
00:16:56
a lot of time of hardware at high but
00:16:58
it would be nice if we just knew that
00:16:59
there was this thing and it's a track
00:17:01
and we submitted that rather than and
00:17:03
not knowing what comedians up in we
00:17:07
need to maintain acceptance levels at
00:17:09
twenty five percent or higher anything
00:17:11
lower missus disruptive papers. Um and
00:17:15
we may be able to accommodate for
00:17:16
submissions by going to shorter talk
00:17:18
size oh well that has its limitations.
00:17:21
And then as I said I think I should
00:17:22
measure expected randomness the within
00:17:24
paper between paper varies between
00:17:26
reviewers course so that's my yeah my
00:17:29
presentation said yeah great think
00:17:43
maxwell. Um so so it's so bring up the
00:17:48
slides okay I was enjoying blue Alt
00:17:51
slide okay. So so roll nicely emphasise
00:18:01
what the history product ideas where it
00:18:04
is now and what what its future could
00:18:08
be so I I was all talked I chair after
00:18:12
role and he gave me instructions about
00:18:16
what what a jury how to do the cheering
00:18:18
infuriating. And it was very important
00:18:21
in that what he was saying was to me
00:18:24
was you know when you're looking to the
00:18:26
rejects you should be looking for
00:18:28
things where you you have a score where
00:18:31
somebody's like five this is the most
00:18:33
amazing paper and see anything like
00:18:34
that's a I for a long time it's
00:18:36
wonderful and somebody else going what
00:18:38
a is of john this paper as I can't
00:18:40
believe somebody what's met such jack.
00:18:43
And so you have these two opposite
00:18:44
extreme scores. Um if they were all low
00:18:47
scores. There's probably a good reason
00:18:50
for that if they're all high scores
00:18:52
that it would be accepted so so you get
00:18:55
this batch of papers that are quite
00:18:57
controversial and so that's really what
00:19:01
the process for me was is starting to
00:19:03
look through those and say you know
00:19:06
let's look at this from the perspective
00:19:07
of somebody attending the conference
00:19:10
what's going to steer emotions what's
00:19:11
going to make people think a lot what
00:19:13
are people going to get inspired from
00:19:16
and and really do look at why that
00:19:19
person was so excited about the paper
00:19:22
and also what was upsetting somebody so
00:19:25
much that they would go to wal that's
00:19:27
trouble the really trash the paper just
00:19:29
you know we all have limited time. And
00:19:32
that's so that really is why I felt
00:19:36
that when we had the sessions and all
00:19:38
that five it was well like it was
00:19:40
standing room only and people were
00:19:42
pissed off that those papers like what
00:19:44
kind of crap was that and other people
00:19:46
like that was fantastic and that's what
00:19:48
I thought was great for high. That's
00:19:51
what people remember that's what people
00:19:53
talk about and that's what you take
00:19:55
bacterial out to your students to your
00:19:57
colleagues to say you know I saw
00:19:59
something really interesting. And what
00:20:03
are we doing that's interesting you
00:20:05
know what are will what boundaries are
00:20:06
we push here. Right and so that's what
00:20:10
I thought the cure asian model really
00:20:12
excelled is capturing that excite and
00:20:17
it was shown in the the current model I
00:20:23
personally am not over early that I I
00:20:27
also agree appreciate the artistic
00:20:29
process of federation there is
00:20:31
something to a lot of value in that and
00:20:34
that it's appropriate at times to have
00:20:36
that model where's the democratic Mall
00:20:39
isn't always the most effective for a
00:20:42
community to move forward. So I'm still
00:20:45
very much in favour that it for finding
00:20:48
those exciting moments and males as
00:20:50
elements. Um so my favour that how that
00:20:56
could be done. There's a couple
00:20:57
different models that that you know try
00:21:01
to make sure that there's not too much
00:21:03
of a concentration in that and say one
00:21:06
chair for example. And spread the load
00:21:08
a little bit and spread the ideas a
00:21:10
little bit. So there are ways of doing
00:21:12
that. Um but I think we're all your
00:21:14
last points were very important to me
00:21:16
to which is you know it's not exactly
00:21:21
so much for all eyes going. It's
00:21:24
actually where kinda going right off
00:21:27
that car is really a reflection of
00:21:29
response to some of the things that we
00:21:31
might be missing in our conference. And
00:21:34
in our communities. Um I I also
00:21:39
typically publish another fields beside
00:21:41
sky where they're much older. They can
00:21:45
they have journals that are over a
00:21:46
hundred hundred some years old. And and
00:21:49
when they start out they have
00:21:51
conferences that are you know lots of
00:21:54
new activities lots of great ideas and
00:21:57
I started out that way. But then what
00:22:00
happens is it matures. And we have a a
00:22:03
doctrine and a process and we have a
00:22:06
body literature that's really
00:22:08
important. And we need to incrementally
00:22:10
grow that and papers need to reference
00:22:14
stores and and and go through very
00:22:18
strict review process because it's
00:22:20
harder and harder to make contributions
00:22:23
in those areas but it's contributions
00:22:25
really really is a major accomplishment
00:22:28
even if it's a small control seemingly
00:22:30
small contribution. But that is a very
00:22:33
strong peer review process a multi
00:22:35
iterative and what's so ironic it's
00:22:37
highs all about iteration and in the
00:22:40
process to really improve and refined
00:22:43
things. And yet are conference process
00:22:46
is not an iterative lets you copy
00:22:49
submitting year after year iterative a
00:22:51
fast iterative review process to
00:22:55
improve those papers that show a lot of
00:22:57
promise but might have some small pause
00:22:59
to get them into our literature and
00:23:01
that's what a journal type of process
00:23:03
is all about is an iterative process
00:23:06
and with online publishing this notion
00:23:08
that it takes a long time to get
00:23:10
published is not true in fact the
00:23:13
journals. Um some of the journals I
00:23:15
work with their publishing speed is
00:23:18
much faster than a annual cycle. So you
00:23:21
can get your papers out much faster
00:23:22
than in the conference. And so I argue
00:23:26
we maybe at a time in our community
00:23:28
where we need to think about iterative
00:23:31
review processes for are really
00:23:34
important papers that add to our body
00:23:36
of knowledge and a separate process
00:23:40
maybe isn't as strict that actually
00:23:42
makes our community push the boundaries
00:23:47
in ways that maybe are not quite ready
00:23:49
for prime time. But give a lot of us
00:23:52
room to talk about and to dream about
00:23:54
what we're here together face to face
00:23:58
to push those ideas. So I think that's
00:24:00
really to talk about the future of dark
00:24:02
I I don't see how we can ignore talking
00:24:05
about the future of cod. And what we
00:24:08
want this conference to be and how our
00:24:11
review process to be and how we want
00:24:13
the external world look better
00:24:15
literature. So that might yeah yeah Oh
00:24:28
yeah by and one million Lars Erik at
00:24:31
that okay we introduced this open
00:24:34
reviewing system and and actually some
00:24:37
of them would even store doesn't exist
00:24:39
or not but I'm gonna talk about email
00:24:44
discussion troopers for years ago
00:24:46
between the posters on this object of
00:24:51
the email exchange words should we
00:24:54
cable oh okay okay what in human beasts
00:24:59
we was such a thing I was one of the
00:25:03
argues that we should kill chi. We lost
00:25:06
the argument perhaps kind of on at
00:25:10
least what hair some of the points that
00:25:11
were made in argument. it's important
00:25:16
to differentiate between things which
00:25:18
are enjoyable. And things which make
00:25:20
intellectual contribution to the
00:25:22
community for it sample having at
00:25:25
children's exterior I do a conference
00:25:28
it's enjoyable might not be
00:25:30
intellectually contribution to the
00:25:32
community. And times well alright is
00:25:35
hope you are an exciting what does it
00:25:39
mean for the contributions of next to
00:25:41
the community there are many strong
00:25:43
strong papers all kind of important to
00:25:46
me what happens to those arguments do
00:25:51
they get fed into the intellectual life
00:25:53
the community of course some of them
00:25:54
are other reference rates on not okay
00:25:57
papers are busily much more course. And
00:26:00
full contributions to cool down
00:26:03
normally that most importantly there
00:26:06
just simply read with the same timeline
00:26:10
conference papers started balls. So the
00:26:14
danger of all time is it becomes a cake
00:26:18
because again to for important ideas
00:26:20
that should be in the mean program and
00:26:22
it's an easy way of saying let's just
00:26:24
put that whole obviously for I
00:26:28
underline again I don't mean this is
00:26:30
any disrespect to the contributions and
00:26:32
ah try that in very many other
00:26:36
conference and pretty good
00:26:38
contributions being a whole time. But
00:26:40
is all about kind a springboard for
00:26:42
those ideas or a grave Thank you I yeah
00:26:53
I signed in yellow Ross there I guess I
00:27:05
should never expected this but this is
00:27:07
certainly can be a panel of very
00:27:10
different thinkers up here and
00:27:11
different opinions about all that chi I
00:27:14
have to be probably the the this mostly
00:27:18
resends the the people kind of co
00:27:21
directing it the the track and also
00:27:24
probably the biggest advocate fits
00:27:26
current instantiation. So I doubt Alt
00:27:30
com what it is that I want no walk
00:27:35
through some of the arguments for why I
00:27:36
love it and what what I think we could
00:27:39
do to kind of continue what we what we
00:27:41
started here. Um just for a some brief
00:27:45
background I started this all only two
00:27:50
thousand and thirteen this is this is
00:27:52
when it was being picked up and by
00:27:54
Amanda williams the fabulous kind of
00:27:57
rendition of what what all that can be
00:28:00
and and ran it for two years thinking
00:28:04
with a lily running in my second year
00:28:06
which is last year and passing it off
00:28:09
the next chairs which continued to grow
00:28:12
these ideas that I would I would give a
00:28:13
lot of credit to Amanda for so so the
00:28:17
backup oh I as we just learned was sort
00:28:22
of form originally as this kind of
00:28:24
controversial space for for papers that
00:28:27
that you know might not fit the
00:28:28
traditional track and and over the
00:28:30
years it can became a space for for
00:28:32
almost saving certain papers right
00:28:35
saving papers that might have been
00:28:36
rejected from client that could've been
00:28:38
like we envisioned and and thought of
00:28:40
in in interesting anyways. And also it
00:28:43
in a in a sense we came out kind of a
00:28:46
space for maybe considering that these
00:28:49
were like the calibre right of the
00:28:51
papers that were in that main
00:28:53
proceedings. So there was a sort of
00:28:55
thinking of like what what we gain from
00:28:57
taking papers that you know had been
00:29:00
submitted to to abstract with regrets
00:29:03
reviewing process we submitting them.
00:29:05
And to something with less rigorous as
00:29:07
we heard just said what what we
00:29:11
considered to be sort of less strict
00:29:13
right. Um and what I see as having
00:29:19
happened is actually and introduction
00:29:21
of that and equally regress or
00:29:24
complementary process in the opener
00:29:27
viewing that very just described and in
00:29:30
the addition of a cheering by edge a
00:29:33
body of yours who's who then come
00:29:36
together across very different
00:29:38
disciplines and begin to think about
00:29:40
the conversations that event happening
00:29:42
online openly. And develop the the
00:29:47
track that you see here today and I
00:29:49
just wanted points out that this is
00:29:51
actually a piece that we can improve on
00:29:53
to to make that part transparent in
00:29:55
some of the amazing voices that'd been
00:29:57
doing that territorial work. Um I'm
00:29:59
just gonna read some of the names that
00:30:01
you might be familiar with so this is a
00:30:03
list of the Churchill Jennifer rocker
00:30:06
hall and Jeffrey bowers Al showing
00:30:08
parse out Alex Taylor mark grows what I
00:30:10
could tell me where ram S I mean daring
00:30:14
gurgle terry Leningrad Brenda laurel
00:30:16
and he dear in your then Paul there
00:30:19
irrespective Grinch or make it well
00:30:21
Jennifer Tennessee jen main often
00:30:23
costly Christine satchel Steve Jackson
00:30:27
call this out about you see such men
00:30:29
they'll go work often just just to have
00:30:32
a K all incredibly well known to our
00:30:36
community in diverse ways. Um and these
00:30:39
are some of the people that have been
00:30:40
contributing to this process and in
00:30:42
ways that I think compliment some of
00:30:45
the the ongoing discussion online and
00:30:49
what I one do is actually talk about
00:30:51
sort of how we've thought to you we
00:30:53
figured some of these ways in which
00:30:55
this discussion is happening in the
00:30:56
last two years to make this more about
00:30:59
the kinds of exciting conversations
00:31:00
that cross that prospect I community.
00:31:03
So that our goal isn't so much to say
00:31:07
presents an alternative right as Alt
00:31:09
might apply to what exists but actually
00:31:12
show that the edges are the places
00:31:15
where the community is going. So yes we
00:31:17
recognise our roots but we also points
00:31:19
the way forward. And so as as a try has
00:31:24
has continued and some of them are most
00:31:28
provocative work that we found. Um
00:31:30
really becomes number. And and valuable
00:31:33
kind of compliment to that to the the
00:31:38
scholarly debates that happened on the
00:31:40
reviewing sites of these we've we've
00:31:42
described that last year was the first
00:31:45
year to propose this kind of
00:31:48
publication format called commentary
00:31:50
and then what what the thinking was
00:32:14
here is the as the the right community
00:32:16
can grow the numbers that these these
00:32:18
spaces for interdisciplinary exchange
00:32:20
are just becoming increasingly crucial
00:32:23
to us so how can we grow that's right
00:32:25
with the commentary. And then you
00:32:29
alongside that we have this idea that
00:32:33
there are existing programs like high
00:32:36
that allow for this kind of
00:32:38
conversation to happen user that panels
00:32:40
right but the controversy controversial
00:32:43
papers maybe that have that have the
00:32:45
need for multiple perspectives to
00:32:47
participate. And indeed I I think that
00:32:49
all kinds extended what the panel
00:32:51
started but in a way that's really
00:32:53
particular in that sense it is not just
00:32:56
a round one theme but then crosses not
00:32:59
just topics but also perspectives that
00:33:02
contribute to this diverse community.
00:33:04
So we've seen a lot of the paper are
00:33:06
the tracks at this at this year's tie
00:33:09
that all kind does sort of bring these
00:33:11
kind of paper stick yeah there that I
00:33:13
don't normally spoken together
00:33:15
sometimes off and the conversations
00:33:17
continue after the paper is to
00:33:18
liberate. Um so in this way I I argue
00:33:22
that old high has and its the reach
00:33:26
that kind house that can extend the
00:33:30
current spaces for interdisciplinary
00:33:32
exchange also brought in the project of
00:33:35
patients that ended voices that can be
00:33:38
brought into these these conversations
00:33:41
and then the scale of this at the
00:33:42
setting has also expanded over the
00:33:44
years. It impact for three has brown
00:33:48
commentaries are archived and say in
00:33:52
the commits even cited right as
00:33:54
archival scholarly work. And lastly I
00:33:57
think it's it's really grown our
00:33:59
community so we brought in new voices
00:34:03
in in the ways that we normally have a
00:34:04
lot of struggle to do in in the
00:34:07
standard contract. So moving forward I
00:34:10
think they're trying two ways that we
00:34:12
can envision that that the future of
00:34:14
this track that I think would actually
00:34:16
build an interesting way is what we
00:34:17
started one is the idea of pushing all
00:34:22
high to the proceedings what happens
00:34:25
when we recognise this work as equally
00:34:28
archival. And I don't know if this is
00:34:30
the right tack to take because I
00:34:32
recognise their aspects that are lost
00:34:34
right. But I think that this is
00:34:36
something to consider deeply is how
00:34:38
much we value right the work that that
00:34:40
has kind of come into its its own in
00:34:43
the states the second area is actually
00:34:46
kind of the reverse what can we do to
00:34:49
context like that this is a kind of a a
00:34:51
round to experiment in some ways with
00:34:54
the you're the curate oreo voice so
00:34:56
exploring alternative modes of
00:34:58
circulating content and I'm publishing
00:35:00
it. And that really has to do with
00:35:02
maybe the addition of other mechanisms
00:35:05
for allowing the the projects that we
00:35:08
produce to change over time right what
00:35:10
are kind of ways we could explore
00:35:12
actually be so it iterative publication
00:35:15
forms where where the archive isn't
00:35:17
static and perhaps more modelling a
00:35:20
paper mode in complementary to that is
00:35:23
the reviewing and commentary as being
00:35:25
also sort of envisioned as a non that a
00:35:28
stable form. So I'll leave you with
00:35:30
those two and we can discuss things.
00:35:33
Thank you so much and yellow so I'm
00:35:41
just to recap we have a multiplicity of
00:35:44
viewpoints here in the tradition of all
00:35:46
chi and some very provocative arguments
00:35:49
I hope that it's given you sow the
00:35:51
seeds for thought we can have a a
00:35:52
discussion now and just as a bit of
00:35:54
framing next year's chairs by their own
00:35:57
admission are very old chi friendly and
00:35:59
are very interested in potentially
00:36:01
experimenting project potentially re
00:36:03
imagining the track they're very
00:36:06
interested in the track we've had some
00:36:08
ideas around that we would really love
00:36:09
to hear your ideas is there an issue
00:36:12
with with the open reviewing process
00:36:14
being a popularity contest what do
00:36:16
people think of the commentary does sac
00:36:18
give a chance to for these to be more
00:36:19
of a discussion on does all to
00:36:21
marginalise too much is that a problem.
00:36:24
Um and and how do we make all to count
00:36:27
in what ways. Um so with that please
00:36:31
come to the microphones and we will
00:36:34
have a discussion sorry which of the
00:36:51
it's a five minutes copy from Indiana
00:37:05
university I just kinda sorta slowly
00:37:08
wanna push back on this idea of car yes
00:37:11
this all cast the spectacle and
00:37:13
entertainment value only at there have
00:37:15
been really really important
00:37:17
contributions to the field for example
00:37:18
the preponderance T patterson's still
00:37:20
three days apparently started does not
00:37:22
quite paper it moved into a big one
00:37:25
Jonathan asset melts in fact on this.
00:37:27
So yes there are some be some cats and
00:37:30
dogs which are usually get a lot of
00:37:31
reviews but I also want to make sure
00:37:34
that when we talk about alternately S
00:37:36
this too much or too little be whole
00:37:39
issues that who gets to decide these
00:37:41
things when you call epistemological
00:37:43
questions matter records and stuff like
00:37:46
that. We do need those of spaces and we
00:37:48
can just dismiss them as less time to
00:37:51
make you probably shouldn't be for
00:37:53
there are other ways of doing reviews
00:37:55
that's so I just kind of slowly one
00:37:57
push back at all classes you know it's
00:38:00
basically just don't play around it
00:38:02
it's not serious research. And I think
00:38:04
that's a skew to be and it was like it
00:38:09
was an exceptional person well my
00:38:13
comment is I I'm not sure if that was
00:38:14
actually in the early tracks I think it
00:38:15
was was it two thousand six or
00:38:17
something I was later and that was very
00:38:21
much what I propose is that is that you
00:38:23
know the we could submit these same
00:38:25
papers if they're not archive will have
00:38:27
to work really hard and then battle
00:38:29
makes him a little bit for those people
00:38:30
not to get hurt I because they wanna
00:38:33
archive everything and and the whole
00:38:34
idea was that they would later go
00:38:36
through some other form of of a process
00:38:38
that was trying to take and this was
00:38:40
just basically a promotional platform
00:38:42
for that kind of thing to happen. And I
00:38:44
I think that that original model
00:38:46
stronger in that sense I mean we really
00:38:48
got fifty percent of the papers
00:38:49
accepted in a in a regular academic
00:38:51
platform and it's a way that I hope
00:38:54
look like into and make it more
00:38:56
relevant archival in scientific I think
00:38:58
that that that works were quite quite
00:39:00
naturally that's a good example of that
00:39:02
no no oh alright what I'm saying is
00:39:13
that if these papers would be rejected
00:39:16
in any journal any and the car and
00:39:17
everywhere like there has to be a place
00:39:20
for them but then they start a new
00:39:22
life. And then they can go through the
00:39:24
process again and so I'm not proposing
00:39:26
little but like this that process
00:39:27
proposal at all the close the
00:39:29
promotional element that then starts
00:39:31
discussion it last paper to get
00:39:32
accepted I alone other problems with
00:39:36
the machine you know that's that's
00:39:38
that's another you can build other
00:39:40
machines. But realise you're more the
00:39:43
machine room. a long time. So these
00:39:46
other machines other reviewing systems.
00:39:49
They're marginal So I'm reformers
00:39:52
performing machine hello and I'm very
00:39:57
quickly and this is the second time
00:39:59
I've been to the old car and there is
00:40:01
like a default was "'cause" last right
00:40:02
attended and I so I'm an exceptional
00:40:04
talk that inspired me and I talked to
00:40:06
one of the old version enough talk
00:40:08
involved in at this you Piper I've many
00:40:11
things to say about this focus on one
00:40:13
thing which is the open review process
00:40:15
I'm not under percent sure that works
00:40:17
as as well as you people might think it
00:40:20
does this we're very proactive and
00:40:23
asking offers we critiqued to pretty
00:40:27
costs. And I don't think they were as
00:40:30
not honest I don't think they were was
00:40:33
a skating or as frank as they would
00:40:36
have been if their names have been
00:40:38
associated with what they were saying
00:40:40
to us and the fact they also not what's
00:40:42
and hopefully like us to some degree.
00:40:44
And and I'm not gonna time picking
00:40:46
anyone but because one of the people
00:40:48
who we work we reviewed we asked didn't
00:40:50
write a review. Um all those I'm
00:40:53
curious if it's the person if they want
00:40:55
to say why they didn't "'cause" maybe
00:40:56
it was particularly it's maybe was
00:40:58
really bad paper the real injection
00:41:00
contrition meister time did it might be
00:41:02
mean maybe just have the time. And I
00:41:05
think it the openness is is great but
00:41:08
even like remember one of the authors
00:41:09
my mighty said you know this papers
00:41:11
come crap what would we have the
00:41:13
enormous intellectual confusion but who
00:41:15
wants to be nice who wants to say nasty
00:41:16
things to nice people so I know what's
00:41:18
the point. So you really think about
00:41:20
the openness of that's because you're
00:41:23
you're getting into the whole social
00:41:24
dynamic people who wanna be nice when
00:41:25
people "'cause" they like them middle
00:41:26
class and you know I can an honest
00:41:29
assessment. I agree with your point I
00:41:32
think you can accept stuff it should
00:41:34
have some sort of impact it should have
00:41:36
some sort of springboard trajectory
00:41:38
just coming and pissing in the wind and
00:41:40
then leaving and going that felt good
00:41:42
is it is useless basically. So I would
00:41:44
actually ask people to put in an impact
00:41:46
statement that it's okay to get
00:41:47
accepted what you going to do not
00:41:49
invest in the don'ts don't know but
00:41:52
just think about so I don't think the
00:41:53
openness works and where you think it
00:41:55
tells many would like to comment on why
00:41:57
they did or didn't feel like it could
00:41:59
really work I first I am I welcome your
00:42:01
way what it might do no you can just
00:42:05
talk off that's good oh okay th oh oh
00:42:32
ah okay yeah okay oh yeah okay "'cause"
00:42:56
I I think what this is I'm really glad
00:42:58
you were able to say "'cause" I was
00:42:59
really curious as to why you didn't
00:43:00
"'cause" you think you'll be bad for to
00:43:02
these great thoughts like yeah stick to
00:43:04
my who really whatever you can you
00:43:06
don't you never know actually came out
00:43:09
the first thing that came out like yeah
00:43:11
I'm irish kilt is part of our DNA and
00:43:13
the thing that came out of those
00:43:15
economy was the community okay let's
00:43:18
all get together pretty well that's
00:43:20
true I I know we want something your
00:43:22
negative that down on them and so we
00:43:24
came up with this idea of having
00:43:26
ignoble warts in I simply suggestion is
00:43:29
to have a noble towards that we come
00:43:30
together we go back and we generate
00:43:33
these ignoble things. And this is
00:43:35
getting traction so there is a there is
00:43:37
a pathway from this could be that
00:43:39
impact of course so that's the problem
00:43:41
anyway a leave it there thank you and
00:43:46
responses I'm I stick it to them and
00:43:52
that's that's so so I am I as is one of
00:43:57
the co chairs for this year and also
00:43:59
next right I I definitely hear your
00:44:01
concerns and I'm not I feel like as a
00:44:05
community this doesn't happen so much
00:44:06
but when I see these sorts of voting
00:44:09
options on the internet more widely it
00:44:12
becomes a popularity contest I mean
00:44:14
these are not democratic they are not
00:44:16
inclusive and I think that is a serious
00:44:21
concern so I'm very happy to possibly
00:44:23
re imagine this for for future years I
00:44:26
mean we we want a my model and but key
00:44:29
innovation with the back to anonymity
00:44:32
we're gonna inspired by the beam
00:44:34
British panic which on which sounds
00:44:36
reviews. yeah to check or something
00:44:40
huge we're just giving a try and see
00:44:42
what happens I would also say that nice
00:44:44
reviews does not necessary mean not
00:44:46
constructive right that there are a lot
00:44:49
of really fabulous a way of of being
00:44:52
polite and giving these sort of in in
00:44:55
new important feedback that would
00:44:57
change I I I add something actually
00:45:01
then So but in all the or five number
00:45:09
of ways my first submission
00:45:11
publications in two thousand Sutherland
00:45:13
papers journals on the Dreyfus here all
00:45:15
so I guess I see as a way of saying I'm
00:45:18
both invested in and supportive all all
00:45:20
pious ever. Um I think one of the
00:45:24
things that's interesting about this
00:45:27
conversation is the way that it raises
00:45:31
broader issues beyond all high as a
00:45:34
particular and I think some of some of
00:45:37
these questions what I where are you
00:45:41
what is it that we I believe the here
00:45:44
review jars for us as an institution as
00:45:48
process there is a way of thinking peer
00:45:51
review as a sort of G right that you
00:45:55
wanna was established anything that's
00:45:57
in it has the proper methodological
00:45:59
doesn't any religious assumptions there
00:46:05
is that Jeff or sell his argue forcing
00:46:10
review alternatively as a process creek
00:46:13
cheek that accomplishes different sorts
00:46:16
of discourse in different ways. And so
00:46:19
I don't know if I have necessarily
00:46:21
stance that a wanna make but what I
00:46:23
draw that out as an important thing to
00:46:25
consider if we're thinking about future
00:46:29
iterations of all five changes that
00:46:31
wine the what is it that we think the
00:46:34
peer review all close as you know
00:46:38
otherwise whatever form what is it that
00:46:41
you want that pure you to accomplish
00:46:43
its well I do have a thought on that
00:46:47
which is doesn't really address what
00:46:49
you're saying it's more sort of
00:46:50
statistical observation is that I I I
00:46:54
think that the purview an argument is
00:46:56
very much seems a gatekeeper certainly
00:46:58
if you're if you're any see that seems
00:46:59
to be out works. I don't one of the big
00:47:01
whether that's a good idea or a bad
00:47:02
idea what's about critique or not what
00:47:04
I'm arguing is that it's way more
00:47:06
random then we think in that as a
00:47:07
problem. And we're just not as a
00:47:10
community we hide that so when you get
00:47:13
you card be rejected you bitching you
00:47:15
say oh it's terrible this year's
00:47:17
terrible I'm not gonna show up whatever
00:47:19
and if you get a paper accepted your
00:47:22
liking and it's like yeah you know see
00:47:25
I don't Mckay paper exactly it can be
00:47:26
both ways okay and so so what that
00:47:29
tells me is that there's a noisy mess
00:47:31
in particularly in the middle papers in
00:47:32
that process and for me although I very
00:47:35
much could serve and certainly in the
00:47:37
beginning use did serve as a means of
00:47:39
reducing that annoys in that process in
00:47:42
the middle papers and those middle
00:47:45
papers could be papers with the one on
00:47:46
the five and and about three a half
00:47:48
with large standard deviations but some
00:47:50
arguing what's weird is that noise
00:47:51
process and let's focus on applying
00:47:53
some of the signs that we're trying to
00:47:55
promote in our community too problem
00:47:56
processes like like all right. So what
00:47:59
was what the processing it's no would
00:48:01
we not agree that we would want the
00:48:03
less noise in it yeah I mean you can't
00:48:08
know when we look at other other fields
00:48:11
that have to address the same issue I
00:48:13
go back to the journal process where
00:48:15
the review the peer review process
00:48:17
there. Um is iterative is trying to
00:48:21
make sure that it you know what it's
00:48:24
you know into the doctrine of field.
00:48:27
And it just needs to go through
00:48:29
generations to make sure that happens
00:48:30
that doesn't mean the ideas no good it
00:48:32
just means somebody needs to make sure
00:48:34
the string and it up. So that when
00:48:37
we're all long dad and somebody in the
00:48:40
future wants to go back and look at
00:48:42
that they can trust that it really is
00:48:44
get a really it it. Um and then so then
00:48:50
what's the point of a conference like
00:48:52
what's peer review for conference all
00:48:53
of that and once you go into the into
00:48:57
these fields where the journal is where
00:48:58
the gate keeper of the knowledge as and
00:49:01
what's the point of a conference here.
00:49:04
And in fact a lot of them are very they
00:49:07
don't have a strong peer review because
00:49:09
the peer review there is to make sure
00:49:12
things that are interesting for your
00:49:13
colleagues that will get them talking
00:49:17
and make sure we're all aware of what
00:49:19
each other doing. And have an
00:49:21
opportunity to have that moment to be
00:49:24
face to face and talk about ideas. But
00:49:27
it's not a place that type of peer
00:49:29
review is not need to be the geeky. And
00:49:33
so you can have different types of peer
00:49:34
review and all that high and it is
00:49:36
about reducing the noise in the system
00:49:38
for sure and and one way of reducing
00:49:40
voices by having them an intricate
00:49:41
process multi pass work you exact words
00:49:43
that's one way of doing it yeah and so
00:49:45
we do things like rebuttal periods and
00:49:46
try and mimic the review process but
00:49:49
it's like somebody said it's you know
00:49:50
the process that's been going on for a
00:49:52
long time become process I would argue
00:49:55
has not been going on for that long of
00:49:56
a time. It's the journal processes that
00:49:59
have been going on how much much longer
00:50:01
period of that process not suggesting
00:50:03
we necessarily adopt exactly that but
00:50:06
it is that there's different types a
00:50:07
review processes in pure processes and
00:50:11
all time high just was part of one of
00:50:14
these that played a very important
00:50:16
role. And we really are missing that
00:50:18
other part I think and it's and we're
00:50:21
just seeing that it's not working
00:50:23
conference process so well with the
00:50:25
twenty five percent with so much noise
00:50:27
nips is going through this other
00:50:29
conferences go through this that the
00:50:31
noise in the system is so large that
00:50:33
the middle papers they're effectively
00:50:35
around. And that doesn't help I mean
00:50:39
that much something yeah you sorry
00:50:44
anyone say something not that so just
00:50:48
then I have from moment like fucking
00:50:50
teachings we can become so I really
00:50:52
enjoyed some of the commons to do with
00:50:55
how I feel this developing and trying
00:50:57
to figure out how to do an analysis of
00:51:00
what these different forms what roles
00:51:02
that concerned not only in terms of
00:51:04
what what is good peer review system
00:51:06
but also about Clinton or conference.
00:51:09
And so when I carry fringe in two
00:51:11
thousand and two or three are can't
00:51:13
remember a long time ago it was because
00:51:15
I was also trying to get the sign into
00:51:18
time and at the time it was the only
00:51:21
way to do that was to have a really
00:51:23
cool assignments slap on a really
00:51:25
crappy uses to go beyond up the paper
00:51:27
and then you would get in even really
00:51:29
crappy design would work with that uses
00:51:31
that yeah yeah so we should do a
00:51:35
slightly different analysis today what
00:51:37
is issues were struggling with today
00:51:40
and my feeling and I might be wrong and
00:51:42
we should probably do remotes of
00:51:43
analysis of this before we we make any
00:51:46
anymore hypothesis is that we are now
00:51:48
if feel that all way more of these some
00:51:52
disciplines and that is getting to be
00:51:54
huge problem to us because we are
00:51:57
getting reviews from people were I
00:51:59
don't know the sinus reviewing you know
00:52:02
a more structured empirical studies and
00:52:06
so on and so on so I think I I agree
00:52:08
with with battery that we need to think
00:52:10
about the intellectual capital that we
00:52:12
are creating and how to deal with that
00:52:15
situation what is the role or all okay
00:52:18
what is the role of the time reviewing
00:52:20
system and posturing that meeting or
00:52:24
perhaps there is no meeting then how do
00:52:27
we deal with that that I think it's an
00:52:29
interest many to jump in here will
00:52:34
create a whatever questions you guys.
00:52:36
So I think years Reading something
00:52:38
really important we've been focusing a
00:52:40
lot on the reviewing process far but
00:52:42
you you just brought up content to and
00:52:44
one of the questions that we had in
00:52:47
mind also for for the panel is what do
00:52:50
you think the state of criticality
00:52:52
actually is with records all start high
00:52:55
and if is that something more outcry
00:52:58
also should be headed in in the future
00:53:00
and what is it what kind what is all to
00:53:03
mean that what what could it mean again
00:53:05
in the future with specifically with
00:53:08
with regards to kind of scholarship
00:53:11
that we want to attract to it and the
00:53:13
kinds of conversation we want to have
00:53:15
the dozens of of course in in part
00:53:17
enacted through the peer review process
00:53:19
of but I think it would be really an
00:53:21
interesting conversation to have what
00:53:23
are the contributions also I'm telling
00:53:26
us about the kind of critical
00:53:28
conversation we have in each the I and
00:53:31
in and that high I was actually gonna
00:53:37
pose that back and to the community
00:53:39
right here are people who are invested
00:53:41
in all kind and Wayne just ask to what
00:53:44
who in this audience have a show of
00:53:46
hands believe that all high actually
00:53:49
had had been able to attract multiples
00:53:53
that disciplines within one you know
00:53:55
one session who here has seen okay okay
00:54:02
so maybe have maybe like okay I mean I
00:54:05
think these are actually the things
00:54:06
that we try to cultivate and that
00:54:09
that's getting directly to your point
00:54:10
of a set of that that the that the
00:54:13
emphasis is on the this the interaction
00:54:16
between each group that generally do go
00:54:17
to the you know separate tracks are
00:54:19
separate fashions. And that that
00:54:21
engages different kind of criticality
00:54:24
than we would normally experience and
00:54:26
the and the traditional track yeah I'll
00:54:28
say despite my criticism of the of the
00:54:30
current process being kind of the
00:54:32
popularity thing with with with
00:54:34
commentaries online I do think that
00:54:37
what has emerged is that all the guys
00:54:39
very much become a conscious of the
00:54:40
conference. So yeah I know we do these
00:54:43
signal I town all meetings I never go
00:54:45
to them I do go tall dark I and I think
00:54:48
that all the chi has a really has
00:54:50
become a really valuables place for
00:54:51
doing exactly what we're doing now
00:54:53
which is talking not so much about sort
00:54:56
of the the day to day management of
00:54:58
sick I or anything like that but
00:54:59
actually the content and and we we we
00:55:02
are seeing probably the most
00:55:04
multidisciplinary discussions within
00:55:06
all talk like today and and I usually
00:55:08
appreciate I propose a probably less to
00:55:15
discover I joined and so what kind two
00:55:17
years ago one it is very yeah I'm very
00:55:21
impressed about to there that
00:55:22
discussion after the final two after
00:55:25
the presentation because that and I I
00:55:28
just basically I think that compress
00:55:30
the body of the copyright C two joining
00:55:32
on getting each other but in the coming
00:55:35
in computer science field that's a to
00:55:37
conference the bicycle because so that
00:55:40
someone six that used to different
00:55:42
projected that there's oral
00:55:43
presentation but it's okay it's but
00:55:45
it's alright but but I have the sum and
00:55:48
binding for several mean communication
00:55:51
present discussion so high scoring one
00:55:54
and maybe a more communication time
00:55:56
it's good for the recognition for
00:55:58
example to taking a more question time
00:56:01
after the presentation well maybe the
00:56:03
technical S support for the recognition
00:56:07
what's up with the age japanese a
00:56:09
compressed with using that see a chance
00:56:12
during the presentation of that
00:56:14
everyone can write code it's this unit
00:56:18
to rest with the comment yeah what does
00:56:21
a presentation so they wouldn't yeah
00:56:23
yeah I think every time I'm coming back
00:56:26
to a dependent on it icky or speaker
00:56:29
will pick up some of the commitment
00:56:32
people have this so it in kind of the
00:56:34
collocation yeah support will be there
00:56:36
for people that maybe someone one and I
00:56:41
will can just for that kind not process
00:56:43
every like how yeah thank you. So yeah
00:56:49
oh one thing no but we failed to do I
00:56:51
think is included technical parts the
00:56:53
community increasingly over here the
00:56:56
last few years then more experimental
00:56:58
approaches I think mean the list well
00:57:01
there contribute is kind of momentary
00:57:05
peace I'm social sign to so whatever
00:57:08
but coming into for for for that to
00:57:11
work so I think it's filled in its
00:57:13
interdisciplinary yeah oh no but there
00:57:20
are still papers like I think and think
00:57:21
we presented some work on like an
00:57:24
interaction and and multi touch last
00:57:27
year yeah I know like I think I think
00:57:29
the the original the original model
00:57:31
faster way more of the sort of average
00:57:35
slice through pick of high then we see
00:57:38
today and I do think that the overall
00:57:39
review process has something to do with
00:57:40
that it's become a bit of a soapbox and
00:57:43
that's why I don't like about the
00:57:44
current all buckeye. But the kinds of
00:57:47
humanities or whatever conversations
00:57:50
that are being had a tall dark card or
00:57:52
in and of itself and disciplinary I do
00:57:54
believe and maybe we're not talking
00:57:56
about P values but you know well and
00:58:00
and I was gonna say one things that
00:58:01
hope that I've that I really appreciate
00:58:06
your pointing out is it really about
00:58:09
conversation about discussion really
00:58:11
encouraging different points of view.
00:58:14
And bring them together and it is very
00:58:16
successful at doing that that and those
00:58:19
there's so much discussion around some
00:58:21
of these things and not a big reason
00:58:23
why we're here so it really six a
00:58:25
independent of the new ones as it's
00:58:28
been extremely successful doing that.
00:58:31
So those so in you know adding to those
00:58:34
for the future ones and finding new
00:58:36
ways of having these conversations that
00:58:38
is part of all the time that can again
00:58:40
transfer eventually because but that's
00:58:42
the place that right now "'cause"
00:58:43
that's so that I agree with you there
00:58:46
yeah I was just yeah hi then you know
00:58:56
wild you know stick something in that I
00:59:00
I think what I wanna do is present
00:59:02
myself as a little bit of a case study
00:59:04
this is my third time I have never had
00:59:07
a chi at paper in the full conference.
00:59:10
Um the first year I submitted all kind
00:59:14
because it seemed a more appropriate
00:59:16
thing you'd from my work and I also
00:59:19
submitted to interactivity actually I
00:59:21
chatted interactivity. And curated the
00:59:25
the first interactivity in austin and
00:59:27
that was the first time I'd ever been
00:59:29
calling and the second year was in
00:59:33
Paris and I also submitted and all kind
00:59:38
type that because it just seemed like a
00:59:40
more appropriate venue for my work this
00:59:44
year I submitted a high paper and it
00:59:46
was rejected and I did not re work for
00:59:49
all combine because it did not seem
00:59:52
like an appropriate venue for the paper
00:59:54
that I was trying to write a haiku
00:59:57
where you work that paper and say that
00:59:59
somewhere else that all backed hi who
01:00:02
knows but what I did was actually take
01:00:05
the the crux of the paper and rework it
01:00:08
into workshop proposal because the
01:00:10
works one of the workshops actually
01:00:13
with relevant to what I was trying to
01:00:16
write about my thought this is great
01:00:18
this will give me a chance to extend
01:00:20
the discussion that I'm trying to have
01:00:22
and deepen my reflection process so
01:00:25
that I can write a paper. I did it it
01:00:29
for me they said two very different
01:00:31
purposes and I didn't grow up in HCIO
01:00:35
computer science. Um I don't have an
01:00:39
undergraduate degree. I have a Masters
01:00:41
in design from the roll college about
01:00:43
an interaction design and I have a PHD
01:00:47
that was done in fine art and materials
01:00:49
science and engineering and contributed
01:00:51
to design interaction design. I'm not
01:00:55
any more unique than anyone else in
01:00:58
that in that unconventional plenty. And
01:01:01
plenty of people who have
01:01:02
unconventional power place and bring
01:01:05
together could converge disciplines
01:01:08
need a place that is relevant for them
01:01:11
to actually not so box they were what I
01:01:15
appreciate that criticism is relevant
01:01:18
in some cases the opinion needs to be
01:01:22
there for the work that the
01:01:23
practitioner that the researcher
01:01:26
doesn't feel fits in the other venues
01:01:29
and that and that's a very important
01:01:31
role that all time has played for me.
01:01:34
And I'm not going to stop submitting
01:01:36
all kind of a pain that that I feel
01:01:38
it's that is the right one forty five
01:01:41
work. Um I see them as to fulfilling
01:01:45
two very different very different roles
01:01:47
in there a little bit different to
01:01:48
what's been discussed you so and I just
01:01:50
wanted to open that out the reflection
01:01:53
could you actually say before you
01:01:55
finish up a little bit about your
01:01:57
reasoning like what what sorts of
01:01:59
criteria why something was or was not
01:02:02
all chi I'm a practise based research
01:02:05
and and my PHD was practised the text.
01:02:08
And the elements that I was trying to
01:02:11
draw around were the elements that
01:02:13
could only be accessed through an
01:02:17
embodied process and then and anybody
01:02:21
processing engaging with materiality in
01:02:23
its broadest sense. And the papers one
01:02:26
not delving deeply into the theoretical
01:02:29
underpinnings. They were really it's
01:02:32
like locating the practise and the
01:02:35
results of that practise and and
01:02:37
reflecting on the experiential nature
01:02:40
of the participatory. Um frameworks
01:02:43
instructions that I was making
01:02:45
available to people and what came out
01:02:47
of that and the paper that I submitted
01:02:49
this year to I went more deeply about I
01:02:54
thought I it's still work to be done
01:02:57
but but the idea was that that it's
01:02:59
intention was to go into a more deeply
01:03:01
reflective theoretically supported
01:03:03
place which I I filled with regard my
01:03:07
discipline to what I say Mike
01:03:08
discipline that's being as relevant to
01:03:12
call I as well as opposed to well time
01:03:15
whereas the other work it's it's just
01:03:17
just theoretical but it's a practise
01:03:18
makes theory it's just as rigorous but
01:03:20
it's a different kind of because that's
01:03:22
applied in different places these on
01:03:23
the in different lines I can I can I
01:03:27
just as a so so I appreciate what
01:03:30
you're saying and I do believe that
01:03:31
that is kind of what's happening is
01:03:33
that the people the paper got all
01:03:34
that's an oxide paper and I've always
01:03:36
battle that notion I I would I would
01:03:38
love to see all dark like be a complete
01:03:41
cross section of the bigger conference
01:03:44
plus what we don't know right so that's
01:03:46
where that fits in but I don't like the
01:03:48
idea that people submit certain kinds
01:03:50
of work tall dark I because it it
01:03:52
becomes all selective right and then it
01:03:54
doesn't how the conference renew itself
01:03:57
and it also doesn't help the process of
01:03:59
becoming part of that's academic
01:04:02
process in the end. And so I think
01:04:04
that's a danger but I do understand
01:04:06
that is what happening in a you know it
01:04:08
has its value but it can it that can
01:04:09
happen even if all that high represents
01:04:13
all papers in the conference plus what
01:04:15
we don't know but can I just say just
01:04:17
clarify one thing quickly heart of my
01:04:21
reasoning as well was that I an
01:04:26
extended abstract isn't worth this much
01:04:29
to me professionally. And that's
01:04:30
problematic. So I can publish
01:04:34
everything in all time because then I
01:04:36
don't get any of the brownie points
01:04:37
that I need to get promoted all
01:04:39
considered for job or to continue to
01:04:43
build my career I you know why what
01:04:46
myself into a position of stasis
01:04:49
professionally. And that is is is part
01:04:53
of the consideration as well it's not
01:04:55
just different venues for different
01:04:58
things but it's also different venues
01:04:59
have different values not just given to
01:05:02
them by the community but imposed upon
01:05:04
us by the funding bodies and by our
01:05:07
employers and all the rest and I think
01:05:09
that's a very big and that is one yeah
01:05:11
these hi Becky grin try a I think
01:05:18
increased slang gauge men with all the
01:05:20
by the time I think I started to make
01:05:23
more sense to me of the more swim
01:05:25
practise I didn't know what it was the
01:05:27
first time it's become clear to me what
01:05:30
could be I will so increased my
01:05:33
engagement well I decided to beat your
01:05:35
parts people but organise it might like
01:05:38
to say thank you to doing it because
01:05:41
it's extra work that we take all right.
01:05:43
And I guess that brings me to what I
01:05:45
want to say no because this is
01:05:47
ultimately about volunteering writes
01:05:49
about volunteering to do work in all of
01:05:51
its forms as we start to talk about how
01:05:54
we might change reviewing process I
01:05:56
think there is an ideal but we can set
01:05:59
up I see a lot the conversations that
01:06:01
are happening across computer science
01:06:03
right about how to make a nice
01:06:07
compliments conferences and things like
01:06:09
that make reviewing better but I think
01:06:11
this was gonna be and in her this that
01:06:14
so I'll I'll draw on the experience
01:06:16
price what most clearly side I agree to
01:06:19
the culture of the papers tracks a is
01:06:22
good but the disciplines might come
01:06:24
papers chair and I spent several days
01:06:28
looking at the previous is the sort of
01:06:31
back through four years of roughly what
01:06:33
kind of paper that where the mix and we
01:06:36
sort of based on that we started to
01:06:38
think about who we would you like to
01:06:40
compose a paper stability that we owe
01:06:42
represented the expertise roughly in
01:06:44
the amounts you might need to have it
01:06:47
you know it's not an exact science
01:06:48
right that this war writers for for
01:06:51
troll but nothing prepared just of the
01:06:54
trouble that was gaining his just
01:06:56
exploded in popularity oh so we had not
01:07:01
planned on having eight people for the
01:07:04
gaming section I think it was the year
01:07:05
we got can in just went crazy and so
01:07:09
you you call contradict right and you
01:07:11
can't predict over a group of people
01:07:14
that are going to volunteer at all
01:07:15
sorts of complexity so as we start to
01:07:17
think about what we want in a reviewing
01:07:20
process and I think it's the the
01:07:22
perhaps the trouble is most rightmost
01:07:24
papers that end up in the middle I
01:07:26
don't want to completely eliminated
01:07:28
because it's gonna be an inherently
01:07:29
subjective process that's not only the
01:07:32
based on who would breeze to do well in
01:07:34
any particular here. But as well sit
01:07:37
here at this conference thinking about
01:07:39
what we might do in the next year we
01:07:41
may be moving interactions that just
01:07:43
calm as a complete surprise you divide
01:07:46
individually but it isn't it so you see
01:07:47
the some of the the the whole white and
01:07:51
you never Cecil because you only have
01:07:52
to see the popular stuff right you
01:07:54
don't see all the other things that
01:07:55
come in not year you just call predict
01:07:59
what kinds of expertise you're ever
01:08:01
going to the so I just want to sort of
01:08:03
course in the note in a assuming a
01:08:05
perfect ideal as opposed to sort of
01:08:07
excepting that that's just gonna be
01:08:08
some of this is the reality set not
01:08:10
calls appropriate yeah I definitely
01:08:15
agree with that point you know one one
01:08:17
way to address that and I know this is
01:08:20
gonna cause issues but if the
01:08:22
acceptance rate is say fifty percent or
01:08:25
sixty or seventy percent of the papers
01:08:28
that come and that will address some of
01:08:31
these issues. And I know that sounds
01:08:33
crazy like I made my career and by
01:08:36
seventy percent that's yeah everything
01:08:39
but on the other hand when you feel
01:08:41
gets big enough and it gets so
01:08:44
difficult to do here for process that
01:08:47
these issues that you'll you know
01:08:49
you'll never get perfect whatever that
01:08:51
means. Um is an alternative way of
01:08:54
looking at what's the purpose of us
01:08:56
getting together in a physical place
01:08:59
and then it and it goes to various
01:09:02
point which is you know all that high
01:09:05
in that and you in that future is
01:09:07
needed because cried becomes the the
01:09:10
journals and all like high becomes the
01:09:14
place of a a conversation in meeting
01:09:17
figure out what where you know what's
01:09:20
important what's interesting and what's
01:09:22
not quite ready for the journal
01:09:23
process. But has really good ideas and
01:09:26
people are making progress on it and
01:09:27
let's talk about it and find it get
01:09:30
ideas together and and and so well
01:09:33
don't cry disappears in the mall. And
01:09:36
that that's one way to go I know how
01:09:39
many people are against that idea. But
01:09:42
that is where we could have a yeah but
01:09:44
it doesn't need to be that extreme and
01:09:45
I think one I really important point to
01:09:47
make is that the the process is
01:09:49
inherently subjective and we are trying
01:09:51
to pretend that it's scientific and
01:09:52
it's not the worst part of being a
01:09:55
scientist is this reviewing process
01:09:58
that we're stability that is completely
01:09:59
subjective even just the choice of
01:10:01
comedian subjective and just have to
01:10:04
talk about this and realise that it's
01:10:05
not scientific. So no that I don't
01:10:08
think we have to go as extreme as what
01:10:09
you what you propose because I do value
01:10:12
the sort of archival state assistance
01:10:14
or most important publication stages of
01:10:16
chi. But we need to do is that if we
01:10:19
except that there is a noise factor we
01:10:21
should not squeeze that exceptions
01:10:24
right to the point where the noise
01:10:26
folds over like where you have
01:10:30
potentially really good work to get
01:10:31
rejected because you have very low
01:10:33
acceptance rates but you should
01:10:35
actually be a little bit less
01:10:36
conservative and except some papers
01:10:38
that may be bad. But to the benefit of
01:10:40
papers that otherwise would have been
01:10:41
rejected and that's exactly what that
01:10:43
twenty five to thirty percent is that's
01:10:46
where it is you don't have to go to
01:10:47
forty percent ideally it's thirty
01:10:49
percent not twenty five percent and
01:10:51
that'll take care of that yeah I was
01:10:53
just these extremes to kinda yeah but
01:10:55
there's there's a lot of science
01:10:57
conferences that except almost
01:10:59
everything and ben but journals is
01:11:01
where you published I I don't I'm not a
01:11:02
proponent of that model. But I do
01:11:04
recognise that if we want to address
01:11:07
the noisy knows it's better to except
01:11:10
on merit and take some chances that
01:11:12
there are some flaws then the opposite
01:11:15
which has occurred some years I just
01:11:17
want to recognise that there was a big
01:11:18
change that happened two years ago not
01:11:21
this year but two thousand four
01:11:22
fourteen which is the move to PCS and
01:11:26
and that has really shifted a lot of
01:11:29
the ways in which these conversations
01:11:30
take place still one example is that we
01:11:33
don't end up seeing as someone that's
01:11:35
not logged into the system the
01:11:36
discussions right you don't have access
01:11:38
to the men in an easy way without
01:11:40
registering without being sort of and
01:11:42
all there in that system in the
01:11:44
positive and we also have a much
01:11:46
greater burden on on the organisational
01:11:48
side we don't have the wrong emails
01:11:50
sent out at all then you can see the
01:11:52
wrong people but but that there was a
01:11:55
burden to to do all this by hand
01:11:57
essentially. And that we had the hands
01:12:00
for every profits and actually this is
01:12:03
a huge part of the of this discussion
01:12:05
is thinking about that that that ship
01:12:07
to the platform what that what that did
01:12:09
to make come out quite actually a lot
01:12:11
closer to high. you'll enjoy in the
01:12:15
base paper words So I I think it's
01:12:19
likely to more questions we've about
01:12:20
six minutes I just want to make sure we
01:12:22
have a chance to address both of them
01:12:23
go ahead a my hands that your group I'm
01:12:26
I think this may be more of a comment
01:12:28
firstly I lo oh quite secondly thank
01:12:30
you for bringing this together because
01:12:32
I think I it's you can take notes which
01:12:34
is great. There's a huge amount of work
01:12:36
all high use things see that it's going
01:12:38
to be able to do this alone but there's
01:12:41
a lot of stuff that we're worrying
01:12:42
about in a community which is
01:12:44
fantastic. And a lot of it's coming
01:12:46
down to all kinds of this is a really
01:12:47
good discussion to have secondly the
01:12:50
the town hall meeting is probably what
01:12:54
what we try to do the executive
01:12:55
committee the on one meeting is to hold
01:12:57
hopefully of all of the things we do.
01:12:59
And reviewing is certainly something
01:13:01
that we would invite you to think about
01:13:03
bringing up at the town meeting. So I
01:13:05
think one of the things I was trying to
01:13:08
to to sell was this reviewing problem
01:13:10
the structure of reviewing but I think
01:13:13
what we would you say was that they
01:13:16
were ideas that could be controversial
01:13:17
because then and what I love about
01:13:20
alcohol use that its its mandate really
01:13:22
is trying to find one of the things
01:13:23
that I like french I like Reading each
01:13:26
what are the things as to to back is
01:13:28
very good point that I'm gonna surprise
01:13:30
this potentially. And I think the piece
01:13:32
that's missing me from the combination
01:13:34
of all kinds like you know the the call
01:13:36
for papers almost I wanted to be more
01:13:38
flamboyant more out that sent out a
01:13:40
different way sending out intentionally
01:13:43
to of the audiences to really kind of
01:13:46
solicit stuff and say we we really are
01:13:48
at the edge of a lot of interesting
01:13:49
innovation we want to be we biting
01:13:52
sense of this thing second thing is
01:13:54
another point speculation. I think that
01:13:57
your age pieces kinda gone away. So
01:13:59
it's almost like at the beginning of co
01:14:01
yeah I would love the all kinda can
01:14:03
stand out having solicited feedback
01:14:06
from the jurors who reviewed are to the
01:14:09
bunches of the stuff the the stuff that
01:14:10
publicly to to say this is what we
01:14:13
still coming I'll have to do this are
01:14:15
reach. Here's what we got contributions
01:14:18
and it turns out that you know pets
01:14:21
it's now extended to you know high and
01:14:23
goldfish. And we think that's gonna be
01:14:26
the big thing in twenty twenty I would
01:14:27
love to hear from the chair it's what
01:14:30
that urination process was what came up
01:14:33
the what the jurors among that yours
01:14:35
that about we use to get content piece
01:14:39
something along and and I'd love to
01:14:41
lobby is kind acted thanks yeah I agree
01:14:47
with that I think I by young boy comes
01:14:53
from our WK often university in Germany
01:14:56
so the first thing I realised is yeah
01:14:59
ten years. I think this probably if you
01:15:02
lose one who even have ever seen a
01:15:03
newsgroup so we'll be all kind of money
01:15:05
might actually be obsolete by itself so
01:15:09
I think about a new name I don't know
01:15:11
that I wanna say I really love the
01:15:14
initial idea vault kind went around
01:15:15
told everybody this is where you don't
01:15:17
find the papers with high scores but
01:15:19
with the high variances course that was
01:15:20
a very simple value proposition I'm
01:15:22
very much liked. However the rebuttal
01:15:26
process that we've since C and D
01:15:27
hopefully good workout bases in going
01:15:30
back and forth and you know there's
01:15:31
more more discussion going on really in
01:15:33
in the main papers we process may have
01:15:36
made that this in you'll necessity a
01:15:39
bit obsolete sell you may not have the
01:15:42
same sort of you chance anymore and it
01:15:45
getting those high you know very
01:15:47
represent because they're being
01:15:49
discussed more thoroughly reviews of
01:15:50
being adjusted. And people just have a
01:15:52
better chance of in interacting and and
01:15:54
iterating that said one of the things
01:15:58
are really loud and you know all kind
01:15:59
was well done thought pieces which
01:16:03
don't really have a place in over very
01:16:05
most other when you say like well if
01:16:06
you just have a you know a a an opinion
01:16:09
but some you know evidence to back it
01:16:10
up don't submit to us you know that you
01:16:13
what you get so I think that's what
01:16:16
I'll try to me was really you know a
01:16:18
very valuable thing. Um so you could
01:16:21
all placard more about those kinds of
01:16:23
things. Um I'm also thinking time for
01:16:27
discussion is also something that is
01:16:28
incredibly short mostly you just this
01:16:31
will form let me get up and mention my
01:16:33
own work that you haven't cited you
01:16:34
know hidden in a question. And so you
01:16:37
instead of having that which we have in
01:16:38
the main tracks you know having longer
01:16:40
discussion here I think would be very
01:16:42
valuable. Um and I don't know I think
01:16:46
my biggest goal for all kind would
01:16:49
probably be to actually hopefully not
01:16:52
know what's gonna be about next year. I
01:16:55
think that's your strength you know not
01:16:57
knowing what what what's gonna look
01:16:59
like how it's gonna work and what
01:17:01
topics are gonna be some keeping it.
01:17:03
That's the elusive you thing or
01:17:07
something just a two minutes so so any
01:17:10
last comments I wonder where the main
01:17:12
program whether removed from being of
01:17:14
systematically for dinner reviewing
01:17:16
some certain things were excluded and
01:17:18
they were just like under
01:17:19
systematically for just I would agree
01:17:24
somewhat with that so I disagree with
01:17:25
your statement that the that the that
01:17:27
the process is better now I think the
01:17:29
process may or may not be better but
01:17:30
the most increase because we just can't
01:17:32
cope with the volume of of submissions
01:17:33
that's the real issue now so well we
01:17:40
would like to hear but you don't really
01:17:41
also think of penalty for being here
01:17:43
today was a wonderful conversation and
01:17:46
Morgan aims Jennifer taxi was gonna be
01:17:48
over their co chair next year where in
01:17:51
the works of incorporating a lot of to
01:17:52
feedback that we all got from all of
01:17:54
you today and from the panel that we
01:17:56
have a couple of ideas around bringing
01:17:59
in actually these new format of
01:18:01
discussion and and and integration and
01:18:04
and and opening up debate with a lot of
01:18:06
your the were speaking to and I I think
01:18:08
we would really actually like to
01:18:10
continue this conversation outside of
01:18:12
this room. So please approach of of of
01:18:14
the court chairs and then play play
01:18:16
with that you know I use them of these
01:18:18
idea of to really continue forward with
01:18:21
without high and yeah Morgan dating
01:18:25
what I'm sure we'll just as a couple
01:18:26
examples of some of suggestions we got
01:18:28
already we one person approaches and
01:18:31
said why how about fiction encourage
01:18:33
more fiction wouldn't that be great we
01:18:35
could have a little booklet of high
01:18:36
fiction through the all right track
01:18:38
we're also talking about the
01:18:40
possibility of emergent panels or maybe
01:18:42
like a fishbowl forms like like kind of
01:18:44
expand the idea of what channels are
01:18:47
there all kinds sort of way so these
01:18:49
are some of the ideas that have been
01:18:50
coming from the community but please do
01:18:51
approach us with with others we are
01:18:54
very open to re imagining it with that
01:18:57
thank you very much. I think that I

Share this talk: 


Recommended talks

RUR: PROJECTS
April 24, 2012 · 6:20 p.m.