Player is loading...
Asynchronous streams in direct style with and without macros
Philipp Haller, KTH Royal Institute of Technology in Stockholm
Thursday, June 13, 2019 · 3:45 p.m. · 47m 59s
Scala-async is a macro for simplifying the use of futures. This talk presents an extension of scala-async which additionally enables writing producers and consumers of asynchronous streams using familiar straight-line, blocking-style code while using efficient non-blocking, event-based concurrency under the hood. Building on the Reactive Streams standard enables integrating with many stream-based Scala libraries and frameworks. The second part of the talk shows how OpenJDK Project Loom fibers can be used to implement extended scala-async without the use of Scala's deprecated macro system. Required Knowledge: Attendees should have some experience with concurrent programming in Scala, for example using futures or actors. Philipp Haller is an associate professor in the department of computer science at KTH Royal Institute of Technology in Stockholm, the leading technical university in Sweden. Philipp is co-author of Scala's async/await extension for asynchronous programming, and one of the lead designers of Scala's futures library. Main author of the book "Actors in Scala," he created Scala's first widely-used actor library. Previously, Philipp held positions at Typesafe, Stanford University, and EPFL.
Embed
Transcriptions
Note: this content has been automatically generated.
00:00:00
okay uh welcome to my talk i'm gonna talk about
00:00:04
asynchronous streams in direct stout with and without macros and um
00:00:12
so i'll i'm are an associate professor at a kid th in stockholm and uh
00:00:19
before going to academia actually work at type safe which is now like bent
00:00:25
and i actually did my p. h. d. here the p. f.
00:00:28
l. so it's a nice to be nice to be back um
00:00:34
so let's that introduce today's topic was a simple task okay so we're
00:00:40
given an asynchronous method that returns the day of the year
00:00:46
as a string wrapped in the future writes a day of your returns future string and it uh
00:00:53
can return strings you know like zero four slash eleven four
00:00:58
april eleven and and so on and um
00:01:03
now sometimes this mess it can return can complete the
00:01:07
future with an empty string or was now or nope and
00:01:14
what we should do is now should create a future that's completed
00:01:19
either with the strain it's months like it's april or whatever um where um
00:01:28
we we turn the number into text or presentation ah or
00:01:33
it should are completed this not update mate okay so that's
00:01:39
shouldn't be too difficult um also we're in the expert track so i'm sure you can uh
00:01:46
come up with the program that does that so we can
00:01:49
for example use 'em futures and for comprehension stew to do that
00:01:56
um first we should actually define our reg x. here and
00:02:01
um actually i think greg's are quite nice it's callously there actually
00:02:07
nicely integrate was extract hours so if you haven't used them before give it a try and actually
00:02:12
this side note since ready p. f. l. the um the red x. support in the standard library actually
00:02:19
originated as a as a student project that i supervise the d. p. f. l.
00:02:23
long time ago so to a time frame two thousand seven or
00:02:26
so um but the basically we we have our right x. here and
00:02:33
well now we're just go ahead and say well when day of your completes when that features
00:02:39
completed we have a base string and we can match on that string ah against our right x.
00:02:46
uh extract demands and what didn't mention before is the other constraint is we
00:02:51
need to use the name of months and that's another method that returns a future
00:02:56
'kay like imagine you're composing real more complex things and you
00:03:00
have a you know you call into services and whatnot so then
00:03:04
you know futures are really everywhere so that's what we're trying to simulate the name of month return the future as well
00:03:11
and um so that means um since we need that result as well
00:03:17
we construct another future that using another for comprehension and nested for comprehension
00:03:23
uh in the which you know reels uh or or or restarts
00:03:27
in the future completed with the ah it's i'll whatever the name is
00:03:33
otherwise if the string doesn't match alright x.
00:03:36
we can create a future that's already are completed
00:03:42
yeah so that's our our how we can do it it are takes a while to explain that so
00:03:48
the question is can we improve readability a little bit and uh of course the answer is
00:03:55
we can and so this is one of the examples actually that uh
00:04:00
uh we used before to to introduce sky sink so
00:04:05
as pricing is essentially ah designed to simplify asynchronous programming
00:04:12
primarily based on futures although you know there was some some work and some ideas on on how to
00:04:18
how to make it more general but impact is people use it pretty much only with features
00:04:24
um guys think was created by jason's out uh from like and
00:04:30
and myself and we we realise that in a first in in twenty thirteen and it's actually
00:04:38
a very popular projects so um uh it it is
00:04:43
used also by the way who view has used it before
00:04:48
some some people not not
00:04:52
uh not everyone um but we can we can use it to to
00:04:56
simplify things a little bit so we can say well what we need to
00:05:01
do is we need to create a future so let's just created a
00:05:04
sink block and was in the a simple arc we can compute the result
00:05:10
right and um so it's a similar to the future apply method
00:05:14
right we can just sort of uh provide a body that computes the result
00:05:19
and so alright we do the same thing we did before with already x. except that
00:05:24
now since we need the day of your uh the result of that future we call wait and
00:05:32
what what it does is it first checks if the
00:05:35
future is completed um if it is we can actually continue
00:05:39
uh with our code if it's if they of here is not completed
00:05:44
yet uh the asian clark is the body of basing part is suspended
00:05:51
um so that the the underline thread which is normally a worker thread a four track cool
00:05:58
that was executing the a simple walk with the body of the asian bloc
00:06:02
that's right now is fried again to execute other tasks tasks that should execute
00:06:08
right is a full thread so it should really process all the tasks in
00:06:11
the pool and so that's what eight uh wait does it frees up the thread
00:06:16
and uh but it's stores the it stores the execution state
00:06:20
um of this a simple walk so that later on we can actually resume
00:06:25
the a sink block exactly from that place from the point where we called a weight
00:06:32
right and so this is what happens when the day of your futures completed
00:06:36
so wait also registers a completion handler which uh when the completion aloe runs
00:06:44
uh then essentially we uh take the result of
00:06:48
the future the day of your future and uh
00:06:52
logically replaced uh wait call with that result and resume the execution
00:06:58
right so we we resume here and do the match on the result
00:07:03
right and so this is um is what a weight does
00:07:07
and so we can really use a weight as well straight forward way to just
00:07:12
uh you know suspend until we have the result and we don't have to worry about
00:07:18
uh as the underlying fred being pinned um because of that
00:07:22
red's not happening so it's very friendly method um to use
00:07:29
right and um so also i think what's interesting to point out in comparison to the previous solution
00:07:35
there's a uh some interesting differences so for example we don't have to name all the intermediate results
00:07:42
okay we on though naming is hard to so if you're forced
00:07:46
to come up with names all the time that's not ideal right
00:07:50
especially if it doesn't improve uh readability so here for example we have to come up with the name for that result
00:07:56
you know we had to come up with the name for the response and get to come up with the name for this name your and
00:08:02
you know we end up with these generic names that don't really
00:08:04
help readability at all so like if we can avoid that that's good
00:08:09
so this is one advantage of course we we are free to introduce names
00:08:13
as much as we want but at least we're not forced to do that
00:08:19
also we don't need to explicitly construct completed futures
00:08:24
okay and and i think this is actually important because it it's due to the fact that we have to do this in a previous solution
00:08:31
tells you that you know the fact that there's an flap map under nice actually kinda leaks to the surface right
00:08:38
um it's fine if you know how that you know we're actually doing fact map and you know how do
00:08:43
right the code um explicitly using fact map that's good but it's that's certainly doesn't help clarity
00:08:51
i'm also what's interesting is that the a simpler actually allocate fewer
00:08:55
closer us right so one is simple walk is exactly one closer
00:09:01
right so i if we if we go back to the for comprehension
00:09:04
code since the the shoulders to map in fact map calls we allocate
00:09:10
work for closures right each um for each call to map
00:09:14
or flat or we have another new closer that's allocated and um
00:09:20
that's a performance hit but also if you pass values into
00:09:25
and out of cruisers there's a lot of boxing going on often
00:09:29
so we can even avoid some of this boxing if you just make sure reserve one closer and you know all the
00:09:36
you know all the shuffling of results and so on that happens internally so
00:09:39
there's even stick can even be a performance improvement here so it's actually rather nice
00:09:47
um but let's do something more complex so that we test the limits here a little bit and so how
00:09:53
about we sort of converting a single string we need
00:09:56
to consume a stream of multiple strings and i'm also
00:10:01
the output of our solutions should actually be a a a publisher that uh
00:10:06
that produces multiple you uh results so it's not enough to just return of future
00:10:12
that eventually is completed and so if you don't want streams we should talk about
00:10:19
reactive streams uh or as they're also known
00:10:23
as which is um job we took current flow
00:10:28
right so since thirty k. nine uh that you know these interfaces actually are
00:10:34
now uh in this flow ah class and so it's a set of a small set of interfaces
00:10:42
i'm very few of them the processor it's not even essential
00:10:46
so that we're actually talking about like three interfaces and um
00:10:52
they so what what do they do well i mean they they they allow you to construct a
00:10:59
asynchronous streams right so producers like publishers that publish events is synchronously
00:11:05
and subscribers that can subscribe to publishers to receive events a synchronously
00:11:11
and um the subscriptions also actually quite important because it's
00:11:14
a allows the subscribers to control the flow of the events
00:11:20
and uh now this work didn't happen in in a vacuum or there was actually
00:11:25
some key prior work and i think we should probably start commissioned observer design pattern
00:11:30
classic kennel for pattern right where you have an observer and uh sorry um
00:11:35
uh as subject subject and observers and um when the subject
00:11:40
changes its state say um we can have zero or more observers modified uh
00:11:48
and um and probably the most important prior workers
00:11:54
actually the the work on reactive extensions by eric meyer
00:11:58
where he's actually puts the asynchronous streams on
00:12:01
a solid foundation right by saying well um
00:12:06
what is actually an observable of t. v. like what does
00:12:09
it mean in relation to other types that we know like uh
00:12:13
uh it ripples for example right and and worry basically
00:12:18
uh it said well let's use functional programming concepts to
00:12:23
provide a ways to compose a observables so observable
00:12:28
and publisher gonna use those interchangeably quite often here so
00:12:33
a bear with me but um so how to compose observables
00:12:38
using higher functions right and uh and that's been really successful
00:12:44
right with many implementations for various languages and uh and so on
00:12:48
and uh and then i think like the key innovation of reactive streams then
00:12:53
is uh the draft back pressure control right so that the subscribers
00:12:58
uh uh are not just flooded with events from publishers but they can actually
00:13:03
tell the uh the subscription object to you can tell the publisher
00:13:07
to say well i'm only i only want to receive at most
00:13:11
five hundred twelve events or whatever right and sort of adjust the the
00:13:17
flow of events that way so just to bring everyone on the same page
00:13:22
use quick run through these interfaces so probably sure is very simple and
00:13:26
i'm gonna use the g. a. k. nine ones because those are the
00:13:30
the there's whatever once i'm gonna migrate to anyway so
00:13:36
the publisher has lot subscribe method that takes a subscriber
00:13:41
and um the subscriber needs to be able to receive
00:13:46
events of type t. because publisher publishes events of type t.
00:13:51
right and then the subscriber is essentially the interface that
00:13:55
the publisher uses to publish events to the to each subscriber
00:13:59
so it has methods for each kind of event uh the first
00:14:03
event actually happens when there's a ah subscription from a subscriber regimen subscriber
00:14:09
subscribes or publisher um the publisher gonna successful case
00:14:14
you know creates a subscription object and calls to
00:14:17
unsubscribe method so that i'm from then on the
00:14:21
subscriber can use that subscription object are to control things
00:14:27
and then following that uh if the subscriber has expressed
00:14:32
interest um that needs to be the case right then
00:14:37
the publisher can probably pants and a polish them to subscribers using those methods searchers next event
00:14:43
of type he would call on next on each subscriber um there's an error if the publisher
00:14:51
terminates wasn't are we the subscribers need to be notified
00:14:56
of that as well and the stars also modified if the publisher terminates terminates normally
00:15:04
okay and then finally the subscription
00:15:07
interface that i mentioned right where these witches messes that should only be
00:15:12
called buyer biceps by subscribers and they can request a number of items ah
00:15:19
from the publisher i didn't say i request two hundred
00:15:22
fifty six items and uh and then the publisher needs
00:15:27
to adhere to contract that says well you cannot emit more than that uh more than those events to the subscriber
00:15:34
and of course we can cancel subscription at as well a
00:15:39
good so now these are so the base interfaces here and uh
00:15:43
they are of course rather low level so normally how people use these things as they build
00:15:49
uh they use libraries that build on top of these interfaces and one important uh such framework is the
00:15:56
this reactive extensions framework right if you you should know about that one if you if you know about
00:16:03
streams right so it's some uh the key is that we that
00:16:07
um we have a library that provides higher order functions to compose
00:16:13
uh publishers and um i just wanted to give you a very short example
00:16:20
which is actually taken from the original article by eric meyer on directive
00:16:24
uh on undirected extensions and so this is written c. sharp um but the idea is that
00:16:31
we have we provide methods that construct publishers
00:16:37
or subscribers read and uh we we can put and have higher of commentators
00:16:42
so for example text changes here um is a method that creates a publisher
00:16:48
that publishes in an event for each each time the text in an input fields changes
00:16:55
right so each time i type in the inner input field i make a change
00:17:00
there's a new event published with the with the current contents of the input field
00:17:05
and um so that's the simple publisher of strings and out on that one i call select
00:17:11
and select actually e. would be called map so in in parks java for example it's
00:17:17
called map right on school select here because because we actually set is integrated with link
00:17:23
and they're the in link the mess it's you know a map
00:17:27
schools select um we can use company engines for that as well so
00:17:32
so we call select and what's like does is it
00:17:35
transforms the publisher into or in this case an observable
00:17:39
transforms into a new observable such that for each word
00:17:44
it's published by the first observable now we've publish a
00:17:49
an event which is the completions for that word right imagine you have a search
00:17:54
engine you type a word and then they're a synchronously there completions been suggested to you
00:18:00
right and so that's what happens here so that means for
00:18:02
each event for each word ah the result of select publishes
00:18:10
and observable so completions actually some observable itself okay so as
00:18:14
a result of select is actually an observable of observables and um
00:18:20
right but select at selected cells just map okay
00:18:24
um and um but then we use which which is
00:18:29
somewhat um more complex right where since we have a publisher off
00:18:36
or an observable of observables what it does is it says well when you
00:18:40
receive the first observable as an event you start publishing it's events these are
00:18:45
the red events initially and as soon as you see if the next observable
00:18:49
you publish its events so we and then so we switched to the yellow observable
00:18:54
'cause that's the next up so we have received and these are the completions for the change the word
00:19:00
right initially we we have a word in the text box and to be at such have completions
00:19:05
and then eventually the user type something and uh and then we receive
00:19:11
an observable for completions of the change work so that's the yellow events
00:19:15
right and about the user can type again so that means we may receive another observable
00:19:19
for the completion so next you and and because we have to switch always to the
00:19:23
conclusions for the latest word okay and this is what's which does so as you can tell
00:19:29
yeah actually built quite a up complex commentators here um
00:19:35
and uh yeah so this is our programs look like here um
00:19:40
and so there's some challenges obviously one is we
00:19:44
often need diagrams to understand what's going on right and
00:19:47
so if you read the documentation for reactive extensions you
00:19:51
see a lot of so called marble diagrams which are
00:19:54
in evolution of um of these of these event uh
00:19:58
diagrams right there so formalised the way how present these things
00:20:03
and um and in fact if you try to create commentators these higher commentators like um
00:20:09
select or switch or zip what have you this can be quite challenging particular if there's state full
00:20:17
and um the the key here is that
00:20:19
actually commentators to combine multiple observables there often
00:20:24
their state full internally right even though the the the staple this cannot be there's no observable state
00:20:32
right but um internally just implement then their state full so if you even just implementing zip
00:20:38
he's got i have stayed inside okay um so and that makes it somewhat challenging
00:20:44
and finally of course we have an inversion of control going on which means we're actually building programs that
00:20:51
like they're not running on the current thread no i mean we're just
00:20:55
building computation centre like submit it for execution by the run time you know
00:21:00
using some schedulers instruct tools and so on so the code we write is of course very different from
00:21:06
sequential code that runs on the current thread does
00:21:10
is very big difference right between these two things um
00:21:15
so now let's now that we know about streams let's go back to
00:21:18
our task at hand so we're supposed to convert a stream of strings
00:21:23
uh into or we we want to create a publisher
00:21:28
that consumes those events in produces a stream of results
00:21:33
now we want to try and do this like without the reaction extensions style
00:21:39
um but instead the question is can we do something similar to a single weight
00:21:44
not to try to do this we run into some issues okay first problem is
00:21:49
i'm trying to uh wait events of public service is actually not really possible because uh wait
00:21:56
takes futures as arguments so we can't um wait
00:22:00
for the next you meant offer publisher for example
00:22:04
um so that doesn't work also a sink traits on your future so we
00:22:09
we um um but we need to create a publisher that can yield multiple events
00:22:16
so that doesn't work unfortunately and um so this is sort of one of the motivations for this um
00:22:24
a proposal which i called stylistic flow now here which is just
00:22:29
saying well can we make this guy is think but more general
00:22:33
um to support streams and so it is that
00:22:36
we extended a single weight well such that um
00:22:44
we ah have a variant of a sink that can create publishers
00:22:49
right we've it's not enough to straighten future um and so forth to
00:22:54
distinguish that from the previous one we call that are a sink and um we also need to generalise the weight so we need to
00:23:02
be able to pass not just the future but um streams in
00:23:06
there so that we can uh wait the different stream events also
00:23:10
and finally we need to be able to yield a events so we need to
00:23:15
will yield the next event of the probably sure are maybe you'll than
00:23:19
error uh it or you'll done to use indicated the publisher is has terminated
00:23:26
so first was the weight is actually quite simple so previously we had
00:23:31
oh wait and we pass in the future of t. and or sell dusty um
00:23:37
so um i forgot to put in this picture which says i can't believe it's not blocking right but um uh so
00:23:44
okay but that's fine the the thing is we need to make this more general right so we need to say well
00:23:49
uh this should be difference about just the features we need some kind of
00:23:53
an object like in a single object need the notion of and a sink object
00:23:58
um and which has a simple interface so here i actually named it somehow but
00:24:02
is it probably is even even better to have a have this is structural notion
00:24:07
to say well it needs to be a a single object needs to
00:24:10
provide the these two methods right where essentially we can query whether it's completed
00:24:18
actually i think the term completed is maybe not ideal but the 'cause the string you
00:24:23
in this case would complete many times but we need them up this is analogous to future okay that's why
00:24:29
it's called like that but we need the message that says do we have something to receive is there an event
00:24:36
that we can get right now and for future it's you know is the future completed
00:24:43
and uh if so please give me the try so failure or success right for screen it would say
00:24:51
is the next event available yes or no right um
00:24:57
and by the way is also if it's if it's available we get to try it's not available we get now
00:25:02
so um it's either now or uh try and and on complete
00:25:06
is of course analogous to the on complete on futures where it takes
00:25:11
a handler from triumph t. two s. and um requires an execution context
00:25:17
and that's of course used so that we can say well um we
00:25:22
can register completion handler that runs when the a single object is completed or
00:25:29
when the basic stream publish its next event or when it terminates
00:25:36
right um now we once we've done this extension right we
00:25:40
can actually formulate the solution quite simply by saying well let's
00:25:46
make use this are a sink to create a publisher of string
00:25:52
so unfortunately have to provide type argument here but you know the
00:25:55
reasons for that but um okay it's gonna be a publisher string and
00:26:02
now we can just wait we can call wait and provide
00:26:05
ah the data streams that's the upstream publisher of data strings
00:26:11
i don't show this and show on the data stream but it's uh it was just a publisher that uh
00:26:16
pushes out a date strings and now the the result of weight as an option
00:26:22
because the stream could be terminated normally in which
00:26:26
case we get non as a result as a result
00:26:30
or we could have some of x. where x. is the next event published by the data stream
00:26:37
and um by the way if the data stream terminated
00:26:40
with an exception then the call to await throws that exception
00:26:47
right so this is just like normal the what we would expect actually for that one right um
00:26:53
and so then we can we can check okay if the option it as long as the option is non empty we can
00:26:59
uh access it and match it against the red
00:27:02
x. inhaled a yield whatever the next event is um
00:27:09
now um what interest up sorry about that what's interesting is that the
00:27:16
inside here we actually call oh wait wis a method that returns a future
00:27:21
right and this is actually interesting why well because here we wait for a stream for this is a publisher
00:27:27
right publisher string and but this is the future of a string so
00:27:33
we need to build a wait for different a single objects we can just fix one of them right
00:27:37
would be too limited if you fix it to publisher in this case right um so that's um important
00:27:45
and uh yeah and so then you've next of course we can cook all multiple times
00:27:50
uh in a loop here right so once we've yielded a the next event we can be
00:27:55
uh go back and wait for the next event from the stream and uh
00:27:59
as long as that's non empty we continue and finally we do you've done um
00:28:06
okay now how can we implement these things um well there's
00:28:13
electronics right we can either extend this guy is a micro
00:28:18
that's certainly possible we can try to put you recompile support in there and uh in fact uh
00:28:25
uh yesterday adrian morse told me that um there's a
00:28:28
branch that he created a branch which that's that's so um
00:28:32
at least for the a plane a single weight so that's possible of course and we can build on
00:28:39
on top of considerations or fibres and that's what i'm also gonna show here so
00:28:44
let's start maybe with the interface with the a. p. i. right so
00:28:47
are a sink takes a body and um now this body of course ah
00:28:55
well we we should have the body needs a context which we
00:28:59
call flow of t. here and um the context is there because
00:29:07
if we call a wait inside the are a simple walk well we need to know
00:29:12
we need to have you need me to be in the context where we can do that
00:29:16
right and also if you want to yield an event we need to be in the context
00:29:19
or we can yield the event of that of the current publisher that's um provided by this context
00:29:26
right and so we use a contextual function ah to do that and um
00:29:32
right and the result will be a a publisher of t. um and
00:29:37
so then i wait here right it will have this a single object
00:29:43
as an argument and it also needs a context uh both the
00:29:46
flow and uh and it needs an execution context as well because
00:29:50
it'll register completion handler and for that one we have to say where it should run
00:29:55
so that's where we need to execution context and i'm you next well that's super easy
00:30:03
wow we're we can use it inside out when when there's the context available here and so then we can
00:30:09
easily just you the next event so it's really kind of like the builder pattern that morgan showed in his keynote
00:30:16
um so now the you know the tricky bit okay how do we
00:30:21
uh transform this are a simple walk and i'm gonna
00:30:25
show you the the probably the simplest possible probably sure which
00:30:30
and and it's it's written in in like excruciating detail on purpose okay i've used the bar on purpose
00:30:36
of use that type annotation purposes and so on so we're just that just to show well we have our
00:30:42
a local variable x. that's reassign all right so initially
00:30:47
it's non but then later we're gonna call uh wait
00:30:50
and assign the result to x. and then after the you wait the only thing we dues record get on x.
00:30:56
right just like the simplest possible thing you can
00:30:59
think off i'm sick forwarding one event somehow so
00:31:05
what we well we need to build a suspend inside the sorry state park
00:31:08
right so and the the way the skylight is think marker works and this
00:31:12
is very similar to how to see shark compiler works is it creates a
00:31:16
state machine where i'm just a simple class and that's essentially are closer right
00:31:22
this one class and it has a state variable so we number or states
00:31:28
because you need to transition between states and um there's a result promise here
00:31:34
that's actually more like an implementation detail but okay and now the part x. option of int
00:31:42
this is this used to be a local variable now it's a field right it has to be a field because
00:31:48
we're gonna have to be able to access it um in different states of the state machine right so we
00:31:53
can't keep it as a local variable and so the
00:31:57
state machine we can start from the beginning from state zero
00:32:02
okay and we do the calling apply takes a try i usually
00:32:06
don't know tried to provide so we just provide now essentially and
00:32:10
we start in state zero and what we do is well we start the
00:32:14
code from the beginning what we'll do the beginning we assign non to x.
00:32:19
this is the initial code here at this is like the first thing we do we assign onto x.
00:32:25
now you see why i probably have this excruciating detail because this is some code we do could be more complex right
00:32:33
and uh the next thing we do because the next thing you do is already calling a weight that's the the very next thing we do
00:32:40
right recall rates so how does that work well s.
00:32:43
is given that's probably sure um we somehow need to find
00:32:48
the corresponding subscription for that one but we have a context flow so we can look it up in the context
00:32:55
and the subscription we can check if there is a next event there we call get completed
00:33:01
ah if it's now it means there's no next event yet so you've got to could suspend
00:33:08
uh it's the only thing you can do is really suspend then register completion handler so we do that
00:33:14
um be but before we register the completion handler we change state to
00:33:19
to so that once we resume the state machine we resume in state too
00:33:24
right the completion handler basically says oh once i'm concluded i'm gonna call the state machine
00:33:32
with that event so that's the try that would were plus cast into the state machine
00:33:37
but this is gonna happen apply is gonna be called when status to so that means
00:33:41
when it completes when it completes recall applying we we're in state too so that means
00:33:47
in state to we have a try right uh and we can check
00:33:52
if it's a failure not fatal we complete our promise was that failure
00:33:56
if not we can we can get the x. get the result out of
00:34:01
the tribe because it's a success can get it out in change state to one
00:34:07
right and one is the state where we run after the weight we can resume after
00:34:13
the weight to this kind of this case to was actually bookkeeping right this is happening
00:34:18
when this thing is completed or we have the next event right and we
00:34:22
do the bookkeeping to know what should happen next to what we need to do
00:34:26
but the actual code that runs after the weight is actually in state one
00:34:32
right to change the state one call apply which means we end up here and the x. dot
00:34:36
get is actually the entire piece of code that comes here is of course it could be complex expression
00:34:41
in this case is just extort get so this is what we do here in the complete the promise
00:34:47
okay so um that's essentially how it works out that made some sense
00:34:52
i don't every time to get that intimate details but um so that
00:34:57
that's one way to do it and to compare can do the marker can do it uh
00:35:01
ah well a a macros may not make maybe should not do it because they just become
00:35:07
compiler phases that do it so um but we can
00:35:10
also think of running on a run time system that provides
00:35:16
a bit more support like condemnations so and that's what i'm project loom in the open dedicate
00:35:21
does and the idea is to provide features
00:35:25
that we can use to build lightweight high throughput
00:35:29
currency constructs in particular provides a dilemma considerations and
00:35:34
fibres the fibres are essentially use remotes rides um
00:35:40
but they're interesting because of their integration with with with the j. d. k. but um
00:35:45
and this is of course still under active development you can't uh there's no release that has that yet but it's uh
00:35:51
there is a lot of people not working on so you it's looks quite promising so how did how does
00:35:58
considerations look like well the basic idea is that you
00:36:02
can take a run double and turn it into a continuation
00:36:07
so is actually quite simple right you just provider audible and
00:36:11
in the body here you can actually suspend and resume the continuation
00:36:16
right um and um another thing is this is a super low level a. p.
00:36:21
i. clearly like to if you want to pass in values and out of the body
00:36:26
you need to build higher level abstractions that's not provided right so you need to do that yourself um
00:36:34
and uh so yours like to hello world example off considerations i'm just very
00:36:38
very quickly run through it but it's it's really simple that you create a continuation
00:36:43
um you call run when you call run started from
00:36:46
the beginning right and it runs until either the end
00:36:50
or until it suspends which is done was healed right
00:36:53
you can heal to the continuation and that makes run
00:36:57
so that so run completes and um okay in check whether the continuation actually finished or not
00:37:03
um this case it's not so we set continue true and then
00:37:07
we can see the receiving the condition coming round again so we run
00:37:11
we we resume by picking up from the yield point here and uh you know a resume things
00:37:19
the very simple and of course this is makes it quite easy to implement um
00:37:24
this model because in our a sink essentially when we were given this
00:37:29
body we just need to create a continuation where our task um evaluates body
00:37:38
and uh it can do that because the flow here the
00:37:43
delegates actually in scope so it works um without any issues here
00:37:48
right the body has access to the delegate and um everything works out fine
00:37:52
and inside the body we could have implications half healed or a weight yeah um
00:38:03
and i'm sure you i'm gonna show you way too but if it runs into completion we
00:38:06
can emit events and uh or you know there's an exception we can captured and so on
00:38:11
um and uh uh wait now it's very
00:38:14
simple to basically were given our context and uh
00:38:20
we can check whether we're already we have something to consume if it's
00:38:24
not all then we don't which means we need to register completion handler
00:38:27
completion handler can a resume the continuation so the
00:38:32
flow context a real provides a way to resume the
00:38:37
the corresponding continuation and um and once the red is
00:38:41
the the handles registered we can suspend and uh i'll
00:38:46
and of course resumption will will happen right after the suspend
00:38:50
call so it's uh that point we can we can resume
00:38:56
okay so this is actually very simple and um there's even more likely even like wrote
00:39:01
a paper about this um so you have like you know thirty something pages of uh
00:39:08
ah text um which which actually has the also formalisation that is actually the
00:39:14
first uh a reduction semantics for a
00:39:17
observables um before that eric meyer and um
00:39:22
and others that that could be notational semantics does actually reduction semantic so
00:39:26
it sell quite nice and you can do type sounds proves and uh
00:39:31
a protocol conformance that a publisher doesn't doesn't publish more events after it has
00:39:37
terminated for example that's like an invariant that uh actually the type system provides
00:39:42
and uh we also talk a bit more about the market based implementation there um yeah so
00:39:48
there's open issues that she mentions so one is the red and blue functions issue right where
00:39:54
if you wanna call suspend method which in our case it's mess it's that
00:39:59
have that require the flow context and uh uh that means this call as
00:40:04
respectable to so that means i'm particular if i have a higher order function
00:40:09
like map it takes um function and uh i assume i want the argument function
00:40:17
to be possibly suspending well then i need to prepare map the map for that i need a
00:40:22
variant of map that allows me to suspend to do you know that allows me to call us expendable
00:40:29
a function as i needed to variance for each of these hiram
00:40:33
functions right so red and blue variance so that's not good um
00:40:40
and uh of course monad suffer from the same issue and so
00:40:45
that's something to work on 'em loom condemnations themselves actually don't
00:40:49
stuff from the problem so we can actually use to spend anywhere
00:40:53
uh as long as there's an active continuation and so this suggests that um we may
00:40:59
actually really relax the need for the for the flow capabilities right and i think there's some
00:41:07
promising um work i think that's possible on on like a p.
00:41:11
i. designed to to do that uh yeah so i'm i'm gonna
00:41:16
come to the end here so we have some resources so um
00:41:21
the prototype implementations the two ones one base markers one on the considerations
00:41:27
is on my guitar bent um you may want to
00:41:30
look at the count branch because well it uses um
00:41:35
project loom combinations and it's actually also the only branch that actually makes use of stella three so it's more
00:41:42
interesting um and uh also here at trying to uh
00:41:47
you know realise up reprints of things so that you're not
00:41:51
behind people's always and um yeah that's it thank you how
00:41:59
how are questions
00:42:09
ah there's a question over there
00:42:13
uh_huh sir microphone okay older
00:42:21
uh_huh
00:42:28
he thinks it's really a interesting which are trained to do two bridges to uh owns
00:42:35
um corinne if you could show an example or explain how it would look for
00:42:41
a little bit more complex example like a flat ah come cat the buffer or
00:42:48
sort of theirs is imperative start a how
00:42:51
we work out in product is uh mm arm
00:42:57
yeah so i mean well there are some more examples line fact if so
00:43:02
this paper i mentioned is actually has and kind of um i i
00:43:05
like it to another um maybe i can find it real quick little
00:43:12
see i copied and pasted couture um so i i would let me show you a bit more
00:43:21
um in fact if you have steak fullness and and things like that it it really becomes much easier
00:43:27
to use a direct style i'm sorry
00:43:36
the yeah so so zip for example zip is actually interesting it's more complex than
00:43:44
you think right um you think you may think oh zip that's easy we just uh
00:43:50
well this case it's a quite general zip mess it because it it zips up a whole sequence of observables
00:43:57
so replace observables publisher and you get the same picture but
00:44:01
the ideas we have uh this uh rather generals that mess it takes
00:44:05
the sequence observables and uh the zipper combines the function that combines the
00:44:13
elements uh commands events of each observable um and so the results of soul of our
00:44:20
and so what we can do here is we need to keep track of a whole bunch of stuff um
00:44:29
so in particular we need to wait for the next event of each source right um
00:44:38
oh someone's calling that's a too bad i've to delay that um but um
00:44:45
so so we we um create an array that can hold
00:44:51
the next events um and we can actually just look through
00:44:55
and just wait for the for the for the events and the cool thing is that this will not decrease performance right because
00:45:02
when we suspend on the first one well we can
00:45:04
still uh the next producer can can already produced event
00:45:09
but will be immediately available once we got the next event uh wait call and so on so
00:45:14
we actually just collect these things um and then decide okay what to do do we have file
00:45:20
you know can we is it okay to run the that combine or function and so on
00:45:25
right so um and um you know at some point these teams terminate and they can terminate at
00:45:30
different points they don't maybe don't produce the same number of items either we have to handle that
00:45:37
i mean this with direct style it's actually it's rather trivial if you can do sequential programming
00:45:42
you should be able to do this also right so i think it's um yeah it's quite nice so i encourage you look at the article
00:45:50
um and also because or some other examples as well so there's like a challenge problem
00:45:55
with the solution also here where it's also a bit like combining different screens and uh have some state
00:46:01
forms in there's well actually with buffers and when doing also going on right so you can like um
00:46:07
you can say well i have two streams and combine them but any the window right and um
00:46:15
and i need to reduce multiple events of one window and check
00:46:19
conditions and and what have you right and uh not always publish
00:46:24
an event for each received event no but i have a condition
00:46:28
that doesn't it's makes it much less regular and so on so
00:46:33
my advice is just look at the examples uh in the paper also
00:46:39
any other questions okay so um in your example you use 'em mutable
00:46:46
viable weird why do my question is would we would it be possible
00:46:50
to rewrite this example using more like idiomatic functional start i would sorry
00:46:56
for compensation or or a recursive function are we talking about this one were
00:47:04
okay um yeah i mean there's nothing that uh prevents you from from just using
00:47:11
commentators to combine publishers to right so i think that
00:47:16
i mean this is maybe not like the best example the example what this shows here is that it's very simple to go from all
00:47:22
future based code to stream based code right like this looks very similar to the initial future based
00:47:28
isn't the way to usage right but i mean i agree with you that i think this is the
00:47:34
this is like a very regular examples so there's there's not so difficult so sure you can just use uh
00:47:40
use map on a publisher or whatever and it will work the same way so
00:47:44
like you're not forced to write this in this way but you can uh
Conference Program
A Tour of Scala 3
Martin Odersky, Professor EPFL, Co-founder Lightbend
June 11, 2019 · 5:15 p.m.
8337 views
A story of unification: from Apache Spark to MLflow
Reynold Xin, Databricks
June 12, 2019 · 9:15 a.m.
1267 views
In Types We Trust
Bill Venners, Artima, Inc
June 12, 2019 · 10:15 a.m.
1569 views
Creating Native iOS and Android Apps in Scala without tears
Zahari Dichev, Bullet.io
June 12, 2019 · 10:16 a.m.
2232 views
Techniques for Teaching Scala
Noel Welsh, Inner Product and Underscore
June 12, 2019 · 10:17 a.m.
1296 views
Future-proofing Scala: the TASTY intermediate representation
Guillaume Martres, student at EPFL
June 12, 2019 · 10:18 a.m.
1157 views
Metals: rich code editing for Scala in VS Code, Vim, Emacs and beyond
Ólafur Páll Geirsson, Scala Center
June 12, 2019 · 11:15 a.m.
4695 views
Akka Streams to the Extreme
Heiko Seeberger, independent consultant
June 12, 2019 · 11:16 a.m.
1552 views
Scala First: Lessons from 3 student generations
Bjorn Regnell, Lund Univ., Sweden.
June 12, 2019 · 11:17 a.m.
577 views
Cellular Automata: How to become an artist with a few lines
Maciej Gorywoda, Wire, Berlin
June 12, 2019 · 11:18 a.m.
386 views
Why Netflix ❤'s Scala for Machine Learning
Jeremy Smith & Aish, Netflix
June 12, 2019 · 12:15 p.m.
5026 views
Massively Parallel Distributed Scala Compilation... And You!
Stu Hood, Twitter
June 12, 2019 · 12:16 p.m.
958 views
Polymorphism in Scala
Petra Bierleutgeb
June 12, 2019 · 12:17 p.m.
1113 views
sbt core concepts
Eugene Yokota, Scala Team at Lightbend
June 12, 2019 · 12:18 p.m.
1656 views
Double your performance: Scala's missing optimizing compiler
Li Haoyi, author Ammonite, Mill, FastParse, uPickle, and many more.
June 12, 2019 · 2:30 p.m.
837 views
Making Our Future Better
Viktor Klang, Lightbend
June 12, 2019 · 2:31 p.m.
1682 views
Testing in the postapocalyptic future
Daniel Westheide, INNOQ
June 12, 2019 · 2:32 p.m.
498 views
Context Buddy: the tool that knows your code better than you
Krzysztof Romanowski, sphere.it conference
June 12, 2019 · 2:33 p.m.
394 views
The Shape(less) of Type Class Derivation in Scala 3
Miles Sabin, Underscore Consulting
June 12, 2019 · 3:30 p.m.
2321 views
Refactor all the things!
Daniela Sfregola, organizer of the London Scala User Group meetup
June 12, 2019 · 3:31 p.m.
514 views
Integrating Developer Experiences - Build Server Protocol
Justin Kaeser, IntelliJ Scala
June 12, 2019 · 3:32 p.m.
551 views
Managing an Akka Cluster on Kubernetes
Markus Jura, MOIA
June 12, 2019 · 3:33 p.m.
735 views
Serverless Scala - Functions as SuperDuperMicroServices
Josh Suereth, Donna Malayeri & James Ward, Author of Scala In Depth; Google ; Google
June 12, 2019 · 4:45 p.m.
936 views
How are we going to migrate to Scala 3.0, aka Dotty?
Lukas Rytz, Lightbend
June 12, 2019 · 4:46 p.m.
709 views
Concurrent programming in 2019: Akka, Monix or ZIO?
Adam Warski, co-founders of SoftwareMill
June 12, 2019 · 4:47 p.m.
1974 views
ScalaJS and Typescript: an unlikely romance
Jeremy Hughes, Lightbend
June 12, 2019 · 4:48 p.m.
1377 views
Pure Functional Database Programming‚ without JDBC
Rob Norris
June 12, 2019 · 5:45 p.m.
6374 views
Why you need to be reviewing open source code
Gris Cuevas Zambrano & Holden Karau, Google Cloud;
June 12, 2019 · 5:46 p.m.
484 views
Develop seamless web services with Mu
Oli Makhasoeva, 47 Degrees
June 12, 2019 · 5:47 p.m.
785 views
Implementing the Scala 2.13 collections
Stefan Zeiger, Lightbend
June 12, 2019 · 5:48 p.m.
811 views
Introduction to day 2
June 13, 2019 · 9:10 a.m.
250 views
Sustaining open source digital infrastructure
Bogdan Vasilescu, Assistant Professor at Carnegie Mellon University's School of Computer Science, USA
June 13, 2019 · 9:16 a.m.
375 views
Building a Better Scala Community
Kelley Robinson, Developer Evangelist at Twilio
June 13, 2019 · 10:15 a.m.
245 views
Run Scala Faster with GraalVM on any Platform
Vojin Jovanovic, Oracle
June 13, 2019 · 10:16 a.m.
1342 views
ScalaClean - full program static analysis at scale
Rory Graves
June 13, 2019 · 10:17 a.m.
463 views
Flare & Lantern: Accelerators for Spark and Deep Learning
Tiark Rompf, Assistant Professor at Purdue University
June 13, 2019 · 10:18 a.m.
380 views
Metaprogramming in Dotty
Nicolas Stucki, Ph.D. student at LAMP
June 13, 2019 · 11:15 a.m.
1250 views
Fast, Simple Concurrency with Scala Native
Richard Whaling, data engineer based in Chicago
June 13, 2019 · 11:16 a.m.
624 views
Pick your number type with Spire
Denis Rosset, postdoctoral researcher at Perimeter Institute
June 13, 2019 · 11:17 a.m.
245 views
Scala.js and WebAssembly, a tale of the dangers of the sea
Sébastien Doeraene, Executive director of the Scala Center
June 13, 2019 · 11:18 a.m.
661 views
Performance tuning Twitter services with Graal and ML
Chris Thalinger, Twitter
June 13, 2019 · 12:15 p.m.
2003 views
Supporting the Scala Ecosystem: Stories from the Line
Justin Pihony, Lightbend
June 13, 2019 · 12:16 p.m.
163 views
Compiling to preserve our privacy
Manohar Jonnalagedda and Jakob Odersky, Inpher
June 13, 2019 · 12:17 p.m.
301 views
Building Scala with Bazel
Natan Silnitsky, wix.com
June 13, 2019 · 12:18 p.m.
565 views
Diversity and Inclusion Panel: Bring the Thunder!
Panel
June 13, 2019 · 2:30 p.m.
245 views
Asynchronous streams in direct style with and without macros
Philipp Haller, KTH Royal Institute of Technology in Stockholm
June 13, 2019 · 3:45 p.m.
304 views
Interactive Computing with Jupyter and Almond
Sören Brunk, USU Software AG
June 13, 2019 · 3:46 p.m.
681 views
Scala best practices I wish someone'd told me about
Nicolas Rinaudo, CTO of Besedo
June 13, 2019 · 3:47 p.m.
2707 views
High performance Privacy By Design using Matryoshka & Spark
Wiem Zine El Abidine and Olivier Girardot, Scala Backend Developer at MOIA / co-founder of Lateral Thoughts
June 13, 2019 · 3:48 p.m.
754 views
Immutable Sequential Maps – Keeping order while hashed
Odd Möller
June 13, 2019 · 4:45 p.m.
277 views
All the fancy things flexible dependency management can do
Alexandre Archambault, engineer at the Scala Center
June 13, 2019 · 4:46 p.m.
389 views
ScalaWebTest - integration testing made easy
Dani Rey, Unic AG
June 13, 2019 · 4:47 p.m.
468 views
Mellite: An Integrated Development Environment for Sound
Hanns Holger Rutz, Institute of Electronic Music and Acoustics (IEM), Graz
June 13, 2019 · 4:48 p.m.
213 views
Closing panel
Panel
June 13, 2019 · 5:54 p.m.
400 views