Player is loading...
Brave New World - tales of PureScript and Haskell in production
Felix Mulder, Snoop Dogg
Friday, June 14, 2019 · 10:22 a.m. · 28m 52s
The rumours are true. Writing code in purely functional languages tends to produce code that is much easier to read, modify and reason about. This talk examines how an experienced Scala team transitioned into writing production code using PureScript in AWS lambda, and services using Haskell. Felix Mulder: Former Scala 3 Compiler Engineer at LAMP, EPFL. Currently writes code for Snoop Dogg, building the next generation of banking at the Swedish payments company Klarna.
Embed
Transcriptions
Note: this content has been automatically generated.
00:00:00
it's like they actually location begin and how many people here have read this book it's a a
00:00:10
like fifteen people okay cool how many people know of this book
00:00:15
alright cool so you're like me like you know this classic you haven't read that's fine
00:00:21
um this talk is not gonna be a technical talk at all i have a few slides
00:00:28
that are gonna be code but most of it is just gonna be a brain down of
00:00:33
last two years of working in industry um at china
00:00:37
so what is this talked now it's uh see concerts let's
00:00:44
yep a it's not a talk that will be you saying like oh
00:00:48
yes i mean use this one tool is gonna solve all of my problem
00:00:52
it's not the point of this talk at all um i so loathe about me i i
00:01:00
used to actually work in this building or not this one in particular but one over there
00:01:04
uh after i finished my masters i went here to be a a part of the lab line plot
00:01:11
um and we worked on the body compiler or as it's known and i guess this coffee from power
00:01:18
um so then i decided that once you really live with my girlfriend in stockholm those huge thing for me
00:01:25
and i started work at this interesting company call quite now
00:01:30
then what twenty other well i write castle first they've got
00:01:35
which is a really nice thing um we have this pretty cool oh
00:01:41
a commercial that the production load on the slide fine that's always interesting thing
00:01:47
uh anyway like big commercial depicts snoop he's
00:01:51
like he goes up any gets coordinated asked smooth
00:01:54
the dog which is like our motto a client everything's supposed to be seen for the customer and
00:02:04
you can actually come to our offices and you can try on these really cool rings so if you're ever buy stock homogeneous out
00:02:11
i want to sort of also have like a historical aspect to this at all
00:02:18
depicting how we as a company went from doing this uh essentially java services
00:02:24
on to do like purely functional scroll and then later on now something different
00:02:29
and this i really like this picture in the bottom left the
00:02:34
evolution of man into like a lamb the man or something like this
00:02:40
which are obviously stolen from oscar and so up up until
00:02:45
sort of like two thousand fourteen a klein was mainly doing java
00:02:50
uh on like at least on on the marker server's side yeah we've
00:02:54
always have like this huge monolithic underline which is really incredible piece of technology
00:02:59
if you're interested in airline or the actor mold and it's it's a really interesting system
00:03:04
um but there were some issues with that system and it kept crashing
00:03:10
and so they they decide like we should just you know break this model
00:03:14
of the park we should have my services they should all be written in
00:03:17
jobless spraying and so on all the scary things all the beans were there i think
00:03:23
um and then at some point in two thousand fifteen there was this one engineer
00:03:29
we said i'm gonna quit if i don't get to write this next services go
00:03:34
and it turned out that this guy was pretty important to the company so
00:03:37
man's was like damn okay we'll let this guy right the thing in school
00:03:42
um and the stars small you know like with not and here's and um
00:03:47
um they started with writing something that wasn't basically you know
00:03:51
job without the cynical i think we've all been there some point
00:03:55
and the next thing they they when they sort of start getting their
00:03:59
toes a wee bit more was start building this a framework for events sourcing
00:04:06
and after a while they were sort of like now are more comfortable
00:04:09
with scarlet language we feel that you know like how to write idiomatic code
00:04:14
so then they took like the services they were they were writing in drop with certain spring buttons are like
00:04:19
creating wrappers around those to make the use of those frameworks a more you mad
00:04:26
and of course like they wrote the second one sourcing library because you know why not
00:04:33
um and then they wrote the third one and this one is really funny because it's called install
00:04:40
fest which in swedish means party and it was supposed to be like a functional again starting to get
00:04:46
but the problem was like it was neither functional as in f. p. or functional as in worked
00:04:55
that's always something right alright and then in two thousand seventeen i was introduced to the company
00:05:01
and i'm not sure there's causation that definitely correlation that the team decide to go fully functional
00:05:07
we decided to go like purely functional a scroll up we were writing our services in the what we call
00:05:13
the title staff and i'm sure you would agree as h. t. v. four s. u. v. yeah et cetera
00:05:20
so we had three teams are so the collaborated around this that and build like sort of that
00:05:27
necessary infrastructure in order for clarinet to create this this you
00:05:31
know it stupid things like boss extraction of certain headers et cetera
00:05:36
and of course like we we could not not write another enforcing library
00:05:42
so i wrote one actually i wrote i'm responsible for this one
00:05:45
i'm sorry it's not one person's based on a route to be
00:05:49
uh it's about like five hundred lines are so like it's really small and it works incredibly well so
00:05:55
yeah yeah we haven't open source this and i'm sorry like we should of uh but i just don't wanna maintain more code so that's why
00:06:02
uh if you're interested like i can zip it and send it to you so that you can talk to me afterwards
00:06:08
um no but well the reason like why we like we
00:06:14
we've all made this like journey words going more more functional and
00:06:18
plan and the engineers the final or really happy doing purely functional programming
00:06:23
and this is sort of like the quintessential thing about their journeys like the more functional stuff we did
00:06:30
he more uh certainly thought that are code was working it was doing the things that was supposed to
00:06:37
um which makes this next section so hard i was sort of pretty
00:06:42
faces with saying that i still love spell i really love start language
00:06:47
and for me it's it will probably always be like to go to language it's the one i'm the most efficient in but
00:06:55
there are some hot and spun doing purely functional programming style so let's talk pros and cons
00:07:03
so one of the pros this those on uh it's compatible with java it's
00:07:07
on the g. d. and i think we can all sort of agree that
00:07:10
the initial part of cost success was due to it being on the j.
00:07:14
v. m. with all the libraries et cetera being available from from the get go
00:07:19
but maybe that's just sits from a purely functional programming perspective that's
00:07:24
what we care about we care about having a mature uh ecosystem
00:07:29
the other thing we really enjoy about writing things installer is
00:07:34
how easy it is to make errors uh uh purina type
00:07:38
so using things like either teeny sorry there is whatever these wanna
00:07:43
constructs might be that encapsulate errors in the way that we care about
00:07:48
and the other thing we like this like not banning these things together that shouldn't be back
00:07:52
together so having a new type rappers is also something that's very valuable to us as programmers
00:07:59
and in doing purely functional programming there's sort of this
00:08:04
there's there's a feature to it where if you make all your errors uh appear in the types
00:08:11
you sort of how a form of effect tracking insider system which means that you really only
00:08:18
need to do edge validation which is sort of the killer feature of function for me i think
00:08:25
and the other pro it's you guys are you folks
00:08:28
um there's a very strong community around functional programming install
00:08:33
and there are many opinions that might be differing in some cases but i think that just makes
00:08:40
the entire case a stronger for uh for doing
00:08:44
functional programming it's got so many people care about
00:08:49
so what about the drawbacks them to doing purely functional programming
00:08:52
style as well as we mentioned it's on the j. v. m.
00:08:58
and frosting purely functional programming having to interact with job well that's sort of means that we have the wrapper
00:09:04
or rap a lot of side affecting code we have to take care of that mentally and maintain those but
00:09:14
the other thing that i really say with a heavy heart is that eating purely functional programming install it's hard
00:09:21
i am a like initially that was my job but climb teaching our engineers how to do purely functional programming skull
00:09:29
and this turns out to be nontrivial i mean we had some success cases for instance
00:09:34
my boss any machine i'm a bit here um my boss did
00:09:37
not know of the keyword bar until six months into our project
00:09:42
pretty happy with the one and but there are some audience like some syntax for instance that
00:09:49
you you'd have to keep explaining over and over to people like what is this underscore do here
00:09:55
like with this mean here with me here with me here there are so many things you have
00:09:59
to teach and there are other audience uh for
00:10:03
purely functional programming uh with things like trying projector
00:10:08
how many people here have explained kind projected with the callings
00:10:14
yes most you excellent how many people think they got it the first time
00:10:19
yeah yeah maybe not so the thing is like in in other languages where you have a caring of type parameters
00:10:27
uh or uh types really this is not an advanced features
00:10:31
more of a basic feature that you that that we talk about
00:10:36
so that's one thing that i think is really hard with digits the other thing is that's all it is an opinionated
00:10:42
if we want to do a very strict subset of scroll up with purely
00:10:45
functional subset of scott well then we have to teach people what not to use
00:10:50
which is much harder than saying like here's a language just use it however anyway
00:10:57
um when it comes to how i know works we had some big
00:11:02
issues with that actually um how many people know that you'll be sees blocking
00:11:08
one person at least good all actually so um
00:11:13
if you have a web service that's fully asynchronous you sort of want it
00:11:16
split execution between two different that puts right so we have this one service
00:11:22
the team was pulling up and they were essentially they were doing the right thing in the application go
00:11:28
but when they call to the database they would shift it
00:11:31
to the uh blocking sample good and then they would shift it
00:11:38
uh_huh back to blocking for people i didn't know which thread probably we're shifting it onto
00:11:44
so we sort of storm the one that we just put all the threads on to
00:11:48
whatever something touch the database just moved all the friends over there which is really hard
00:11:52
to see in code and you can you can sort of new type your way around this
00:11:56
one type for the blocking in one type for non blocking but still and
00:12:00
unclear thing to see in the code you don't actually see where things are executed
00:12:06
and the other thing is type inference uh with f. e. as well i think we've all been there were we like
00:12:11
the compiler should get this and you're like be in we
00:12:14
we explicitly put the arguments and the compiler agrees with us
00:12:19
i mean this is much better and saw three might be but it's still something that's pervasive in school too
00:12:26
right and
00:12:29
i have this opinion i think it's sort of in popular or unpopular
00:12:35
is that i don't think that purely functional programming is did you manage to scroll all the way we're doing it right now
00:12:42
and i think a lot of the reason why this isn't is because we
00:12:46
took a lot concepts from high school and copy them verbatim on to spell
00:12:51
and this is an area where we as a community need to experiment and proof
00:12:56
um i think that's the things are done with intersection types does that they owe for instance
00:13:01
is one of those things that sort of move the needle in the right
00:13:04
direction using the features of the language to make a functional programming more ergonomic
00:13:16
yeah i i still want to say that i really love this language as as a power user
00:13:24
it's the best possible language you can that you can mold language to be exactly what she wanted to be
00:13:29
and that is something that at least i greatly appreciate about language and
00:13:34
as a library offer that's something i sure most people here would agree with
00:13:40
no tomatoes yes
00:13:43
cool site so this r. c. l. i'm pretty sure this photoshop like tea
00:13:49
he's not that athletic and all the fires of this but uh anyway so he
00:13:54
said as he said like china is gonna be the it'll be as of nineteen
00:13:58
so what he meant with this was like if you are you can take start up you want to have back
00:14:03
bending capabilities you should be able to go to us use our banking license and then just you know be trappers services
00:14:09
so okay cool great uh the problem is that my team has to build this so my teams paul the but the ruse
00:14:16
where part of the core banking the meaning that far enough where we are a team
00:14:20
responsible for inter ledger transactions that means that we move money from or between different ledgers systems
00:14:28
and when i was given this task starting the steam 'em you might callings we were debating like
00:14:33
what section use it should we continue with the type of that should we adopt something else and
00:14:40
we sort of came to this volume were like let's try to wait or
00:14:43
utopia let's try to create something that's even better than what we had before
00:14:48
we knew what we love about a functional programming type levels call and we also knew what we liked
00:14:55
so in trying to you or in wanting to stay
00:14:59
uh in the purely functional world we decided that the next
00:15:04
stepping stone this direction would be to actually use a a purely functional language
00:15:12
so that's what we decided to do it and we also decided to expand their novelty
00:15:17
budget and use this thing called service why not write like you got expand service but um
00:15:24
we're deeply invested made obvious but okay so we're using their version of service
00:15:29
and the you start this project by using these three technologies really so we're using next as like a
00:15:36
for we have um on a rico as we use next demands all
00:15:40
dependencies and is used to get a real recognisable bills and so on
00:15:45
we just pure script initially and i'll tell you why the meyer away from that that
00:15:50
and your script is language is very similar to hassle but it's
00:15:55
it compiles to uh to javascript so can use with no g. s.
00:15:59
which was at the time one of the few supported confines for it'll be just service and we start from for the point
00:16:06
and in this manner you go there a bunch of other languages we have a bunch of closer we have a bunch of python um unit
00:16:13
right so the deployment strategy for this we both this uh
00:16:18
something that really matters here is what we're building is like a
00:16:21
white label solution where we have to be able to like be deployed the world which also gives us some really nice features because
00:16:28
if we build platform in the correct way we can deploy the world when we want to test something
00:16:34
which is a really neat feature you can just tell everything would interact in the cup
00:16:39
so we have this concept of environment test was just justice he's been up entire environment
00:16:44
the complete platform for everything's we're building and you can run your tests yes of services
00:16:50
um so then we have the usual sort of strategy where you know you have a
00:16:54
branch to do some arm and tested the system test the unit tests you name it
00:16:59
then we merge that the master if we are at the head of the branch we don't actually have to be test any anything
00:17:05
um we can simply promotes to staging and and from staging we'll just go
00:17:10
to production because you have these four zones of a deployment new us staging inflection
00:17:17
alright so that's sort of the promise right so once we have
00:17:22
this now we sort of have to decide how to f. e.
00:17:26
and i do a little confused and this guy came up i've no idea line
00:17:31
um but seems like you would have a hard time choosing between tigers final and t. l.
00:17:36
three mon ads and haven't had errands or whatever um
00:17:42
so here's the thing all these diets work um if you could if you
00:17:48
want to do one of these than commit to it and don't go halfway
00:17:55
uh and and also my advice is to make it easy for yourself
00:17:58
um someone once told me that when you write code you have to be
00:18:02
uh twice as smart when you're debugging the code so if you write the code
00:18:05
in the most smart way possible then you're gonna have hell when you're trying to but
00:18:11
um so pick something that you really understand we do this
00:18:15
and and part of this goes like into the novelty but i was talking about before
00:18:19
um and you can like if you have a new review product
00:18:23
because you can't really ways things and on things you don't understand
00:18:28
uh of course like we did not follow this at all and we chose to do all of these things like why not
00:18:34
so we start with the handler patterns and then with the user uh for a while
00:18:39
no it is really cool thing in a pure script that's called prototype all morphing functions
00:18:44
um and then of course like you guys have tiles finals i mean we're all hips or spirit
00:18:49
um and wrote type amorphous and it's something that we just want to show really quickly because
00:18:55
pretty cool concert ah so in a impure script you can actually have like an alias for
00:19:03
a specific role inside record so you can say like i have a a record here that's
00:19:10
persist uh that has a specific field called persist
00:19:15
that's to ask the function signature ate half of unit and you don't really have no
00:19:19
after this if you equate that with io uh but it's the circular shape variant of that
00:19:25
and this is really neat because um this allows you to sort of empower functions
00:19:31
so in the function that you see here at the bottom do persist and call back
00:19:36
you can say that this one takes a record which hath
00:19:39
if you persist will pigments and something that's called back trouble
00:19:44
this means that we sort of we can empower functions to have a certain traits o. or
00:19:50
actually certain capabilities so that means that we can sort of limit
00:19:54
the scope very much in what we have to test for each function
00:19:57
and as you can see like we're passing this uh it's like this a union type here we pass
00:20:02
some or at the end so we don't really care about the other feels we just say stating simply that
00:20:08
we want these two capabilities at least for the thing that's passed into the do persistent call back function
00:20:15
so this would be something that's equivalent to the handler problem but then uses a a function of language which is pretty cool
00:20:23
alright so we had this and then we went to this
00:20:27
so what happened really well why don't you switch it's all
00:20:33
comes back to this novelty budget that was talking about before
00:20:37
just give us a really cool language and improves on hospital in a
00:20:40
number of ways one of which is that it has the dog operator
00:20:44
which is something that sounds very trivial to anyone
00:20:47
who's done and like see a syntax style programming before
00:20:51
but the doctor britain has bill doesn't exist the doctor britain has call this function composition um so uh
00:20:59
having this impure she is actually something you nice on the other hand
00:21:02
your script has sort of positioned itself as being a front end line which
00:21:07
which means that there isn't that much support for doing back and things and
00:21:10
my team is sort of like where the back ends back in the back and
00:21:15
so we don't ever see front and code which to me is really awesome but it also means that sure script might
00:21:21
not be the best fit for things that we're doing it
00:21:25
ends up with us having to maintain all these wrapper funk
00:21:29
calling it an odious libraries and then we're sort of back to the same thing we had
00:21:34
uh with a a purely functional spot where we had the meeting all these
00:21:37
rappers digit embarks so that's that was something that was here to really and
00:21:46
the thing is the castle is just so much more mature and your script and
00:21:51
i sort of feel that one of the reasons why we can say this is
00:21:55
because it took us one which to me write an application we'd written for six months
00:22:01
it's really cool thing and parse this i believe is because it is a
00:22:05
purely functional application and that your script is sort of like a dialect passable popular
00:22:13
and the other thing that we stopped doing was the service though
00:22:19
so sort of this is a really interesting concept and i do believe in a few years time it will
00:22:25
be that it would be the fact that way of doing things but right now it's sort of not really there
00:22:31
you have to test the entire application to make sure that things are working which is something that's
00:22:36
not ergonomic it doesn't give you have passed the backlit that you want them as a developer
00:22:42
and also this thing we had before where soak oh it took sixteen minutes deplore
00:22:48
well that was a lie because we were not able to
00:22:51
a pair rises for various reasons so these are actually sequential
00:22:55
so then that would mean that the pipeline take something like twenty five minutes round
00:23:01
uh we eventually got this down to six minutes so now we're running like easy to
00:23:06
uh containers that the servant applications and we do component
00:23:10
tests and integration tests on locally in doc arrest way
00:23:15
yeah so
00:23:18
how do we write code now well this is the just that it used tae was
00:23:22
final like this to sort of empower each function so you can see the handle request here
00:23:28
it's literally only able to do these things that are in the uh requirements list here it's only
00:23:35
able to persist things it's only able to respondents only able to block it can't do anything else
00:23:43
having having this amount of control v. we really
00:23:48
minimises the scope of what each function can do
00:23:51
and that's something that's incredibly powerful very happy with um
00:23:55
i was planning on showing um more of like how
00:23:59
we deal with this um and how this is implemented but i think we're gonna have to cut that a bit short um
00:24:06
all speed to instead so what you know it's like you the findings type classes like this that
00:24:11
becomes interface and use those functions inside um the
00:24:15
implemented function so uh having handle request like this
00:24:20
the first thing you do is you can imagine what type request to this then you can say i want to
00:24:24
present the transaction and we can put the type whole high school tell you what's the type of this what's expected
00:24:30
they say i want to respond with either error or created depending on the outcome of the first function
00:24:38
alright and then there is a is short example of
00:24:41
how to implement the persist um the mona persist interface here
00:24:46
and it's a pretty simple thing the nice thing to note here is that uh the requirements for building an
00:24:51
instance of this is that you have a database connection through some are and that you're able to log that's it
00:24:59
right and testing is simply new type wrapping the instance and
00:25:05
starting out the things you don't care about which is really good
00:25:10
um so the pros and cons of doing castle is like the record thing is really a big
00:25:15
deal actually like you you early on your you
00:25:19
name your feels inside records things like uh amount currency
00:25:24
but then that turns out to be a horrible idea because
00:25:27
uh records are actually just syntactic sugar for functions onto product so
00:25:32
terrible idea um if you know about this up front you
00:25:37
uh we'll of course prefix everything nicely and you'll have
00:25:40
a lenses and so on the right automatically for these types
00:25:43
but that's something that we as a team did not invest in that from the get go
00:25:49
and the other thing is extensions you sort of have to a
00:25:52
a line in the team on which extensions you use some are
00:25:56
some change type inference and will change the meaning if you program others are very
00:26:01
innocuous like overload the strings that's something that i think every hassle program you use
00:26:06
and the other thing is like a great bill to war it's always nice to have we're not yet
00:26:13
okay so um there are some other guy just to using a hassle friends of
00:26:19
the cup the libraries based on the body cafe which is the library that fully synchronous
00:26:23
which is really cool because it's incredibly fast on the other hand
00:26:27
the call back type lies to you it's sort of tells you that
00:26:31
uh you get back like when you send something you get back and am of maybe half care so if you get back nothing
00:26:38
then everything went fine no that's not true it's like if if you
00:26:42
get back nothing then that means that the pretty fly checks worked out
00:26:46
but it will actually everyone's trying to send it on a different apple so yeah those are some things to watch out for
00:26:54
but the pros and so we sort of get the same um
00:26:58
see features as we got with a purely functions colour where we're
00:27:02
able to re factor entire code base uh without fear of breaking anything
00:27:06
we um get something that's very performance you don't can
00:27:10
call starts which for us in the thin tech industry is
00:27:13
something that's a non starter like we have to respond within a certain
00:27:16
timeframe if you have like three instances that will call each other and you
00:27:20
have a a dependency on but the latency can't be more than two seconds
00:27:25
and all these instances are called i know nothing before you're gonna lose income
00:27:31
right and the ecosystem is really great all libraries that you could possibly want exist
00:27:37
and the last thing that is important to me is that it's training people on hospice shockingly easier
00:27:44
it takes us a i think three to four weeks to get people fully
00:27:47
productive whereas with purely functional scoffed assessments so that's sort of like the biggest thing
00:27:54
and in the end you have to ask yourself what are you optimising for me i'm up devising for my team
00:27:59
into them staying productive and for us half plus just solution for you it might be something different i think remote
00:28:14
ah okay we have time full one would question all the next speaker sitting up
00:28:22
how do you could base how big is michael but it's um
00:28:27
so we run mike or services right so i think our our
00:28:31
main has a product now it's about ten thousand lines of code um and then worse you have different services that are small and it's
00:28:39
okay it was very quick maybe another yeah
Conference Program
Keynote: Some Mistakes We Made When Designing Implicits (And Some Things We Got Right)
Martin Odersky, Professor EPFL, Chairman Typesafe
June 14, 2019 · 9:13 a.m.
303 views
Brave New World - tales of PureScript and Haskell in production
Felix Mulder, Snoop Dogg
June 14, 2019 · 10:22 a.m.
151 views
Actors Design Patterns and Arrowised FRP
June 14, 2019 · 10:53 a.m.
Taking Resources to the Type Level
June 14, 2019 · 11:48 a.m.
Lord of the rings: the Spire numerical towers
Denis Rosset, researcher in quantum physics
June 14, 2019 · 1:47 p.m.
Formal verification of Scala programs with Stainless
June 14, 2019 · 2:20 p.m.
118 views
Exploring Scala Tooling ecosystem
Jeferson David Ossa, Colombia
June 14, 2019 · 3:12 p.m.
TwoFace values: a bridge between terms and types
June 14, 2019 · 3:43 p.m.
GADTs in Dotty
June 14, 2019 · 4:41 p.m.
302 views
Closing Remarks
June 14, 2019 · 5:49 p.m.
Recommended talks
Fast, Simple Concurrency with Scala Native
Richard Whaling, data engineer based in Chicago
June 13, 2019 · 11:16 a.m.
624 views
Closing panel
Panel
June 13, 2019 · 5:54 p.m.
400 views