Player is loading...
Building Scala with Bazel
Natan Silnitsky, wix.com
Thursday, June 13, 2019 · 12:18 p.m. · 45m 58s
Compilation of very large Scala codebases can be quite slow, Enter Bazel - an open source build tool from Google that allows for very fast build times while maintaining correctness and reproducibility. In this talk I will share with you the story of how at Wix we have embraced Bazel and managed to dramatically improve our build times. I will also talk about "Rules Scala" - the Scala plugin for Bazel that has cool features such as "strict deps" and "unused deps" that help keep your build configurations clean and up-to-date. Required Knowledge: Basic knowledge of build tools such as sbt and basic jvm knowledge such as .class and .jar files. Natan Silnitsky has more than 12 years of experience as a software engineer. Natan has joined wix.com 3.5 years ago. For the first two years he founded and then served as backend tech lead for payments solutions services offered for visitors of Wix sites. During the last year and a half he has been part of a task force that is responsible for building the next generation CI system at Wix on top of Google's Bazel.
Embed
Transcriptions
Note: this content has been automatically generated.
00:00:09
okay um i would like to start with the quick poll
00:00:13
who here the raise of hands using a maven under the today
00:00:19
okay as this and a great all was using great on a day to day
00:00:25
okay nice and s. b. t. okay so a visitor
00:00:30
stick uh as the t. a crowd so uh welcome everyone
00:00:35
my name is uh not uh so let's key and then
00:00:38
a backend in for infrastructure developer at a with dot com
00:00:44
and i was part of the task force at weeks back and that was in charge of well
00:00:52
and like i have a okay we're
00:00:58
um okay so uh i was part of the task force that was
00:01:02
and charge of migrating always back and code base from even to baseball now
00:01:09
basal is google was a bill tool which they use to build their try my report
00:01:15
with more than two billion lines of code which they open source the few years ago
00:01:19
and once we did this is a migration and weeks
00:01:25
we saw that we had a dramatic improvement
00:01:28
in development velocity and much happier developers so
00:01:34
you can see for example we have a one or a larger uh repositories and with may
00:01:41
then it took roughly forty five minutes to build
00:01:45
the entire ripple compile everything run all the tests
00:01:49
and no no matter what to realise the change that uh was a down
00:01:53
to the coat now with basal on average usually takes less than a one minute
00:01:59
i feel a numbers on weeks back and so we have a around
00:02:03
the two hundred fifty developers more than seven hundred and fifty micro services now
00:02:10
uh we've put in terms of code base you have more than ten million lines of code
00:02:14
mostly in scholar and but there is still some uh in java and also in o. j. s.
00:02:21
now we have thousands of builds running every day and we hit production uh as often as possible
00:02:29
so i guess so we we migrated from avon and our uh bill sorrows density and
00:02:36
the way uh our um call this is structured is we have many many sources
00:02:41
and a lot of code and also most of it depends on the same infrastructure code
00:02:47
so imagine thousands of a maven modules uh
00:02:51
depending on the same code the for infrastructure stuff
00:02:56
and it's all snapshot dependency meaning we wanted to
00:03:00
get changes immediately uh to all the dependence so
00:03:06
uh on this uh change to any of such a code we
00:03:10
have thousands of thousands of uh the elections uh keeping up until city
00:03:15
for each of the major modules and it took hours sometimes to get everything depleted
00:03:22
now with basal we just uh run it on
00:03:26
the cool cloud and we have a all repositories there
00:03:31
and they all compile and run the test simultaneously and all it usually takes just a few minutes to get
00:03:37
the root level change propagated to all of the dependency graph
00:03:44
okay so we'll today i'm gonna show you the
00:03:48
key features that make base so fast correct and scalable
00:03:52
and specifically how base so can help us improve
00:03:56
the scale scout built specifically and make them faster
00:04:01
so the key difference uh with base so uh
00:04:06
usually if the the paradigm shift to with other um build tools
00:04:10
is the explicit this and flexibility of the uh configuration definitions so
00:04:18
you can uh decide on which granularity you you would like to have the the basic build unit
00:04:25
you can decide to have like once colour source file and that will be
00:04:30
the action of compilation that you run or you can decide to have a the package level dependency
00:04:37
um or you can have a set of packages
00:04:42
that you run and this in the same action
00:04:46
that'll be closer to the s. b. t. project or may even module
00:04:51
now this basic building on it is called a target in baseball
00:04:56
and targets are instances of rules so you may ask yourself what is a rule
00:05:01
so are all is basically a function that has the set
00:05:06
of inputs and it provides a set of outputs it usually
00:05:10
are files right and there are many different kinds of rules
00:05:16
and that are defined for baseball we have sky library of course
00:05:21
but you can also have you also have rule for c.
00:05:24
plus plus binary or running a python test running shows script
00:05:29
basically they is all it is a generic dependency graph
00:05:34
action execute or so it has many many different kinds
00:05:39
of rules for almost any language you can think of but uh we wanna focus on scholars so for example
00:05:47
you have here a very a simple a scar a library target a definition
00:05:54
so it's named a. e. and it as uh the source files
00:05:58
you explicitly declare which also files uh are for this uh target and
00:06:04
the and this case it's a that's caught and also depends
00:06:07
on target c. which in turn will compile see that's colour
00:06:13
and the output will be a dutch are and how do you invoke it you just
00:06:17
uh type basal build colour nay if you're in the working directly out of this package
00:06:27
okay so what other kinds of inputs can these uh rules and a half so
00:06:34
we saw that you could be source files can be dependencies but they can also be all the tool configurations
00:06:40
so for instance all these colour compiler options are considered the uh an
00:06:45
input for this um a rule or which j. d. k. version you run
00:06:50
so basically any sort of input that has any uh relevance and effect
00:06:56
for the actions that are run by this rule has to be explicitly defined
00:07:01
for baseball and wise that because of the sandbox property so
00:07:08
if you have some implicit dependency in your code and some file
00:07:13
that you do not explicitly declaring your configuration then your build will fail
00:07:19
so this sandbox property actually guarantees that
00:07:24
no matter a watch you uh you have to define everything in
00:07:31
sight of the sandbox meaning that they still will isolate all of
00:07:36
the files that you declare in the file system and not give you access to any any other part of the file system
00:07:42
and uh also limits your network so that networking
00:07:46
some boxing means when you're running compilation running any test
00:07:50
will not be able to just go to the internet and doubt anything you
00:07:53
want no that does that's done in a previous stage so this the sandbox feature
00:08:00
actually gives us deterministic rip reducible belts because you always know exactly
00:08:08
which out input you gonna get there's no surprises so you always have the
00:08:13
same output no matter how many times you going to run the same target bill
00:08:20
i now and as i mentioned the flexibility of the build unit
00:08:25
usually it'll mean that you'll have many many
00:08:27
small targets um that's a great advantage because
00:08:32
when you run to completion it means that it would have a small impact for the specific target
00:08:39
and also the part of the tree that depends on the
00:08:42
start it will be smaller because we're talking about smaller granularity
00:08:47
but of course there is a a it's a trade off there are disadvantages
00:08:53
you mean you need to manage many more build configurations and you
00:08:58
need to maintain them and add dependencies between the different targets so
00:09:04
that is a disadvantage uh for this uh particular
00:09:09
feature but you can have a great tooling in place to help you with this management
00:09:17
now we have many more targets and that probably means that
00:09:22
not all of them down one over and they can run independently
00:09:26
and run more in parallel so the day so you you you tried by default
00:09:33
to run as much as many actions as possible at the same time on your
00:09:38
local machine you will be limited to the amount of course on your machine and
00:09:44
of course with uh the alternative tools you probably have a lesser degree of parallel work
00:09:52
now if you uh have a testing your code and then you probably get
00:09:58
parallel running of compilation and tests so you can see in a blue we
00:10:03
already have targets that i started to wander tests but at the same time
00:10:07
dependence targets are now running compilation and
00:10:11
so you get even higher degree of parallelism
00:10:17
and the great a feature of the sandbox and deterministic
00:10:24
uh running off the your build means you have a great incremental um function right because
00:10:31
you only need to a rebuild the small part of
00:10:37
your graph that uh you change some code and because everything
00:10:40
else is put in a cat in the cash with the hash key and you you you know that you can guarantee
00:10:48
that you don't need to re run it because basically guarantees that and think about your test code
00:10:54
if you don't change test code for really really slow integration tests there's no reason to run them
00:11:01
so you save a lot of time by using this uh correct incremental feature of basal
00:11:09
and we're not really limited by the local machine that uh running no
00:11:13
matter if it's on our laptop on this or the c. i. server
00:11:17
because basically has the cool feature of remote execution so each
00:11:22
action that is being executed actually sent to a remote worker farm
00:11:28
where you can have a crazy degree of parallelism so
00:11:32
we tweaks use the google clout solution all remote execution
00:11:36
have it as a service and we see that we have hundreds upon hundreds of
00:11:42
actions running at the same time and building are cool bass and running the tests
00:11:49
they're also open source uh solutions uh that you can
00:11:53
set up on your um cloud servers or on
00:11:57
promise and also enjoy the benefit of the remote execution
00:12:02
now if you so it's great for the c. i. server the the the
00:12:06
workers on this farm populate the cash and the c. i. server can of course
00:12:11
utilise the cash results uh from one go to the next
00:12:16
and if you want to run this uh actions from your local machine on your laptop you can do as well
00:12:23
now of course there will be network latency here so there
00:12:27
is a feature uh quite recent one in basal call dynamic execution
00:12:33
that means that you can decide whether it will try to run both locally
00:12:38
the action and remotely and it will stop this lower action was the fast direction
00:12:45
already finished so you get a the benefit of both
00:12:49
having a a local and remote execution and just even
00:12:55
uh having of fast to build a just managed automatically by the school feature
00:13:02
so at wicks we really start the benefits here uh i
00:13:06
uh shows a few numbers so you can get the sense
00:13:10
if you if we run the one of a large repositories like i said in a van and
00:13:15
now using um any caching mechanism whatsoever it's forty five minutes now if you use
00:13:22
basal on your laptop um and not using any cash you
00:13:27
all the jobs at the time to twenty minutes why because
00:13:31
it's a fast they have higher degree of terrorism we have smaller uh targets
00:13:36
so you get faster time and of course if you run it on the remote
00:13:42
um environment with a work of far it all it drops down
00:13:46
to a point five minutes even though we didn't use any caching whatsoever
00:13:50
and this example now why do we still have a there's a bigger
00:13:55
number because we're still limited by the amount of terrorism again have because
00:14:01
there's the critical path of running um the dependencies right
00:14:06
uh because you can start running a dependent action before the last one finished
00:14:11
although yesterday we saw a great talk buys to put from twente are about work being done
00:14:18
um at whatever to have outlining which will mean that you can get the
00:14:25
header information from the action as soon as you can have it with a compiler
00:14:30
and then already you can start to depend action before the
00:14:34
previous action complete compilation uh finish completely so that can really speed up
00:14:40
uh the performance and is also work uh in this colour centre
00:14:45
about a type lining and outlining so it's really exciting times force colour compilation
00:14:53
and of course with the the fully cash an option
00:14:58
once you have the second run basal allies uh the depends
00:15:01
a grassy that it doesn't really need to add them and write anything and only takes a few seconds to finish
00:15:09
okay so i think that they still is definitely worth
00:15:14
it but it it's not worth it in all cases right
00:15:18
if you are small start up just getting started and you have a small code base and you just want to have
00:15:24
tools working out of the box naive to think about it
00:15:28
but maybe base so uh is not right for you because
00:15:32
there is work that you need to do of course to maintain the bill configurations
00:15:37
but i'll still you probably want to do the adjustments to your code to adhere
00:15:42
to these sam boxing principles this uh like i mentioned about file access and network access
00:15:48
et cetera so these are very good changes that you want to have for clean code and
00:15:54
module courts other r. f. but maybe for a small company that's a want want to run
00:16:01
as fast as possible it's not my field now as your code base roles and
00:16:09
you have many more dependencies the dependency graph looks bigger more complex
00:16:14
any maybe starting to think i know what does integration tests this
00:16:19
uh these automated automated tests they take a long time to run so
00:16:25
maybe i will run after every any code change uh just won a match each night or something like that
00:16:31
so then you start compromising between the quality of
00:16:36
your code and how fast you can get the production
00:16:40
so once you reach that point i really think that you should consider
00:16:44
switching to bass all because they shouldn't get part compromise you want to have
00:16:49
fast correct builds and get to production uh fast with a quality um build an action so
00:16:57
of course if you have a very large code
00:17:00
base then i think basal really fits in really naturally
00:17:05
and really is really optimise isn't even shines in such examples
00:17:10
of large code bases it really really makes a huge difference
00:17:16
okay so we saw a few of the uh basic um call a basic features
00:17:23
and then alan a switch over to how a base so works with car so
00:17:29
with basal out of the box it has support for c. plus plus in java
00:17:33
and for any al language you have yeah these extensions or plug ins for days
00:17:40
i'll call rules and in this case we're talking about the rules colour which is
00:17:46
the open source project in the bayes all a good health organisation
00:17:51
it's written and scar and stall arc so stark is the extension language
00:17:56
for for basal it's a subset of python which is not cheering complete
00:18:02
so you don't get into infinite loops and you
00:18:07
uh you so you build a any so you write anything that you want to have
00:18:12
as extension will see a few examples uh later on
00:18:15
so this uh project is call maintained by stripe and
00:18:22
by stride at and wakes and uh it's adopted
00:18:27
by uh many companies for example you have let's see
00:18:30
meet up dot coms modify the all use um this uh rules card version
00:18:40
okay so i just wanna have a quick a
00:18:44
refresher on the the difference in the structure although configuration
00:18:50
uh between baseball and other tools so for s. b. t. or a maven agreeable
00:18:55
you have a convention based approach where they uh have they know that
00:19:00
they'll find a weather for the production code whether find the test code
00:19:04
but with basal uh the build configuration is found adjacent to
00:19:10
your code so each package will have its on though configuration definitions
00:19:16
where you have well the definition for compilation and pass a et cetera
00:19:23
so which basic rules rules cost supports uh so the first one we already talked about is call library
00:19:29
which the action that needs to be executed this of course calling invoking this car compiler
00:19:35
and the output will be a jar file so here's the example i already showed
00:19:42
you get a da da char and the sources and dependencies
00:19:49
and the second the basic rule is colour binary you
00:19:52
want uh to have you want to run executables so
00:19:57
of course it will compile any source code and uh dependencies that you require
00:20:02
and it will also output other than the jar file it was awful the shell script and just to be we'll
00:20:09
just run the main class on a j. v. m.
00:20:12
that will spend up to act run the executable scale application
00:20:17
so here we can see an example um where you define in the skull binary definition
00:20:23
the main class in this example column example full
00:20:28
and and then uh when you invoke it with days
00:20:31
will run colby it will uh actually invoke the
00:20:36
shell script and find the appropriate to a main class
00:20:42
and the s. r. basic one is of course colour test so here
00:20:46
it's exactly the same mechanisms colour binary but with the shell script about
00:20:51
instead of executing the your main class it will execute this colour test runner
00:20:56
and here you can see an example where you have a set of a test files
00:21:03
so you define um do the wild card but uh all the files
00:21:07
that and with this that's colour will be part of um this uh
00:21:12
build target and you see that when you run basal test call an
00:21:18
example test it will invoke the basal test runner so quite simple there
00:21:24
so let's see a quick uh example of a working in repository
00:21:31
so i have a really really simplistic a scar people
00:21:34
here are just a simple additions and the numbers so
00:21:40
you see that uh the wild card can be used here and we have the it's
00:21:46
very simple scott test here and this is an example of how to depart depend on
00:21:53
the complete label from uh the relative to the workspace file
00:22:00
so the workspace file defines the entire ebb isn't we pro
00:22:05
and you get all the external dependencies that you need here including a
00:22:09
rose colour itself of course that once you compile it will first compiles colour
00:22:15
and then you'll be able to use it everything in baseball
00:22:17
is source dependency as much as possible so now just a run
00:22:24
all the tests uh in this report that just like riding basal test says that dot dot dot
00:22:33
i hope uh you can see and it's uh i've
00:22:38
already running before so it's got all the test results from
00:22:42
the cash so it's really a was really fast results and
00:22:47
of course if i change uh some of the one test
00:22:52
and make it will be a fail failure we see now it in a standard
00:22:57
some filers changed and will need to be re run so too 'cause i can't
00:23:03
so i'll just corrected again and you see that it also always has the cash results and
00:23:10
i was only to run the source file that has changed okay so that was really quick demonstration
00:23:19
now
00:23:20
what uh and of course you have many more rules that was caught provides you have
00:23:25
jane it uh spec status first libraries provider is which of course are also generated uh
00:23:32
code the other other code generators and them anymore and you can also add your own so
00:23:38
like i said it you need to have some familiarity with stark
00:23:42
but you know python is really easy to get into and you get
00:23:47
a few constructs yet if you have a bayes uh like the build context
00:23:52
et cetera and you can get going and add anything that she use find is missing from rules colour
00:24:00
okay so uh i will show okay fuel the key features that uh yeah are inside was car
00:24:06
that um will help you have um correct fast builds
00:24:11
so the first thing to know is that there's a can extensive and two and sweet it's battle tested you have
00:24:17
a lot of a regression at put inside there and
00:24:20
it also interesting lee verifies the would produce ability principles so
00:24:26
it checks that it builds all of the um build a at targets inside of the
00:24:32
report and then it makes sure that on a subsequent run with the time delay so
00:24:38
there was no random this put in and the hash is are exactly the same so
00:24:41
rules colour then makes sure that it adheres to do reproduce ability a function of baseball
00:24:50
now in your repository you may have modules from different kinds of a j. v. m. languages
00:24:56
uh to have jobs saw star groovy cock land and you we want
00:25:01
to have everything built with a source dependencies we don't want to manage
00:25:07
binary versions between them right so with basal and the rules for each of these
00:25:12
language that's possible uh because each of them had here's to a convention that they provide
00:25:19
java infrastructure so you can have the scout target
00:25:24
depend on the java target or something with caught land
00:25:28
uh there's no problem there and everything will be built from source and cash
00:25:35
now what about a dependency management so i talked about having more
00:25:39
verbose bill configuration morse small smaller targets meeting you have
00:25:45
a more dependency that you you need to add so
00:25:49
there are few and nice features here you have a flag for news dependencies that it will if you
00:25:56
set it up then it will scan each time you you billy run the build and will provide for you
00:26:03
warnings or errors on depends is that you no longer need and
00:26:08
you can remove them with bill those are which is a tool
00:26:10
that automatically manages the bill configurations for you and you just a
00:26:16
hack you keep your dependency graph smaller and that means faster built right
00:26:22
and also an interesting uh feature is strict dependencies so
00:26:28
this means that if uh you have a full library and it depends on the specs too
00:26:35
just a sweet and you also and it
00:26:38
in turn specs to depends on scads so what's
00:26:42
if you have in your full library source code import of cats
00:26:48
but you didn't specify it as other dependency in baseball
00:26:51
then streak that us a tool will emit an error
00:26:55
and say you need to the the the uh add a direct dependency here and why is that important well
00:27:01
first it's a basic uh code hygiene and and have
00:27:06
a more correct and uh write a book information is
00:27:11
and also a it means that your code will be less fragile it because the
00:27:15
specs to decide to drop cats at some point then you full library uh won't break
00:27:23
and both the and use the dependency checker understood dependence
00:27:27
is uh based on jason's dogs a class that frank appliance
00:27:33
okay we have a a very nice a tool we developed
00:27:37
at weeks that we use uh with our uh yeah it's
00:27:42
we to go with the intelligent environments built in with the additional plugin
00:27:47
and this was caught that fixer so what it does it once invoke
00:27:51
it it ought to next item at sorry automatically builds the basal um
00:27:57
runs the basal build and understands from the missing symbols which dependencies
00:28:03
it needs to add to the base of the bill definition now
00:28:07
um it knows which depends is uh it needs
00:28:12
because it indexes all the jars during the
00:28:15
build internally and also uses uh extol indexing server
00:28:20
for any code residing in other repositories or third party dependencies
00:28:28
and this uh to was inspired by a pants which has a similar feature for
00:28:34
automatically adding a dependency spends is tutors a bill tool
00:28:42
okay so very important a performance issue
00:28:46
uh with the basal and scar is
00:28:50
a warming up scott compiler and incur mentality in the
00:28:56
class level so out of the box basal will not
00:29:01
when we have actions that are independent of one another that means that
00:29:06
if you run an action it will spin up and you j. v.
00:29:09
m. with this call compiler cold spell compiler underline it so that really is
00:29:14
i'm quite expensive now if you have if you set up the persistent workers
00:29:20
like rules colour does that means that this is a basal feature persistent workers
00:29:25
it means that it will have like a pool of a scout compile hearts cotton powers that it
00:29:31
can use for the actions that are being run so this is really important for your local development
00:29:37
'cause you continuously one to to to check that your code still still correct now
00:29:45
with uh there's with who do we have an alternative rules car called
00:29:50
was colour at x. from the heart kind there's a a good have organisation
00:29:55
and the difference here is that it uses zinc uh uh i've come sure
00:29:59
that uh almost all of you know but think is these days full incremental compiler
00:30:04
used by blue and the s. b. t. so
00:30:09
will scale an ex uh well instead of using the standard colour compiler uses inc
00:30:15
so that covers us both in the caching the of the level
00:30:20
of the targets the bayes look ashes and also stop target level
00:30:25
with the class classes inside of the your source file
00:30:30
that zinc can help you with the the caching there
00:30:34
now it's not being used currently in the official um a standard rules
00:30:40
car because the base basal basically adheres
00:30:44
to status compilers and have the cash
00:30:47
uh being a handle the managed by diesel itself and it and you can
00:30:52
also not work like this in remote execution because my decision does not support currently
00:30:58
having state it's everything is status there's just been up new containers and run these actions
00:31:05
but it can that was gonna annex can definitely uh improve your experience
00:31:09
locally especially if you have bigger targets so you might want to consider that
00:31:15
and uh like i already said before there's active work
00:31:19
done at whether and it's cost centre to improve the situation
00:31:24
uh even more with the state for compilers and um
00:31:29
optimised compilers without finding a pipe like pipeline like i
00:31:32
mentioned so this is a a space that is actively worked on and uh we'll have improvements in the feature
00:31:39
so we'll see very a simple demo of uh
00:31:46
of using a persistent workers so here i have two identical
00:31:50
copies of rules collect 'em ripple itself and i just the
00:31:56
running the uh build off part of the test suites fools colour
00:32:02
and you can see here that all of the their eight actions
00:32:06
rank because i have eight course on my machine so bases trying
00:32:09
to utilise as much as possible and degree of
00:32:13
parallelism but um this is a a quite slow here
00:32:18
you can see it utilise is uh the darren sandbox
00:32:21
so that means it training in sandbox mode by default
00:32:25
uh there's no special flag here's automatically it invokes the sandbox
00:32:30
and means that you have you can rely on it and have a incremental
00:32:35
um build which are faster but in this case when it keeps on a building
00:32:42
and running in huge even instances with the calls colour compilers then it's much slower
00:32:48
okay i think it's about finnish had vanished took sixty one seconds
00:32:53
now uh i all i do hear the the only difference is that i'm running it
00:32:59
with strategies cause see worker which means that the persistent
00:33:04
worker is going to be utilised and you can see that uh the actions are
00:33:10
completing much much faster here uh and uh yeah it's going to find it's much sooner
00:33:24
yeah i took thirty seconds so it's a fifty percent improvement uh in this case
00:33:29
like i said you can have even better results with rules call an x. and
00:33:34
with the exciting a compilation optimisation is we have had
00:33:42
so that's uh that's about roles colour you can this find out the link uh
00:33:48
you can go to the link here in the base will build or the organisation
00:33:52
it's an old open source of course so we really recommend a really
00:33:57
like to get uh some uh contributions and for a fifty anything is missing
00:34:03
and uh if you wanna uh do we play around a little bit with the was colour uh of with a very simple uh
00:34:10
and exercises you can check out this link um
00:34:15
i have a link at the end the for a slight share so you don't need to um copy it now
00:34:23
if you're interested in learning more about the migration stories to base will
00:34:28
so i have a i give i gave a talk at g. conned for
00:34:33
about different things you need to consider
00:34:36
while migrating uh around the granularity about uh
00:34:41
how many repositories you want because basal works best with the mari po
00:34:46
and dependency management um the that's really quite different and basal and
00:34:53
also have an equivalent to the block series on on this topic
00:34:58
and i also recommend the x. on a block posed by the how it the data breaks
00:35:03
on space car built so they took a little bit different approach
00:35:07
but they have amazing results so really recommend checking that out as well
00:35:13
and we also open source the on automatic migration tool
00:35:18
from even two basal that will let you the dual does all the
00:35:22
hard work for you and also gets you to find a green targets
00:35:27
but only works currently in maven from even to base so of course we would be happy to
00:35:33
work on the the s. b. t. equivalent and we left to get the contributions here as well
00:35:41
so to summarise they still uh is worth it in most
00:35:45
cases and it gives you faster builds and ecosystem is becoming better
00:35:51
um weeks is actively working on tools for
00:35:55
intel a. j. to help with dependency management
00:35:59
and really let you focus on your source code and having
00:36:05
the tooling in place to help you with uh the build configuration
00:36:11
and we hope to open source uh the digit plug ins and the the that fixer tool very soon
00:36:19
and will sell itself is mature and battle tested and used quite
00:36:23
widely like i i'm mentioned so i really uh recommend checking it out
00:36:30
thank you very much
00:36:37
so here's the link first lighter than mentioned and you can check out a a follow me
00:36:42
on medium to it or to get more updates on the easel and wicks and yeah questions
00:36:49
uh_huh ha and it's for the talk and um i actually had two
00:36:55
cases what you just answered one which was about their you know id supported
00:36:59
is does to go into the idea so it will come out soon i
00:37:03
guess yeah you really have the official basal plug in of course um but
00:37:09
we have all kinds of cool additions an on additional plugin so we really
00:37:13
hope to open source it's okay that's great thank you and a second question is
00:37:18
uh so i'm using a as collateral for testing so in s. p. d.'s
00:37:25
just easy chip are prepared i had joe uh into u. s. b. t. settings
00:37:31
but uh well how ease ah how easy to use
00:37:34
two years changed it almost make it support ah there
00:37:39
are test frame locked and the ones that you mentioned
00:37:42
that likes cartels specs to stay right so basically um
00:37:48
you will probably need to create a a a to have the rule in
00:37:54
place to have the the test runner flavour that that you're working with so
00:38:01
it's a it's a matter of i'm getting from a from a
00:38:04
little bit with the style or cut the extension language for basal
00:38:08
and the are the building blocks in place to have
00:38:13
test runners um that alternative test one is that that you would like
00:38:18
and then uh you just have like the x. m. l. outputs for the test
00:38:26
or uh the i. d.s et cetera so i i guess
00:38:30
the basic work for just getting to work will not be
00:38:34
that great but uh for supporting the i. d.s environment it
00:38:38
it may be a bit more work it's it's it's possible i mean we did it for specs
00:38:42
to it was missing before and i didn't take too uh too much time too much effort thank you
00:38:53
it's consistent because it's not just compiling so it's a
00:39:01
ca good cop should work to do some work you see people all t. v. should system
00:39:09
is there's for for two years if if we could do there is loads of
00:39:15
plugin for the question alright right so yeah as the t. m. s. n. r.
00:39:23
both are with the school quite a long time so have varied ship like environment
00:39:28
uh so the of course uh with basal uh uh you have less of that
00:39:34
i'm not sure but coverage i'll check that but uh there will be if you if you
00:39:39
if you see something amazing you may need to to contribute it
00:39:43
to base so that anyone else can enjoy it as well it's
00:39:57
ah how car is circulation orders mm i mean uh if we change
00:40:02
some private lessons for example will all the balances be recompiled or not
00:40:08
or or it just looks for up abruptly changes
00:40:14
yeah so uh there is a tool in basic although i jar
00:40:19
that's a really works really great with the with java there are some
00:40:24
limitations all of it with basal and the idea is that um it only
00:40:30
checks the public a. p. i. is then you have a faster because the public a.
00:40:34
p. i. hasn't changed uh that means that
00:40:37
uh the only any re compilation and uh um
00:40:42
it's the same with the outlying principles that are actively worked on and uh at the
00:40:47
moment uh that once you get the headers and and you see that there's any change
00:40:53
then you only to recompile the or not depending uh on the result
00:40:58
uh so the there there is a solution for that i know that
00:41:01
i does not support uh macros uh tufts comeback was at the moment so
00:41:07
if you use that you probably uh won't be able to gain this feature
00:41:17
and thank you for the talk i have a very down to earth question in it's mainly around
00:41:22
or using version name so uh i mean that's got more deploying in made in main trends like
00:41:27
how do you get your charged artifact server can use result product and how how would you how
00:41:34
do you do version in how to do these things are two weeks come okay my question so uh
00:41:40
that is all you can do you can do a they are like generic rules
00:41:43
that you can run you can actually run anything you want in basal uh but actually
00:41:48
we it with and decided to leave basal for the build build a phase
00:41:55
and then you have access to all of the uh created the
00:41:59
jars and and wilbur job was you know you pack them up and
00:42:05
and then we have a a separate a service
00:42:09
that uh listens to the build events you have
00:42:13
the builder protocol for baseball and uh we just uh listen we know that
00:42:18
a new deployable uh was created and we send it to our by repository
00:42:24
and in terms of version is uh the the service just uh does a um
00:42:30
an r. c. just add does a increment the r. c. version at every time does
00:42:36
that that new the uh deployable and uh yeah so we do it uh with semantic burgeoning
00:42:51
i am how much work is maintaining all the build files as for example having to
00:42:58
build file in each package uh seems like a lot of work especially ball restructuring for example
00:43:06
yeah so it depends i mean you can't you can't stay
00:43:08
in the maven or a a as bitty levels project some modules
00:43:14
and then you it will basically be the same uh like uh uh
00:43:19
with the these tools when you add depends seem a varsity
00:43:24
but uh usually would for you want to have a smaller targets so
00:43:31
that if you would work without any eh additional tooling say
00:43:35
every time you do an import of something you don't have yet
00:43:39
you to go to the bill configuration an added
00:43:42
now uh that's exactly why weeks is developing these tools
00:43:46
uh for helping you with automatic a dependency
00:43:51
management for you so it's not the part
00:43:55
that's not a a basic part of base so you have the bulldozer that you can run
00:44:00
uh commands and have scripts that adam but you need to know what you want to add so the
00:44:05
automatic a dependency management uh something that baseball does not provide and we hope to open source that's all
00:44:15
hope to answer your question yeah
00:44:19
one more question so ah how those spaces or
00:44:23
are handled a complex independence emotions so is it
00:44:28
handled by to automated in those means what users
00:44:31
major though you have to worry right so um
00:44:36
first you can have sickly dependencies between uh
00:44:39
between a target so basal will air on that
00:44:43
now for the party dependencies you mean about versions of third parties so
00:44:48
the uh a whole bunch of tools
00:44:51
uh that resolve the maven dependencies basically
00:44:56
for your basal repository and usually they uh work like a one version policy
00:45:02
so that means you need to change your a. p. i.
00:45:05
there are um variations that to allow you to have multiple
00:45:11
sets of dependencies that you you should be really careful because the dependency
00:45:16
is defined for the entire repository that means you
00:45:20
may end up with the target that's using dependencies
00:45:24
from different trees and you know transit usually you get a conflict
00:45:29
and but they still not air on that so you'll get ran
00:45:32
the result in the class passed so you don't even you you're
00:45:36
not even guaranteed which version will uh will win the class paths
00:45:41
so the recommendation is just work with one version for a part
00:45:49
the more questions
Conference Program
A Tour of Scala 3
Martin Odersky, Professor EPFL, Co-founder Lightbend
June 11, 2019 · 5:15 p.m.
8337 views
A story of unification: from Apache Spark to MLflow
Reynold Xin, Databricks
June 12, 2019 · 9:15 a.m.
1267 views
In Types We Trust
Bill Venners, Artima, Inc
June 12, 2019 · 10:15 a.m.
1569 views
Creating Native iOS and Android Apps in Scala without tears
Zahari Dichev, Bullet.io
June 12, 2019 · 10:16 a.m.
2231 views
Techniques for Teaching Scala
Noel Welsh, Inner Product and Underscore
June 12, 2019 · 10:17 a.m.
1295 views
Future-proofing Scala: the TASTY intermediate representation
Guillaume Martres, student at EPFL
June 12, 2019 · 10:18 a.m.
1156 views
Metals: rich code editing for Scala in VS Code, Vim, Emacs and beyond
Ólafur Páll Geirsson, Scala Center
June 12, 2019 · 11:15 a.m.
4695 views
Akka Streams to the Extreme
Heiko Seeberger, independent consultant
June 12, 2019 · 11:16 a.m.
1552 views
Scala First: Lessons from 3 student generations
Bjorn Regnell, Lund Univ., Sweden.
June 12, 2019 · 11:17 a.m.
577 views
Cellular Automata: How to become an artist with a few lines
Maciej Gorywoda, Wire, Berlin
June 12, 2019 · 11:18 a.m.
386 views
Why Netflix ❤'s Scala for Machine Learning
Jeremy Smith & Aish, Netflix
June 12, 2019 · 12:15 p.m.
5026 views
Massively Parallel Distributed Scala Compilation... And You!
Stu Hood, Twitter
June 12, 2019 · 12:16 p.m.
958 views
Polymorphism in Scala
Petra Bierleutgeb
June 12, 2019 · 12:17 p.m.
1113 views
sbt core concepts
Eugene Yokota, Scala Team at Lightbend
June 12, 2019 · 12:18 p.m.
1655 views
Double your performance: Scala's missing optimizing compiler
Li Haoyi, author Ammonite, Mill, FastParse, uPickle, and many more.
June 12, 2019 · 2:30 p.m.
837 views
Making Our Future Better
Viktor Klang, Lightbend
June 12, 2019 · 2:31 p.m.
1682 views
Testing in the postapocalyptic future
Daniel Westheide, INNOQ
June 12, 2019 · 2:32 p.m.
498 views
Context Buddy: the tool that knows your code better than you
Krzysztof Romanowski, sphere.it conference
June 12, 2019 · 2:33 p.m.
393 views
The Shape(less) of Type Class Derivation in Scala 3
Miles Sabin, Underscore Consulting
June 12, 2019 · 3:30 p.m.
2321 views
Refactor all the things!
Daniela Sfregola, organizer of the London Scala User Group meetup
June 12, 2019 · 3:31 p.m.
514 views
Integrating Developer Experiences - Build Server Protocol
Justin Kaeser, IntelliJ Scala
June 12, 2019 · 3:32 p.m.
551 views
Managing an Akka Cluster on Kubernetes
Markus Jura, MOIA
June 12, 2019 · 3:33 p.m.
735 views
Serverless Scala - Functions as SuperDuperMicroServices
Josh Suereth, Donna Malayeri & James Ward, Author of Scala In Depth; Google ; Google
June 12, 2019 · 4:45 p.m.
936 views
How are we going to migrate to Scala 3.0, aka Dotty?
Lukas Rytz, Lightbend
June 12, 2019 · 4:46 p.m.
709 views
Concurrent programming in 2019: Akka, Monix or ZIO?
Adam Warski, co-founders of SoftwareMill
June 12, 2019 · 4:47 p.m.
1974 views
ScalaJS and Typescript: an unlikely romance
Jeremy Hughes, Lightbend
June 12, 2019 · 4:48 p.m.
1377 views
Pure Functional Database Programming‚ without JDBC
Rob Norris
June 12, 2019 · 5:45 p.m.
6374 views
Why you need to be reviewing open source code
Gris Cuevas Zambrano & Holden Karau, Google Cloud;
June 12, 2019 · 5:46 p.m.
484 views
Develop seamless web services with Mu
Oli Makhasoeva, 47 Degrees
June 12, 2019 · 5:47 p.m.
785 views
Implementing the Scala 2.13 collections
Stefan Zeiger, Lightbend
June 12, 2019 · 5:48 p.m.
810 views
Introduction to day 2
June 13, 2019 · 9:10 a.m.
250 views
Sustaining open source digital infrastructure
Bogdan Vasilescu, Assistant Professor at Carnegie Mellon University's School of Computer Science, USA
June 13, 2019 · 9:16 a.m.
374 views
Building a Better Scala Community
Kelley Robinson, Developer Evangelist at Twilio
June 13, 2019 · 10:15 a.m.
245 views
Run Scala Faster with GraalVM on any Platform
Vojin Jovanovic, Oracle
June 13, 2019 · 10:16 a.m.
1340 views
ScalaClean - full program static analysis at scale
Rory Graves
June 13, 2019 · 10:17 a.m.
463 views
Flare & Lantern: Accelerators for Spark and Deep Learning
Tiark Rompf, Assistant Professor at Purdue University
June 13, 2019 · 10:18 a.m.
380 views
Metaprogramming in Dotty
Nicolas Stucki, Ph.D. student at LAMP
June 13, 2019 · 11:15 a.m.
1250 views
Fast, Simple Concurrency with Scala Native
Richard Whaling, data engineer based in Chicago
June 13, 2019 · 11:16 a.m.
624 views
Pick your number type with Spire
Denis Rosset, postdoctoral researcher at Perimeter Institute
June 13, 2019 · 11:17 a.m.
245 views
Scala.js and WebAssembly, a tale of the dangers of the sea
Sébastien Doeraene, Executive director of the Scala Center
June 13, 2019 · 11:18 a.m.
661 views
Performance tuning Twitter services with Graal and ML
Chris Thalinger, Twitter
June 13, 2019 · 12:15 p.m.
2003 views
Supporting the Scala Ecosystem: Stories from the Line
Justin Pihony, Lightbend
June 13, 2019 · 12:16 p.m.
163 views
Compiling to preserve our privacy
Manohar Jonnalagedda and Jakob Odersky, Inpher
June 13, 2019 · 12:17 p.m.
301 views
Building Scala with Bazel
Natan Silnitsky, wix.com
June 13, 2019 · 12:18 p.m.
565 views
Diversity and Inclusion Panel: Bring the Thunder!
Panel
June 13, 2019 · 2:30 p.m.
244 views
Asynchronous streams in direct style with and without macros
Philipp Haller, KTH Royal Institute of Technology in Stockholm
June 13, 2019 · 3:45 p.m.
304 views
Interactive Computing with Jupyter and Almond
Sören Brunk, USU Software AG
June 13, 2019 · 3:46 p.m.
681 views
Scala best practices I wish someone'd told me about
Nicolas Rinaudo, CTO of Besedo
June 13, 2019 · 3:47 p.m.
2702 views
High performance Privacy By Design using Matryoshka & Spark
Wiem Zine El Abidine and Olivier Girardot, Scala Backend Developer at MOIA / co-founder of Lateral Thoughts
June 13, 2019 · 3:48 p.m.
753 views
Immutable Sequential Maps – Keeping order while hashed
Odd Möller
June 13, 2019 · 4:45 p.m.
276 views
All the fancy things flexible dependency management can do
Alexandre Archambault, engineer at the Scala Center
June 13, 2019 · 4:46 p.m.
389 views
ScalaWebTest - integration testing made easy
Dani Rey, Unic AG
June 13, 2019 · 4:47 p.m.
468 views
Mellite: An Integrated Development Environment for Sound
Hanns Holger Rutz, Institute of Electronic Music and Acoustics (IEM), Graz
June 13, 2019 · 4:48 p.m.
213 views
Closing panel
Panel
June 13, 2019 · 5:54 p.m.
400 views