sbt-big: faster and more stable sbt
This project is a proposal to create a new module in sbt/sbt called sbt-big
that is a fat jar with all sbt dependencies shaded under an internal namespace. By updating the sbt launcher to use org.scala-sbt:sbt-big
instead of org.scala-sbt:sbt
, we gain
- faster startup time, by making the launcher download 1 fat jar containing all dependencies instead of resolving and downloading 71 jars for dependencies. This speedup should be particularly noticable in clean CI and docker environments and from locations with high latency such as Asia or Australia.
- more stable binary compatibility, by shading sbt dependencies under the
sbt.internal.*
package so that sbt plugins don't conflict with the sbt boot classpath. See sbt/zinc#546 for an example issue. By usingsbt-big
, the sbt project is safe to upgrade to the latest versions of its dependencies without breaking the sbt plugin ecosystem.
To try out sbt-big
locally,
git clone https://github.com/olafurpg/sbt-big.git
cd sbt-big
sbt publishM2
What happens during publish is that:
- sbt-assembly includes the full dependency tree of sbt
$ du -h $HOME/.m2/repository/org/scala-sbt/sbt-big/1.1.0/sbt-big-1.1.0.jar
21M /Users/ollie/.m2/repository/org/scala-sbt/sbt-big/1.1.0/sbt-big-1.1.0.jar
- the published artifact contains 0 dependencies
$ cat $HOME/.m2/repository/org/scala-sbt/sbt-big/1.1.0/sbt-big-1.1.0.pom/sbt-big-1.1.0.pom | grep dependencies
<dependencies>
<!--the dependency that was here has been absorbed via sbt-assembly-->
<!--the dependency that was here has been absorbed via sbt-assembly-->
</dependencies>
- the fastparse dependency has been shaded into the
sbt.internal.fastparse
package
$ jar tf $HOME/.m2/repository/org/scala-sbt/sbt-big/1.1.0/sbt-big-1.1.0.jar | grep fastparse | head
sbt/internal/fastparse/
sbt/internal/fastparse/core/
sbt/internal/fastparse/parsers/
sbt/internal/fastparse/utils/
...