Manage library breaking changes with Scalafix

When you’re a library maintainer, you’re going to ship improvements and some of them may involve breaking changes. Once it’s released and you’re starting to have users, how do you handle them? Is it the responsibility of your users? What could you do better?

In this article, let’s explore one way to handle breaking changes in Scala, with Scalafix.

Introducing Scalafix

Scalafix is an open source library created and maintained by the Scala Center. It has 3 main motivations:

• Code refactor: based on existing or custom rules, automate the migration from a legacy codebase.
• Linter: identify and report code patterns that may cause production incidents or do not match your coding style.
• Enforce the CI: ensure in an automated way that the code quality is matching your criteria at each code review.

Here, we will focus on the refactor point.

Motivations

As a library maintainer, your main responsibility is to provide the features expected by your users. When introducing breaking changes, you should guarantee the library stability. That’s why there are many reasons to use Scalafix to that purpose:

• Codebase: since you’re comfortable with the library codebase, there are no better people than you to fix it.
• Automate: it will not require any particular actions for your users other than applying the rule(s) you have defined for them.
• Trust: Knowing that you took the time and energy to make their upgrade path smoother will make your users trust you and your library more.

Scalafix rules

A rule automatically refactor an original code by choosing the updates to apply. The following chart gives an overview of the different types of rules in Scalafix today.

Scalafix comes bundled with some builtin rules (ExplicitResultTypes, LeakingImplicitClassVal), but you can also write your own (either syntactic or semantic).

Syntactic rules don’t require compilation, as they won’t interfere with the types. For example, the builtin rule ProcedureSyntax refactor a deprecated Scala feature allowing to leave out the result type and the assignment character = .

Semantic rules can do more advanced analysis on the syntax tree but requires compilation. The RemovedUnused rule from Scalafix remove unused imports and terms reported by the compiler under -Ywarn-unused.

Syntactic rule

DisableSyntax is a rule provided by Scalafix that reports errors when a “disallowed” syntax is used. For example, you can prevent the usage of mutable variables through the keyword var in Scala.

var a : Int = 42

Running Scalafix, you will obtain the following error message:

error: [DisableSyntax.var] mutable state should be avoided
[error]   var a: Int = 10
[error]   ^^^

Since this rule is only managing the different tokens you’re using in the code, it doesn’t require any compilation information to run.

Semantic rule

A classic pattern in a large codebase is to find some unused code such as imports or variables. Scalafix provides the rule RemoveUnused to remove them. Here the compilation is required because to know a variable is not used, the abstract syntax tree of your code should be available to apply analysis and identify the dead code. The below example shows the changes after applying the rule.

// before
import scala.List
import scala.collection.{immutable, mutable}
object Foo { immutable.Seq.empty[Int] }

// after
import scala.collection.immutable
object Foo { immutable.Seq.empty[Int] }

Using the built-in rules

Let’s set up a multi-project with sbt to see how to use a rule provided by Scalafix.

lazy val root =
  project
    .in(file("."))
    .aggregate(buildin_rule)
    .settings(
      name := "scalafix-onboarding"
    )
    .settings(commonSettings)

lazy val commonSettings =
  Seq(
    scalaVersion := "2.13.8",
    semanticdbEnabled := true,
    semanticdbVersion := scalafixSemanticdb.revision,
    organization := "com.contentsquare"
  )

lazy val buildin_rule = project.settings(commonSettings)

ThisBuild / scalafixScalaBinaryVersion := CrossVersion.binaryScalaVersion(
  scalaVersion.value
)

The next step is to create a Scalafix configuration file .scalafix-conf at the root level of your project with all the rules to apply. In this example, we’re going to use the ExplicitResultTypes that will insert types annotations for the public members.

$ cat .scalafix.conf
rules = [DisableSyntax, ExplicitResultTypes]

Now let’s apply the rule on some existing code and see its effect.

$ sbt scalafix
[info] compiling 1 Scala source
[info] compiling 1 Scala source
[info] Running scalafix on 1 Scala sources
[info] Running scalafix on 1 Scala sources

// Before
def compute(x: Int) = x * 10
val v = compute(10)

// After
def compute(x: Int): Int = x * 10
val v: Int = compute(10)

Here, you can see that all the types have been inserted after applying the rules making the code more explicit for the readers. There are other rules provided by Scalafix if you want to play with them.

Define a custom rule

Setup the tests

If the existing rules do not match your needs, you can define your own rules. The Scalafix documentation recommends writing unit tests that will check their correctness.

Let’s take a simple example with a feature in your library that renamed some packages and classes. For example, all the occurrences of DBLookUp in your code are now named RelationalDatabaseLookUp and the package fix has been renamed to database.

// Before
package fix

trait DBLookup {}
object DBLookup {}
object A {
  def f: DBLookup = ???
}

// After
package database

trait RelationalDatabaseLookup {}
object RelationalDatabaseLookup {}
object A {
  def f: RelationalDatabaseLookup = ???
}

If you don’t provide any Scalafix, the users have to manually fix all the breaking changes introduced in the new release. With Scalafix, you can avoid this situation. We’re going to add the custom rules as separate projects into our sbt multi-project in order to publish this rule later.

lazy val `scalafix-input` = (project in file("scalafix/input"))
  .disablePlugins(ScalafixPlugin)
lazy val `scalafix-output` = (project in file("scalafix/output"))
  .disablePlugins(ScalafixPlugin)
lazy val `scalafix-rules` = (project in file("scalafix/rules"))
  .disablePlugins(ScalafixPlugin)
  .settings(
    libraryDependencies += "ch.epfl.scala" %% "scalafix-core" % "0.10.0"
  )

lazy val `scalafix-tests` = (project in file("scalafix/tests"))
  .settings(
    scalafixTestkitOutputSourceDirectories :=
      (`scalafix-output` / Compile / sourceDirectories).value,
    scalafixTestkitInputSourceDirectories :=
      (`scalafix-input` / Compile / sourceDirectories).value,
    scalafixTestkitInputClasspath :=
      (`scalafix-input` / Compile / fullClasspath).value,
    scalafixTestkitInputScalacOptions :=
      (`scalafix-input` / Compile / scalacOptions).value,
    scalafixTestkitInputScalaVersion :=
      (`scalafix-input` / Compile / scalaVersion).value
  )
  .dependsOn(`scalafix-input`, `scalafix-rules`)
  .enablePlugins(ScalafixTestkitPlugin)

You have to create 4 folders:

1. scalafix/input contains the input code to change.
2. scalafix/output contains the expected code after applying the rule (be careful, it’s case sensitive).
3. scalafix/rules contains the custom rule implementation.
4. scalafix/tests contains the test suite to verify the correctness of your rule. There is a test kit provided by Scalafix to assess if your rule has been correctly applied on the input file.

Now that our testing plan is ready, let’s implement our custom rule.

Implement the rule

The following snippet of code implements the changes we have decided for our rule.

package fix

import scalafix.v1._
import scala.meta._

class RewriteNames extends SemanticRule("RewriteNames") {

  override def fix(implicit doc: SemanticDocument): _root_.scalafix.v1.Patch = {
    Patch.replaceSymbols(
      "fix" -> "database"
    ) + replaceTerms("DBLookup", "RelationalDatabaseLookup")
  }

  def replaceTerms(oldValue: String, newValue: String)(implicit
      doc: SemanticDocument
  ): Patch = {
    doc.tree.collect {
      case t: Term if t.toString() == oldValue =>
        Patch.renameSymbol(t.symbol, newValue)
    }.asPatch
  }
}

Let’s focus on the keys elements in this implementation:

1. Patch.replaceSymbols is in charge of renaming the package fix to database.
2. doc.tree.collect performs a top-to-bottom traversal of the syntax tree.
3. Path.renameSymbol is in charge of changing all the occurrences of DBLookup to RelationDatabaseLookup.
4. The 3 previous functions are provided by Scalafix.

If you run the test, it will pass; ensuring that the rule is applied as expected.

$ sbt scalafix-tests/test
[info] RuleSuite:
[info] - fix/RewriteTest.scala
[info] Run completed in 731 milliseconds.
[info] Total number of tests run: 1
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.

Publish externally

Once you have implemented your rule, you can publish it to a public or private repository to make it available for your users. If you have published your rule into one of the public repositories supported by Scalafix, your users can apply the rule using the following sbt command.

$ scalafix dependency:RULE@GROUP::ARTIFACT:VERSION

Let’s explain the semantic of each part in the Scalafix command:

• RULE: the name of the rule to apply
• GROUP::ARTIFACT:VERSION: your library version that contains the rule.

If you want to download the rule once and make it always available, you can also add the following lines in your build.sbt and use it by only calling the rule name.

ThisBuild / scalafixDependencies += "GROUP" %% "ARTIFACT" % "VERSION"

Then:

$ scalafix RULE

For the private repositories, you have to add your resolver(s) using the sbt task scalafixResolvers.

ThisBuild / scalafixResolvers ++= Seq(
  coursierapi.MavenRepository.of("your_private_repository")
)

Plug-in with Scala Steward

If you’re using Scala Steward, you can set the option --scalafix-migrations scalafix-migrations.conf to fetch and apply all the rules defined in the provided file in addition to bumping the library version.

migrations = [
    {
        groupId : "your_group_id",
        artifactIds: ["artifact_id1", "artifact_id2"],
        newVersion: "v3.0.7", // the version that contains the breaking change
        rewriteRules: ["dependency:RULE@GROUP::ARTIFACT:VERSION"] // the dependency handling the breaking changes by
    }
]

Conclusion

Here at Contentsquare, we discovered Scalafix when we saw it was used by famous Scala projects like Akka or Scalatest and decided to give it a try. We now consider it an invaluable tool to both put constraints on our coding style but also to provide smooth and safe migrations when introducing breaking changes.

We hope that this article will make you try it and apply it on your own codebases.

Thanks to Tim Carry and Yohann Jardin for reviewing drafts of this.