When it comes to writing good computer programs, avoiding mistakes is key. To prevent errors, several processes are generally put in place, such as thorough review procedures and the use of static analysis tools. However, despite these measures, human error remains a stubborn obstacle in software development. Issues frequently slip through the review process, and tools like Checkstyle, SonarCloud, and SpotBugs only highlight some common problems. This leaves the developer with the task of manually applying the suggested improvements.

Static analysis tools are beneficial as they help detect bugs before they enter our codebase. They save time for those reviewing pull requests, as these issues don’t need to be pointed out during a review. Even better, it saves the frustration of solving or pointing out the same issues over and over again. It gets even more frustrating if developers are pointing out trivial things in pull request comments as they lead to repeated useless discussions.

Observing the boost that such tools can provide, we began to ask ourselves: How can we achieve a similar level of automation for bug patterns that we frequently encounter, which are not covered by existing tools? And, can we transition from tools that merely identify problems to tools that actually fix them?

Let’s explore Error Prone!

Error Prone

Error Prone is a static analysis tool for Java that catches common Java mistakes and flags them as compile-time errors. Originally developed by Google and open-sourced in 2012, Error Prone integrates with the Java compiler. As a compiler plugin, it is not tied to a specific build tool, meaning it works seamlessly with Maven, Gradle, Bazel, and other build systems. It performs its error checking after the compiler’s flow analysis phase, ensuring that type checking is complete and all symbol information is available. By this stage, all errors detected by the standard compiler have been reported.

When Error Prone detects an issue, it is presented as a compiler warning or error. These diagnostics include a helpful message, a link to documentation, and suggested code to fix the issue. Additionally, Error Prone can generate patch files containing suggested fixes, which can optionally be applied directly to the code. This feature significantly simplifies the process of applying large-scale changes to a codebase.

Examples

Consider a simple example where valid Java code doesn’t quite achieve the desired outcome. It is easy to forget the ‘throw’ keyword in front of an exception initialization expression. The following code compiles, but no exception will be thrown when executed:

When compiling this code with Error Prone configured, you’ll encounter the following error:

"">[ERROR] Example.java:[3,35][3,25] [DeadException] Exception created but not thrown
[ERROR]     (see https://errorprone.info/bugpattern/DeadException)
[ERROR]   Did you mean 'throw new IllegalArgumentException();'?

The developer is not only presented a clear error message, but also provided a fully contextualised suggested fix. This is where Error Prone distinguishes itself as a static analysis tool.

Out of the box, Error Prone comes with an extensive set of over 500 checks, known as BugChecks. Examples include:

• ArrayEquals: Flags comparing arrays by reference equality.
• FormatStringAnnotation: Flags invalid format string usages.
• Immutable: Flags type declarations annotated with @Immutable that appear mutable.
• JUnitParameterMethodNotFound: Flags parameterized tests that reference a nonexistent provider method.

Developers can configure the checks as desired. For example, each check can be enabled or disabled, and warnings can be raised as errors or treated as suggestions.

In addition to the provided checks, Error Prone also allows you to create custom checks! This extensibility is what makes the tool even more attractive. As evidenced by the last example, checks can apply to any Java library. Error Prone built-in checks cover more than just JDK APIs, with checks for popular libraries like JUnit and Mockito. However, writing BugChecks means traversing and manipulating the Abstract Syntax Tree (AST) of the Java code, which isn’t entirely trivial. Meaning, the developer needs to have some knowledge of Error Prone’s APIs, and occasionally work with scarcely documented compiler APIs.

Internal use of Error Prone

At Picnic, we use custom rules to catch common mistakes or incorrect usages of APIs. From the code example below, we flag an unnecessary application of String.valueOf to "foo".

"">String str = String.valueOf("foo");

This results in the following compiler warning:

"">> [WARNING] Example.java:[18,31] [IdentityConversion] This method invocation appears redundant
> [WARNING] (see https://error-prone.picnic.tech/bugpatterns/IdentityConversion)
> [WARNING] Did you mean 'String str = "foo";'?

As another example, for incorrect SLF4J log statements, we identify cases where the number of format string placeholders does not match the number of supplied arguments.

"">LOG.info("format-string-with-{}-{}-placeholders", "value1");

This result in the following compiler warning:

"">> [WARNING] Example.java:[10,12] [Slf4jLogStatement] Log statement contains 2 placeholder(s), but specifies 1 matching argument(s).
> [WARNING] (see https://error-prone.picnic.tech/bugpatterns/Slf4jLogStatement)

Refaster

Writing a check in Error Prone is not trivial. That’s why Google created Refaster, which comes bundled with Error Prone. Refaster is a DSL that lets you define rewrite rules in Java code using before- and after-templates. First, these Java files are compiled into .refaster files. Then, the Error Prone compiler scans the codebase for code that matches the structure of a before-template. The matching code is then rewritten according to the associated after-template.

The code snippet above is an example of a Refaster rule. A rule consists of at least one before-template and an after-template, indicated by the @BeforeTemplate and @AfterTemplate annotations. Note how in the example each template method has a parameter String str. Refaster treats such parameters as typed placeholders. When traversing the AST to find expressions that match the structure of any before-template, placeholders will match any subexpression of a compatible type. For example, the arbitrary piece of code String.valueOf(getInteger()).length() == 0 will also be matched by the shown Refaster rule. When a match is found, Refaster automatically refactors the code as described in the after-template. After applying the shown Refaster rule to a code base, the isEmpty() method will be the only operator used to check whether a string is empty.

Google open-sourced the Refaster tool in 2014. While they haven’t open-sourced any of their large internal body of Refaster rules, there is a GitHub issue created in 2017, where they mention plans to open source (parts of) their Refaster rules. It’s unlikely this will happen in the near future. Meanwhile, developers can still use Refaster’s framework to create custom rules tailored to their codebases. At Picnic, we’ve created over 900 Refaster rules that are continuously applied to our codebase.

Refaster also supports advanced features. You can use generics to have Refaster rules match many (sub-)types. Conversely, you can define restrictions using predicates on what code should be matched. Refaster supports matching block statements in addition to individual expressions. Moreover, you can match arbitrary chunks of code by using a placeholder. The latter is especially useful for matching lambda expressions with a specific signature.

One downside of Refaster is that it can only patch expressions and statements, meaning it is limited to making changes in method bodies and (static) initializer blocks. This contrasts with the capabilities of Error Prone’s BugChecks, which have full control over the AST and can perform larger refactoring operations.

Custom BugChecks

BugChecks are more powerful than Refaster rules, but they’re also more challenging to write.

Below a simple Java class is defined. Its abstract (and vastly simplified) representation in tree form, as seen in the figure below that, consisting of various nodes. Each node of the tree contains information about the code construct it represents. For instance, a method node specifies a method’s name, return type, arguments, body, type arguments, and modifiers. Error Prone handles traversing the AST, allowing developers to define logic in the BugCheck to analyse, swap, edit, or remove a full node from the tree.

Imagine we want to remove empty methods from our codebase. Error Prone would analyse the classes and check each method tree node to see if it contains a body with statements. In the example, method2 doesn’t have a body, so that node can be dropped from the tree. This is a simple example, but Error Prone also supports more complex rewrites that make multiple changes throughout a file.

Error Prone performs analysis at the CompilationUnit level, meaning it analyses each source file individually. It can traverse the structure of the file and classes, like parent classes and interfaces, but it can’t analyse multiple complete files at once. This means we cannot check whether method2() is actually used in other CompilationUnits.

Extensibility of Error Prone

As previously discussed, we can define custom BugChecks with Error Prone. When they are available on the annotation processor path, the plugin mechanism will dynamically load the plugins using Java’s service loader mechanism.

There are some organisations like Mockito, Palantir, Chromium, and Uber that offer Error Prone extensions in the form of BugChecks. For example, Uber created a tool called NullAway to help eliminate NullPointerExceptions. Using annotations, NullAway performs a fast type-based analysis to catch possible NPEs at compile-time.

Leveraging Custom Rules

Both Error Prone and Refaster offer the flexibility to create custom rules, proving invaluable in addressing various coding challenges. Whether it is updating deprecated APIs, handling common issues, refining code style, transitioning between libraries, or reducing technical debt, these rules can streamline the development process, saving time and manual effort.

At Picnic, we are avid users of Error Prone and Refaster. Over the years, we’ve created many additional rules to streamline our codebase. These rules are constantly applied across our 70+ Java repositories. They run as part of the build by default. This setup minimises unproductive discussions about coding standards, bugs, or code style, enabling more focus on delivering value. Additionally, we allow developers to make contributions to our shared repository containing the refactoring rules, ensuring uniformity and ever-increasing code quality across all Java projects. As this consistency extends across teams, it has become easier for developers to switch between codebases and collaborate effectively.

Next up

At Picnic, we’ve open-sourced our repository, Error Prone Support, where we maintain our Refaster rules, BugChecks, and modules that extend and improve the functionality of Error Prone itself. Next time, we’ll discuss Error Prone Support, the main challenges we faced during its development, and rolling it out at Picnic, where it is now a core technology used by all our Java repositories. This brings some additional challenges, as introducing new rules can have a large impact, the new rules need to work for all codebases, and all developers need to agree with the changes. Stay tuned for the next installment, where you will learn how we created, and continue to manage, such a plugin in the Java ecosystem!

This article is written together with Stephan Schroevers