Building Native Covid19 Tracker CLI using Java, PicoCLI & GraalVM

When it comes to building CLI apps, Java is not your first ( not even among the top 3) choice that comes to mind. However, one of the amazing things about java is its ecosystem and vibrant community, which means that you can find some tools/libraries for (nearly) everything.

Golang particularly excels in this area for several reasons, but one aspect where Go sparkles is the ability to compile a program to a single, small native executable file, that makes the program runs faster and much easier to distribute. However Java Apps have been traditionally hard to distribute since they require the JVM to be already installed on the target machine.

In this post, I describe my experience building a small CLI app to track covid19, using picocli and turning it into a lightweight, standalone binary that is easy to use and distribute, using Graal VM.

The complete source code for this application can be found in this Github repo.

PicoCLI

Picocli is a modern library for building command line applications on the JVM.
Picocli aims to be the easiest way to create rich command line applications that can run on and off the JVM. It offers colored output, TAB autocompletion, nested subcommands, and comes with couple of grear features compared to other JVM CLI libraries such as negatable options, repeating composite argument groups, repeating subcommands and custom parameter processing.

Picocli based applications can also easily be integrate with Dependency Injection containers. Picocli ships with a picocli-spring-boot-starter module that includes a PicocliSpringFactory and Spring Boot auto-configuration to use Spring dependency injection in your picocli command line application.

The Micronaut microservices framework has built-in support for picocli.

Covid-19 Tracker app

Covid-19 Data

The CLI app gets data from Novel COVID API. A free and easy to use API, that gathers data from multiple sources (Johns Hopkins University, the New York Times, Worldometers, and Apple reports)

Dependencies

There are a couple of libraries that I used to build this app. First and foremost, picocli as the heart of the CLI app. I opted for Jersey Client to handle HTTP communication with Rest server and collect data, as well as Jackson the well know java json library.

The hard part was finding some Ascii based tables and graphs, and honestly my choices were very limited. I ended up using ascii-table to create and customize ASCII tables and ascii-data t to generate some nice looking text-based line-graphs.

This what my pom-file dependencies section contians:

...
    <dependency>
        <groupId>info.picocli</groupId>
        <artifactId>picocli</artifactId>
        <version>4.2.0</version>
    </dependency>
    <dependency>
        <groupId>org.glassfish.jersey.core</groupId>
        <artifactId>jersey-client</artifactId>
        <version>2.30.1</version>
    </dependency>
    <dependency>
        <groupId>org.glassfish.jersey.media</groupId>
        <artifactId>jersey-media-json-jackson</artifactId>
        <version>2.30.1</version>
    </dependency>
    <dependency>
        <groupId>com.github.freva</groupId>
        <artifactId>ascii-table</artifactId>
        <version>1.1.0</version>
    </dependency>
    <dependency>
        <groupId>com.mitchtalmadge</groupId>
        <artifactId>ascii-data</artifactId>
        <version>1.4.0</version>
    </dependency>
    <dependency>
        <groupId>org.glassfish.jersey.inject</groupId>
        <artifactId>jersey-hk2</artifactId>
        <version>2.30.1</version>
    </dependency>

Show me the code

Now that I've everything the app need, let's have a look at the code. Below is the main class:

@Command(description = "Track covid-19 from your command line",
        name = "cov19", mixinStandardHelpOptions = true, version = "cov19 1.0")
public class Covid19Cli implements Callable<Integer> {
    @Option(names = {"-c", "--country"}, description = "Country to display data for", defaultValue = "all")
    String country;
    @Option(names = {"-g", "--graph"}, description = "show data as graph history of last 30 days")
    boolean graph;
    @Option(names = {"-a", "--all"}, description = "show data for all affected countries")
    boolean all;

    CovidAPI covidAPI = new CovidAPI();

    public static void main(String[] args) {
        int exitCode = new CommandLine(new Covid19Cli()).execute(args);
        System.exit(exitCode);
    }

    public Integer call() throws Exception {
        if (this.all && !this.country.equals("all")){
            System.out.println(Ansi.AUTO.string("@|bold,red, ****** Cannot combine global (`-a`) and country (`-c`) options ****** |@\n"));
            return 1;
        }

        this.colorise(this.country);
        if(this.graph){
            PrintUtils.printGrapgh(covidAPI.history(this.country));
            return 0;
        }
        if (this.all){
            PrintUtils.printCountryStatTable(covidAPI.allCountryStats());
            return 0;
        }
        if(this.country.equals("all")) {
            PrintUtils.printGlobalTable(Arrays.asList(covidAPI.globalStats()));
            return 0;
        }
        PrintUtils.printCountryStatTable(Arrays.asList(covidAPI.countryStats(this.country)));
        return 0;
    }

A couple of interesting things here:

• The @Command annotation from picocli enables us to define the general information about the command.
• mixinStandardHelpOptionsconfig option adds magically --help and --version flag to CLI.
• The class implements Callable<Integer>, as picocli needs a predictable way of executing command, parsing params and options and returning exit code.
• The execute method shows the usage help or version information if requested by the user
• Invalid user input will result in a helpful error message. If the user input was valid, the business logic, present in call method, is invoked.
• Finally, the execute method returns an exit status code that can be used to call System.exit if desired. By default, the execute method returns CommandLine.ExitCode.OK (0) on success, CommandLine.ExitCode.SOFTWARE (1) when an exception occurred in the Runnable, Callable or command method, and CommandLine.ExitCode.USAGE (2) for invalid input.
• The fields of the class are annotated with @option, to declare what options the application expects. Picocli initializes these fields based on the command line arguments which commonly start with - or --.
• Note that some options have one name and some have more
• Option can have default values using the defaultValue annotation attribute.

Building and testing the app

As any java app using maven, we run mvn clean package to compile our app and generate the jar file. I used the maven shade plugin to package the artifact in an uber-jar (including all its dependencies).
Now, we can verify that our CLI is working using:

$ java -jar covid-java-cli-1.0-SNAPSHOT.jar --help                                                                                                   

Usage: cov19 [-aghV] [-c=<country>]
Track covid-19 from your command line
  -a, --all                 show data for all affected countries
  -c, --country=<country>   Country to display data for
  -g, --graph               show data as graph history of last 30 days
  -h, --help                Show this help message and exit.
  -V, --version             Print version information and exit.

So far, the application is working but doesn't feel too much like an actual CLI. Ideally, we should aim for a more native experience and simply run ./mycly instead calling java -jar each time!
This is what will try to accomplish in the next section with GraalVM.

GraalVM, Building a native image

This was the hardest part while working on this app, for the simple reason that GraalVM native image compiler supports for reflection is partial and requires additional configuration.
This impact my application in 2 ways:

• Picocli uses reflection to discover classes and methods annotated with @Command, and fields, methods or method parameters annotated with @Option.
• Jersey client uses reflection as well

Picocli includes a picocli-codegen module, that contains an annotation processor to generate GraalVM configuration files at compile time rather than at runtime. So the first one was easy to fix by adding the below config to my pom.xml file:

             <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.8.1</version>
                <configuration>
                    <annotationProcessorPaths>
                        <path>
                            <groupId>info.picocli</groupId>
                            <artifactId>picocli-codegen</artifactId>
                            <version>4.2.0</version>
                        </path>
                    </annotationProcessorPaths>
                </configuration>
            </plugin>

It generate configuration files for reflection, resources and dynamic proxies.

target
├── classes
│   ├── META-INF
│   │   └── native-image
│   │       └── picocli-generated
│   │           ├── proxy-config.json
│   │           ├── reflect-config.json
│   │           └── resource-config.json

As for jersey, I had to do some testing and debugging to generate the reflection.json to make our application Graal-enabled! Below a snippet from it:

...
  {
    "name" : "org.glassfish.jersey.internal.config.ExternalPropertiesConfigurationFeature",
    "allDeclaredConstructors": true,
    "allPublicConstructors": true,
    "allDeclaredFields": true,
    "allPublicFields": true,
    "allDeclaredMethods": true,
    "allPublicMethods": true
  },
  {
    "name" : "org.glassfish.jersey.message.internal.MessageBodyFactory",
    "allDeclaredConstructors": true,
    "allPublicConstructors": true,
    "allDeclaredFields": true,
    "allPublicFields": true,
    "allDeclaredMethods": true,
    "allPublicMethods": true
  },
  {
    "name" : "com.fasterxml.jackson.module.jaxb.JaxbAnnotationIntrospector",
    "allDeclaredConstructors": true,
    "allPublicConstructors": true,
    "allDeclaredFields": true,
    "allPublicFields": true,
    "allDeclaredMethods": true,
    "allPublicMethods": true
  },

Now that our reflection config is in place, We are pretty done with our application. The next natural step is to compile our application ahead of time and generate the native binary.

First off, we need to install GraalVM native-image tool and call it manually. However, recent GraalVM releases added the possibility to build native images right out of maven without running the native-image tool as a separate step after building the uber-jar. In order for it to run, the plugin expectsJAVA_HOME to be set as Graal CV, it will not work otherwise.

          <plugin>
                <groupId>org.graalvm.nativeimage</groupId>
                <artifactId>native-image-maven-plugin</artifactId>
                <version>20.0.0</version>
                <configuration>
                    <mainClass>me.aboullaite.Covid19Cli</mainClass>
                    <imageName>cov19-cli</imageName>
                    <buildArgs>
                        --no-fallback
                        --report-unsupported-elements-at-runtime
                        --allow-incomplete-classpath
                        -H:ReflectionConfigurationFiles=classes/reflection.json
                        -H:+ReportExceptionStackTraces
                        -H:EnableURLProtocols=https
                    </buildArgs>
                    <skip>false</skip>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>native-image</goal>
                        </goals>
                        <phase>verify</phase>
                    </execution>
                </executions>
            </plugin>

Everything is ready. Now we can generate a native image by running mvn clean verify, which will trigger native image compilation. The process will take about a minute to complete.
At the end, we have a native executable under target/cov19-cli.

$ ./target/cov19-cli --help 
                                                                                                                               
Usage: cov19 [-aghV] [-c=<country>]
Track covid-19 from your command line
  -a, --all                 show data for all affected countries
  -c, --country=<country>   Country to display data for
  -g, --graph               show data as graph history of last 30 days
  -h, --help                Show this help message and exit.
  -V, --version             Print version information and exit.

Comparing startup time

I couldn't resist the thought of comparing the startup times of running the application on a normal JIT-based JVM to that of the native image. Below the results I got for on my machine:

$ gtime -p java -jar covid-java-cli-1.0-SNAPSHOT.jar --help                                                                                        
real 0.32
user 0.67
sys 0.09
                                                                          
$ gtime -p ./cov19-cli --help                                                                                                                    
real 0.01
user 0.00
sys 0.00

Finals words

Building Java-based native CLI tools is becoming possible nowadays with Picocli and GraalVM. Of course, there are several limitations in native-image compiler, mainly the refliction support. Neverthless, the combination of both tools to create CLI tools, without JVM overhead, looks promising.

Ressources:

• https://picocli.info/#_introduction
• https://medium.com/graalvm/simplifying-native-image-generation-with-maven-plugin-and-embeddable-configuration-d5b283b92f57
• https://www.infoq.com/articles/java-native-cli-graalvm-picocli/