I recently wanted to update our JOSM package, the OSM editor written in Java. The newer version had slightly different dependencies than the previous one, and required me to update the build system. I wanted to write a little something to explain what I did and maybe give you a bit more understanding of some of the details.

The Problem

Currently, the default Java compiler is Java 8 (icedtea version 3 as it's called in Guix). Since we try to bootstrap our packages, meaning we do not want to rely on any binary version outside what we can build ourselves, Java is a bit of a challenge.

Most packages build with Maven, but Maven requires quite a lot of dependencies. Although we have a maven-build-system, which can use Maven to build packages, it's still quite hard to use as it misses many commonly-used plugins. It can currently only use the essential plugins (maven-compiler-plugin and friends).

Maven itself has a lot of dependencies that would normally be built with Maven, but as long as we don't reach Maven in the dependency graph, it's not possible to build use it. So, how do we build Maven, if we can't use it to build its dependencies?

One of the older build systems it Ant, which is easy to build and doesn't have dependencies. So, the first build-system we created for the Java ecosystem is the ant-build-system which can build any package using Ant. It was extended to generate a simple build.xml for any project, with just a few arguments. Then it runs the standard Ant phases to build the package.

Of course this is a very rudimentary build script which doesn't work for complex packages, but it works sufficiently well for simple packages, which represents the majority of packages we currently have.

Supporting Java Modules

As I tried to update JOSM, I found that it required two new packages that I needed to add to Guix: java-jakarta-json and java-parsson. These two packages require some maven plugins we do not have yet and I didn't want to add them too, as that would be too much work just to get a simple update.

So, I used the ant-build-system which generated a build.xml for me. Unfortunately, the build failed very quickly for the first package, as it threw an error when encountering the unknown module keyword:

    [javac] /tmp/guix-build-java-jakarta-json-2.1.3.drv-0/source/api/src/main/java/module-info.java:20: error: class, interface, or enum expected
    [javac] module jakarta.json {
    [javac] ^
    [javac] /tmp/guix-build-java-jakarta-json-2.1.3.drv-0/source/api/src/main/java/module-info.java:22: error: class, interface, or enum expected
    [javac]     exports jakarta.json;
    [javac]     ^
    [javac] /tmp/guix-build-java-jakarta-json-2.1.3.drv-0/source/api/src/main/java/module-info.java:23: error: class, interface, or enum expected
    [javac]     exports jakarta.json.spi;
    [javac]     ^
    [javac] /tmp/guix-build-java-jakarta-json-2.1.3.drv-0/source/api/src/main/java/module-info.java:24: error: class, interface, or enum expected
    [javac]     exports jakarta.json.stream;
    [javac]     ^
    [javac] /tmp/guix-build-java-jakarta-json-2.1.3.drv-0/source/api/src/main/java/module-info.java:25: error: class, interface, or enum expected
    [javac]     uses jakarta.json.spi.JsonProvider;
    [javac]     ^
    [javac] /tmp/guix-build-java-jakarta-json-2.1.3.drv-0/source/api/src/main/java/module-info.java:26: error: class, interface, or enum expected
    [javac] }
    [javac] ^
    [javac] 6 errors

Scary errors!

Since Java 9, modules were introduced as a way to encapsulate multiple classes and resources. It is possible to declare a module, make it depend on other modules. It also allows for hiding internal classes and exporting only classes that should be accessible outside the module.

As I am not a Java developer, and I only care about building packages, the only thing I need to know is that, whenever I see a module-info.java in the source code, it requires Java 9 or later because it's using modules.

To support this, I changed the JDK used for building the package, which is easy to do in the arguments field of the package declaration:

(arguments
 `(#:jdk ,openjdk9; or later versions
   ...)))

Although it was possible to build the first one like that, the second package's module declared a dependency on the first package, and the build failed with the following error:

    [javac] /tmp/guix-build-java-parsson-1.1.5.drv-0/source/impl/src/main/java/module-info.java:21: error: module not found: jakarta.json
    [javac]     requires transitive jakarta.json;
    [javac]                                ^

Turns out, Java doesn't look for modules in the same path as the CLASSPATH. Instead, one needs to explicitly give it a modules path. One can do that with ant:

<javac ...>
  <modulepath refid="some.path.variable" />
</javac>

After modifying the ant-build-system's generate-build.xml phase to add the modulepath in addition to the classpath, I had a few issues.

First, icedtea@3 relies on one of the phases to remove jar non-determinism, meaning that changing the ant-build-system requires rebuilding the java compiler. This significantly slowed down my ability to work on the build system.

Second, I found that the classpath was set incorrectly. The intent was to have it reflect the CLASSPATH environment variable, but it did not work correctly. I replaced:

<path id="classpath">
  <pathelement location="${env.CLASSPATH}" />
</path>

with:

<path id="classpath">
  <pathelement path="${env.CLASSPATH}" />
</path>

Then, I could use classpath as the modulepath.

Third, since we still use Java 8 by default, all builds that use Java 8 (that is, the vast majority of Java packages) initially failed since Java 8 doesn't support the modulepath option. Making it optional via a build-system argument solved the issue. To build with modules and the ant-build-system, you need to do:

(arguments
 `(#:jdk ,openjdk9; or later
   #:use-java-modules? #t ; to use modules
   #:jar-name "..." ; to generate a build.xml
   ...))

Future Work

This small work was quite hard because I had to rebuild the JDK each time I tried something new. It might sound simple now that I figured what needed to be done, but it still took me a few days to figure out.

Anyway, I think this is a great first step towards upgrading our default Java compiler to something that is still supported. I suspect our bootstrap packages being very old will probably not all build with a newer JDK, and since Java 9 introduced new features (such as modules), it's probably the most reasonable next step: Java 9 by default.

Of course, there's still more work to be done to reach the long-term goals:

• Java 11 by default
• Keeping Java packages up-to-date
• Using the Maven build system in more than one package
• Bootstrapping more of the Java/Android ecosystem
• Add your own goals here :)

And that's it for today!