Abstract

The NetBeans build harness performs a number of tasks when building a NetBeans platform application. This article describes some of the issues that can be encountered when building platform applications in other ways than the standard use case of creating and building it on a single PC in the NetBeans IDE. It also provides some tips for customising build behaviour.

1. Introduction

The NetBeans platform ships with a comprehensive build harness, containing build scripts and Ant tasks that make the building of a platform application very easy. Since it consists of ant scripts, it can be used to build the application from the IDE or the command line. Most of the time the build harness does its job so well that the platform application developer is only vaguely aware of its existence. However, there are instances where you need something just slightly different. And that is what this article is all about!

This article focuses on the build process of a NetBeans Platform application, which has a different structure from a plain Java application project or a web application project. At the time of writing this article, I was using NetBeans 6.7.1, but still had 6.5 and even some older versions installed.

2. Managing Multiple Platforms in the IDE

When you first install NetBeans, a folder is created for that version. For example, version 6.5 would (by default) have been installed to C:\Program Files\NetBeans 6.5. A few months later, version 6.7.1 is release and you install the second version with its own folder. It is now possible to build on top of either of the two platforms. On the Tools menu, you will find a NetBeans Platforms option, which opens up the NetBeans Platforms Manager window shown below.

Figure 1. NetBeans Platform Manager.

Here you can configure multiple platforms, allowing you to choose which platform to use when building a suite and/or modules. Note that that 6.7.1 platform is the default, and that it uses the harness supplied with the IDE. For the 6.5 platform, you could choose to rather use the one supplied with the platform, i.e. the 6.5 one.

Keep in mind that the harness uses Ant, and was running perfectly well with the version of Ant that was included with the release. However, years down the line, some Ant tasks might not exist anymore and hence a very old harness may not run entirely as expected with a much later version of the IDE.

3. The Project, the User Folder and the Build Harness

Firstly, a brief look at the physical structure of a NetBeans platform project. This is again something that the developer often does not need to know much about since the IDE manages all the details for you. However, when exploring the build harness, this is a good place to start.

3.1 The Project

Let’s start off with a new NetBeans Platform Application (a bare basics suite)…

Figure 2. A new Platform Application in project view.

… and now inspect the project in files view…

Figure 3. A new Platform Application in files view.

The build.xml file is the main project build file (called Build Script in the projects view). This file includes the build-impl.xml file in the nbproject folder. The developer is free to edit build.xml since all the automatically generated build details are kept in build-impl.xml.

The file build-impl.xml is responsible for handling a number of project-specific property files, as well as the inclusion of the entry point into the build harness, specifically suite.xml. More about that later. The properties are read from two files:

• nbproject/platform.properties: This file indicates which platform to use and which platform clusters and more specifically modules form a part of the suite.
• nbproject/private/platform-private.properties: This file specifies where the user folder, and more specifically build.properties, is located. Note that the files in the nbproject/private folder is by default not checked into source repositories. The idea behind the folder is maintaining data that is private, i.e. specific to a PC.

The build-impl.xml file (as well as build.xml) inherits all the targets from the build harness scripts, such as build, clean, debug, run and build-zip.

It is worthwhile briefly mentioning what is contained in the other files that are not directly involved right now:

• nbproject/branding folder: The splash screen and other branding related data is stored here.
• nbproject/private/private.properties: Information that is private to the project in your IDE, such as which files were open the last time the project was saved.
• nbproject/genfiles.properties: Used by NetBeans to maintain the integrity of generated files. Do not edit.
• nbproject/project.properties: Properties of the suite such as the application name.
• nbproject/project.xml: It contains the settings that the IDE requires to handle the project, indicating that it is a org.netbeans.modules.apisupport.project.suite project.

So far, we have covered the following structure (only showing the relevant files):

Figure 4. File structure showing the suite folder and the very first entry point into the build harness.

Before we continue on to the other folders, let’s add a new module called MOD_HarnessExample to the suite and see what happens.

Figure 5. The file view after adding a new module.

The new module is created by default inside the suite folder, as seen above. For the purpose of this example, that is perfect. Note that the module has its own build.xml file and nbproject folder. It contains a file we have not encountered yet – suite.properties. It specifies where (relative to the module) the suite that it belongs to is located. The module has no platform.properties files – it inherits those properties from its parent suite.

The build-impl.xml for a module imports the build.xml file from the harness.

Note that a module can also be created as a standalone module, in which case it has no suite.properties file and has its own platform.properties.

Since the module is part of the suite, the suite needs to know where to find this module. The relevant properties are set in the nbproject/project.properties file of the suite:

modules=\
${project.za.co.kitt.examples.buildharness}
project.za.co.kitt.examples.buildharness=MOD_HarnessExample

Now the build file in the suite is able to locate all the modules that it has to build, and is able to call the build.xml file in each one. Let’s take a look at the complete picture now:

Figure 6. The file structure, now including a module.

3.2 The User Folder

The user folder is where the NetBeans IDE saves its settings for a specific user on the computer. The location of the user folder is specified in etc/netbeans.conf in the NetBeans installation folder, together with other IDE settings. In 6.7.1, the property is by default:

# ${HOME} will be replaced by JVM user.home system property
netbeans_default_userdir="${HOME}/.netbeans/6.7"

On Windows XP, for example, this is effectively something like

C:\\Documents and Settings\\user\\.netbeans\\6.7

As mentioned earlier, the build.properties file is of special interest to us. The locations of all the platforms configured in the IDE are persisted in this file. For example, the default platform is specified as follows (in version 6.7.1):

nbplatform.default.harness.dir=${nbplatform.default.netbeans.dest.dir}/harness
nbplatform.default.netbeans.dest.dir=C:\\Program Files\\NetBeans 6.7.1

So when the platform.properties file specifies nbplatform.active=default, this refers to the platform as specified above. Remember that the harness folder can be specified independently of the platform folder, so do not assume for platforms other than the default that it will be inside the platform folder.

Let’s add this information to our ever growing diagram:

Figure 7. The file structure, showing the relationships of the build.properties file.

3.3 The Build Harness

The build harness folder contains, amongst other things, a number of scripts that are of interest to us. The scripts are shown in the completed diagram below:

Figure 8. Finally the entire structure.

4. You must define ‘nbplatform.default.harness.dir’

If you create platform applications in the NetBeans IDE, and only ever build them on a developer PC, you will probably never encounter the dreaded “You must define ‘nbplatform.default.harness.dir’” error. However, if you try to build the platform from the command line on a build server, especially if the server is running some flavour of Linux, you will sooner or later run into this issue. Since there are several aspects involved, and different solutions available, lets explore how the harness occurs.

This exact error message only occurs in NetBeans 6.5 and earlier. Since version 6.7 the check has been changed, and more recently the error message has been changed to be more descriptive.

Let’s start off by discussing where this check occurs and how the variable is normally defined on a developer PC. The check to see whether the variable is defined, is done in the generated build-impl.xml. If you inspect the suite-private.properties file, you will see that the user.properties.file property is defined there. And it points to the build.properties file containing the nbplatform.${nbplatform.active}.harness.dir property. If we look at the relevant parts of the file structure diagram, we can see the same path defined there:

Figure 9. File structure showing where the harness.dir property is defined.

By explicitly passing the relevant properties (nbplatform.${{nbplatform.active}}. netbeans.dest.dir and nbplatform.${{nbplatform.active}}.harness.dir) to ant, a build system can build a platform application without the need for a platform-private.properties file.

If your build server is running Linux, ensure that the folder where the platform and harness resides is accessible to the operating system user that executes the build process.

5. Suite chaining

From 6.7, it is now possible to chain suites. So you could have suite SUITE_A with module MOD_A, and include it in another suite as you would have with a platform module. See Figure 10 for an example.

Figure 10. Suite chaining.

The following screenshot of the suite’s properties shows the new features:

Figure 11. Suite properties showing suite chaining.

Each suite (project) or cluster that is added here is listed in the cluster.path property.

6. Disabling the Generation of Auto Update Information

I posted this information a while ago on the technews@kitt.co.za blog. However, it is worthwhile including it in this more comprehensive discussion of the build harness. Below is slightly extended version of the same facts.

NetBeans platform application developers will be very familiar with one of the steps in the build process: “Generating Information for Auto Update”. The reason is that it takes quite a while compared to most of the other steps in the process. After a reasonably extensive search of the Internet, I realised I will have to go look for this step myself. Here is the process that was followed:

• The first step was to search through the NetBeans platform source code (version 6.1, the only source version I had available) for the log message “Generating Information for Auto Update”. This message was found in the source file nbbuild\antsrc\org\netbeans\nbbuild\MakeListOfNBM.java. (The nbbuild source folder contains a number of ant targets that are used in the build process.)
• Armed with the knowledge that the ant target class that is responsible for this process is called MakeListOfNBM, a search was conducted through the source folder once more to find what the target is called. The ant task is called ‘genlist’.
• Now a search of the harness folder provided the last bit of information. The ‘genlist’ task is called by the ‘netbeans’ target in the harness/common.xml ant script. If you comment this out, the “Generating Information for Auto Update” step will not be performed.

This is very useful when developing new applications, since it really speeds up the build process significantly if you skip this step. However, it existed for a reason. And that reason is to manage automatic updates when using NBMs. So do be aware that if you skip the Auto Update Generation step, you will not be able to build NBMs.

Commenting the task out is perhaps too hard-coded. It would be better to execute it (or not) depending on a property that is set in, for example, the project.properties file.

7. About the Custom Java Compiler

First it is worthwhile having a look how the source code was built in 6.5. The relevant part of the compile target in common.xml is shown below:

<target name="compile" depends="init,up-to-date" unless="is.jar.uptodate">
<mkdir dir="${build.classes.dir}"/>
<depend srcdir="${src.dir}" destdir="${build.classes.dir}" cache="build/depcache">
<classpath refid="cp"/>
</depend>

<javac srcdir="${src.dir}" destdir="${build.classes.dir}" debug="${build.compiler.debug}" debuglevel="${build.compiler.debuglevel}" encoding="UTF-8" deprecation="${build.compiler.deprecation}" optimize="${build.compiler.optimize}" source="${javac.source}" target="${javac.target}" includeantruntime="false">

<classpath refid="cp"/>
<compilerarg line="${javac.compilerargs}"/>
</javac>

...

And how it is built now in 6.7.1:

<target name="compile" depends="init,up-to-date" unless="is.jar.uptodate">
<mkdir dir="${build.classes.dir}"/>
<depend srcdir="${src.dir}" destdir="${build.classes.dir}" cache="build/depcache">
<classpath refid="cp"/>
</depend>

<nb-javac srcdir="${src.dir}" destdir="${build.classes.dir}" debug="${build.compiler.debug}" debuglevel="${build.compiler.debuglevel}" encoding="UTF-8" deprecation="${build.compiler.deprecation}" optimize="${build.compiler.optimize}" source="${javac.source}" target="${javac.target}" includeantruntime="false">

<classpath refid="cp"/>
<compilerarg line="${javac.compilerargs}"/>
<processorpath refid="processor.cp"/>
</nb-javac>

In short, the javac ant task was replaced with something called nb-javac. This is defined in the -javac-init target (again showing only the relevant part):

<presetdef>
<custom-javac>
<bootclasspath>
<path path="${bootclasspath.mac}"/>
<path path="${nbjdk.bootclasspath}"/>
</bootclasspath>
<javacclasspath refid="javacimpl"/>
</custom-javac>
</presetdef>

The property nbjdk.bootclasspath points to the harness/antlib/openjdk-javac-6-b12.jar file, which contains the custom NetBeans compiler.

Why is does this compiler exist? The answer, as I discovered quite accidentally when integrating the AspectJ compiler, is simple. It exists to process the new annotations introduced in 6.7.1. The @ServiceProvider annotation, for example, is processed at build time by this custom compiler.

8. Integrating the AspectJ Compiler

Many articles can be written on AspectJ alone. However, here the focus is just on how to build module code with the AspectJ compiler instead of the normal Java one or the custom NetBeans compiler.

The first step is to put the AspectJ compiler in a folder close to the harness folder in your platform’s folder structure. In my folder structure, it is located at ${harness.dir}/../aspectj1.6. Then simply replace this part of the common.xml build script

<target name="compile" depends="init,up-to-date" unless="is.jar.uptodate">
<mkdir dir="${build.classes.dir}"/>

<depend srcdir="${src.dir}" destdir="${build.classes.dir}" cache="build/depcache">
<classpath refid="cp"/>
</depend>

<nb-javac srcdir="${src.dir}" destdir="${build.classes.dir}" debug="${build.compiler.debug}" debuglevel="${build.compiler.debuglevel}" encoding="UTF-8" deprecation="${build.compiler.deprecation}" optimize="${build.compiler.optimize}" source="${javac.source}" target="${javac.target}" includeantruntime="false">

<classpath refid="cp"/>
<compilerarg line="${javac.compilerargs}"/>
<processorpath refid="processor.cp"/>
</nb-javac>

with

<target name="compile-aspectj" if="aspectjcompiler">
<property name="cpProperty" refid="cp"/>
<property name="aspectj.lib.dir" location="${harness.dir}/../aspectj1.6/lib"/>
<property name="aspectjtools.jar" location="${aspectj.lib.dir}/aspectjtools.jar"/>
<property name="aspectjrt.jar" location="${aspectj.lib.dir}/aspectjrt.jar"/>

<taskdef resource="org/aspectj/tools/ant/taskdefs/aspectjTaskdefs.properties">
<classpath>
<pathelement path="${aspectjtools.jar}"/>
</classpath>
</taskdef>

<iajc destdir="${build.classes.dir}"
srcdir="${src.dir}"
source="${javac.source}"
fork="true"
forkclasspath="${aspectjtools.jar}"
classpath="${aspectjrt.jar};${cpProperty}"
failonerror="false"/>

</target>

<target name="compile-nb-javac" unless="aspectjcompiler">

<nb-javac srcdir="${src.dir}" destdir="${build.classes.dir}" debug="${build.compiler.debug}" debuglevel="${build.compiler.debuglevel}" encoding="UTF-8" deprecation="${build.compiler.deprecation}" optimize="${build.compiler.optimize}" source="${javac.source}" target="${javac.target}" includeantruntime="false">

<classpath refid="cp"/>
<compilerarg line="${javac.compilerargs}"/>
<processorpath refid="processor.cp"/>
</nb-javac>
</target>

<target name="compile" depends="init,up-to-date" unless="is.jar.uptodate">
<mkdir dir="${build.classes.dir}"/>
<depend srcdir="${src.dir}" destdir="${build.classes.dir}" cache="build/depcache">
<classpath refid="cp"/>
</depend>
<antcall target="compile-aspectj" inheritAll="true" />
<antcall target="compile-nb-javac" inheritAll="true" />

To activate the AspectJ compiler for a specific module, simply define the aspectjcompiler=required property in the project.properties file of the module.

However, since the NetBeans compiler is now no longer used, the new annotations are no longer processed. A better way would be to compile the Java and aspect files separately and then weave them.

9. Conclusion

The NetBeans platform build harness does its job pretty well, and is configurable without even having to modify scripts. Knowing how the harness works enables the developer to resolve issues that may arise when using the build harness in other ways such as on a build server. The scripts are available for modification if weird and wonderful things are required of it, such as switching off auto-update information generation and replacing the Java compiler with a different one.

---

Hermien Pellissier
Last Updated: February 11th, 2010