Data-Driven Code Generation of Unit Tests Part 3: Java, Maven, StringTemplate, JUnit

This post is part 3/5 of my Data-Driven Code Generation of Unit Tests series.

This blog post explains how I used Java, Apache Maven, StringTemplate, and JUnit to perform data-driven code generation of unit tests for a financial performance analytics library. If you haven’t read it already, I recommend starting with Part 1: Background.

As mentioned in Part 2: C++, CMake, Jinja2, Boost, all performance analytics metadata is stored in a single file called metadata.csv. This file drives all code generation and is what helps ensure inter-platform consistency.

In order to integrate code generation into the Maven build process, I was forced to create three separate Java projects:

A Maven project which implements the unit test generation using metadata.csv and StringTemplate (java-gentest-srcgen)
A Maven project which builds a Maven plugin which calls the unit test generator at the right point in the Maven build lifecycle (java-gentest-maven-plugin)
A Maven project which implements the calculations and uses the Maven plugin to generate the source code (java-lib)

All three projects are tied together using a single parent POM which looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    ...
    <packaging>pom</packaging>
    ...
    <modules>
        <module>java-gentest-srcgen</module>
        <module>java-gentest-maven-plugin</module>
        <module>java-lib</module>
    </modules>
</project>

Unit test source code generator

The Java unit test generator project (java-gentest-srcgen) creates a JAR with a single class, JavaUnitTestGenerator, with a single public method generate(). This method parses metadata.csv and calls StringTemplate to generate a unit test class for each found calculation.

The code looks something like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69


public class JavaUnitTestGenerator {
    // A simple logging interface defined in this project so we can integrate
    // into any logging system, including Maven's
    private SimpleLogger logger;
    private File targetDirectory;

    public JavaUnitTestGenerator setLogger(SimpleLogger logger) {
        this.logger = logger;
        return this;
    }

    public JavaUnitTestGenerator setTargetDirectory(File targetDirectory) {
        this.targetDirectory = targetDirectory;
        return this;
    }

    public void generate() throws Exception {
        // The Maven build process adds metadata.csv as a resource to
        // the JAR so that the JAR becomes self-contained
        try (InputStream is = getClass().getResourceAsStream("metadata.csv")) {
            try (InputStreamReader sr = new InputStreamReader(is)) {
                try (BufferedReader br = new BufferedReader(sr)) {
                    String[] headers = br.readLine().split(",");
                    String line;
                    while ((line = br.readLine()) != null) {
                        String[] arr = line.split(",");
                        Map<String, String> kvp = new HashMap<String, String>();
                        for (int i = 0; i < arr.length; ++i) {
                            kvp.put(headers[i], arr[i]);
                        }
                        generateUnitTest(kvp);
                    }
                }
            }
        }
    }

    private void generateUnitTest(Map<String, String> kvp) throws Exception {
        // Instantiate the template
        URL url = Resources.getResource("unit-test.stg");
        STGroup g = new STGroupFile(url, "US-ASCII", "<", ">");
        ST template = g.getInstanceOf("unit_test");
        for (Map.Entry<String, String> entry : kvp.entrySet()) {
            if (template.getAttributes().containsKey(entry.getKey())) {
                // Don't pass in false values so that if(variable) evaluates to false
                if (!entry.getValue().equals("false")) {
                    template.add(entry.getKey(), entry.getValue());
                }
            }
        }
        // Add various transformations and manipulations to the attributes
        // found in metadata.csv that cannot easily be done in StringTemplate's
        // templating language
        template.add("algorithm_type_pascal_case", toPascalCase(kvp.get("algorithm_type")));
        template.add("function_name_pascal_case", toPascalCase(kvp.get("function_name")));
        template.add("function_name_camel_case", toCamelCase(kvp.get("function_name")));
        ...

        // Generate the source code file
        File tgtFile = new File(targetDirectory, toPascalCase(kvp.get("function_name")) + "UnitTest.java");
        logger.info("Generating " + tgtFile + "...");
        try (FileOutputStream os = new FileOutputStream(tgtFile)) {
            try (OutputStreamWriter osw = new OutputStreamWriter(os)) {
                STWriter stWriter = new AutoIndentWriter(osw);
                template.write(stWriter);
            }
        }
    }
}

The template itself (unit-test.stg) looks something like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


group javagentest;

unit_test(algorithm_type,
          algorithm_type_pascal_case,
          function_name,
          ...) ::= <<

package com.morningstar.perfanalytics.tests;

...

public class <function_name_pascal_case>UnitTest {
    // BEGIN ARRAY TESTS
    @Test
    public void testArrayUnannualized() {
        ....
        double expected = <expected_value_unannualized>;
        double actual = ...;
        assertEquals(expected, actual, 0.00001);
    }

    ....
>>

Unfortunately, because StringTemplate’s templating language is so weak, we’re stuck with the following limitations:

The JavaUnitTestGenerator class must perform a number of presentation-oriented transformations on attributes found in metadata.csv, such as case conversions or creating small Java snippets. This means that the JavaUnitTestGenerator and the template are extremely tightly coupled in non-obvious ways.
The lack of a for-loop construct in the template language means we cannot perform tricks like we did in Part 2, where we generate all possible combinations of annualization, frequency, etc. for a given calculation.

Maven plugin

The Maven plugin project creates a maven-plugin that wraps the unit test source code generator. The source code looks like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25


@Mojo(name = "generate", requiresProject = true, threadSafe = false, requiresDependencyResolution = ResolutionScope.COMPILE_PLUS_RUNTIME, defaultPhase = LifecyclePhase.GENERATE_SOURCES)
public class JavaUnitTestCodegenMojo extends AbstractMojo {
    @Parameter(defaultValue = "${project}")
    private MavenProject project;
    @Parameter(property = "outputDirectory", defaultValue = "${project.build.directory}/generated-test-sources/unit-tests")
    private File outputDirectory;

    @Override
    public void execute() throws MojoExecutionException {
        getLog().info("Generating unit tests for Java calculation library");

        // MojoLogger adapts Maven's logging class to the SimpleLogger
        // interface in the java-gentest-srcgen project
        JavaUnitTestGenerator g = new JavaUnitTestGenerator()
            .setLogger(new MojoLogger(getLog()))
            .setTargetDirectory(this.outputDirectory);
        try {
            g.generate();
        } catch (Exception ex) {
            throw new MojoExecutionException("Error generating source code", ex);
        }

        project.addTestCompileSourceRoot(outputDirectory.getPath());
    }
}

Note how the plugin instructs the calling project to add the generated source code directory to the test compile source root. This way, the calling project doesn’t need to remember to configure this in its pom.xml.

Calculation library

The calculation library contains the implementation of all calculations. It references the Maven plugin project in its pom.xml in order to have the Maven plugin create its unit tests as part of the build process. The relevant stanza looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


...
<build>
    <plugins>
        <!-- Generate unit test code -->
        <plugin>
            <groupId>com.morningstar.perfanalytics</groupId>
            <artifactId>perfanalytics-java-gentest-maven-plugin</artifactId>
            <version>...</version>
            <executions>
                <execution>
                    <goals>
                        <goal>generate</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

The Java library also has a few other, hand-written unit tests, in the normal directory (src/test/java/...). The build system knows how to compile and run both the hand-written and the machine-generated unit tests at build time.

I found this approach reliable, and mostly straightforward, but with a few drawbacks. First, the unit test generator, as coded, will generate the source code every time, even if nothing has changed. This means we will perform a lot of unnecessary re-compilation, slowing down builds. Second, weaknesses in the templating language make it frustrating to use; I have to frequently switch back-and-forth between the template file and the Java code to achieve my desired result. Third, it’s unfortunate that we have to write a custom Maven plugin just to perform source code generation at build time; it’d be nice if there was a simpler way. Fourth, the versions of all the Maven projects must be kept in sync at all times; fortunately, the Maven release plugin does this for us automatically.

Next time I’ll talk about doing this in C#, MSBuild, T4 Text Templates, and the Microsoft Unit Test Framework for managed code!