Put all Maven dependency versions in properties

Because security vulnerabilities in third-party software are so common, it is important to manage the versions of your dependencies and to be prepared to upgrade them quickly. For Maven projects, I recommend putting the versions of all your dependencies in the properties section of your pom.xml. Furthermore, for libraries which require multiple dependency entries in pom.xml, this allows you to ensure you use the same version for all of those dependencies.

For example:

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <properties>
    <spring.version>3.2.6.RELEASE</spring.version>
    <jackson.version>2.2.3</jackson.version>
  </properties>
  <dependencies>
    <!-- spring -->
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-core</artifactId>
      <version>${spring.version}</version>
    </dependency>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-beans</artifactId>
      <version>${spring.version}</version>
    </dependency>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-context</artifactId>
      <version>${spring.version}</version>
    </dependency>
    <!-- Jackson -->
    <dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-core</artifactId>
      <version>${jackson.version}</version>
    </dependency>
    <dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-annotations</artifactId>
      <version>${jackson.version}</version>
    </dependency>
    <dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-databind</artifactId>
      <version>${jackson.version}</version>
    </dependency>
  </dependencies>
</project>

Publishing a Java Servlet to Azure Website Using Maven

Azurelogo

This blog post shows how to publish a Java Servlet, encapsulated in a WAR file, to an Azure Web Site using FTP via Maven.

The first step will be to generate a simple servlet using Maven:

mvn archetype:generate -DgroupId=com.example -DartifactId=hello-world -DarchetypeArtifactId=maven-archetype-webapp

This creates a simple, Hello World application in the hello-world directory. We can verify it works by running it in a local servlet container using the instructions found in Supporting mvn jetty:run in Maven applications. Add the Jetty maven plugin to pom.xml under the build section:

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <build>
    <plugins>
      <plugin>
        <groupId>org.eclipse.jetty</groupId>
        <artifactId>jetty-maven-plugin</artifactId>
        <version>9.2.0.M0</version>
      </plugin>
    </plugins>
  </build>
</project>

You can now run the application in Jetty using mvn jetty:run and visit http://localhost:8080 to ensure it works. Next, you can use mvn package to generate the WAR file target/hello-world.war.

Now let’s switch to the Microsoft Azure Portal to create a Web Site and configure it to run Java applications:

  1. Click New: Website.
  2. Give the website a name (I’ll use hello-world) and then click Create.
  3. Open the website and click Site settings (in the lower left-hand corner).
  4. For Java version, change the setting from OFF to 1.7.0_51.
  5. In Web Container, choose your preferred servlet container (I use Tomcat).
  6. Click Save.

Your Java website is now running at http://hello-world.azurewebsites.net. You can upload WAR files via FTP to the directory /site/wwwroot/webapps and they will become available as part of your site.

Next, set up your website for deployment:

  1. Open the website in the Azure Portal.
  2. Click Set deployment credentials.
  3. Create a deployment username (I’ll use deployme) and password and click Save. Write down the username and password
  4. Click Properties
  5. Write down the FTP host name. Mine is ftp://waws-prod-blu-011.ftp.azurewebsites.windows.net. Also note that the FTP/Deployment User has a hello-world prefix (i.e. it is hello-world\deployme)

We’ll now configure Maven to deploy the website by uploading the WAR file when the user uses mvn install.

  1. Set up the deployment username and password for Maven by editing ~/.m2/settings.xml and add the following section:
    <settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
              xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0                               http://maven.apache.org/xsd/settings-1.0.0.xsd">
      <servers>
        <server>
          <id>azure-hello-world</id>
          <username>hello-world\deployme</username>
          <password>password</password>
        </server>
      </servers>
    </settings>
    
  2. Add wagon-ftp as a build extension to your pom.xml so we can upload files via FTP:
    <project xmlns="http://maven.apache.org/POM/4.0.0"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
      <build>
        <extensions>
          <extension>
            <groupId>org.apache.maven.wagon</groupId>
            <artifactId>wagon-ftp</artifactId>
            <version>1.0-beta-3</version>
          </extension>
        </extensions>
      </build>
    </project>
    
  3. Add wagon-maven-plugin as a build plugin in your pom.xml and instruct it to upload the WAR via FTP when the user runs mvn install:
    <project xmlns="http://maven.apache.org/POM/4.0.0"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
      <build>
        <plugins>
          <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>wagon-maven-plugin</artifactId>
            <version>1.0-beta-3</version>
            <executions>
              <execution>
                <id>upload-war</id>
                <phase>install</phase>
                <goals>
                  <goal>upload</goal>
                </goals>
                <configuration>
                  <fromDir>${basedir}/target</fromDir>
                  <includes>${project.build.finalName}.war</includes>
                  <url>ftp://waws-prod-blu-011.ftp.azurewebsites.windows.net</url>
                  <toDir>/site/wwwroot/webapps</toDir>
                  <serverId>azure-hello-world</serverId>
                </configuration>
              </execution>
            </executions>
          </plugin>
        </plugins>
      </build>
    </project>
    

You can now deploy your website using mvn install, and visit it at http://hello-world.azurewebsites.net/hello-world.

Supporting mvn jetty:run in Maven applications

jetty-header-logo

When I’m writing a Java servlet using Maven, I find it convenient to be able to run the Java servlet in a local servlet container for testing purposes.  This is very easy to do using the Jetty Maven plugin.

To add the Jetty Maven plugin to your project, modify pom.xml as follows:

<build>
  <plugins>
    <plugin>
      <groupId>org.eclipse.jetty</groupId>
      <artifactId>jetty-maven-plugin</artifactId>
      <version>9.2.0.M0</version>
    </plugin>
  ...
  </plugins>
</build>

You can then run your project in Jetty using the following command:

mvn jetty:run

If you want to run the website from the compiled WAR rather than from the source code, then use the following configuration:

<build>
  <plugins>
    <plugin>
      <groupId>org.eclipse.jetty</groupId>
      <artifactId>jetty-maven-plugin</artifactId>
      <version>9.2.0.M0</version>
      <configuration>
        <war>${basedir}/target/${project.build.finalName}.war</war>
      </configuration>
    </plugin>
  ...
  </plugins>
</build>

Then execute:

mvn jetty:run-war

Work Email Statistics Through 2013-12-31

In a previous post, I noted that I have a copy of every single email I’ve sent or received at work since mid-2003.  Here is the updated chart of the number of work emails I send and receive per year through December 31, 2013.

emails-by-year

Here’s the same data charted by month:

emails-by-month

This data was calculated by a simple Outlook PST parsing program I wrote called pststats.

Instrumenting a Java Web Application with JMX

Java Management Extensions (JMX) is a technology for managing and monitoring Java applications. JMX instructs application developers how to instrument their applications to expose metrics and management mechanisms, and instructs operations teams how to collect metrics and manage Java applications. JMX is ubiquitous: there are a large number of off-the-shelf commercial and open source tools which speak JMX.

This article is intended for application developers to introduce JMX, to teach them how to create some simple metrics via JMX, and to encourage them to thoroughly instrument their applications with JMX.; This article uses only J2SE 5 and the Java Servlet specification. There may be better, more efficient, or more appropriate ways to add JMX counters to applications using other frameworks. For example, Spring developers should read http://static.springsource.org/spring/docs/2.0.x/reference/jmx.html.

Why Instrument Your Application Using JMX?

High-quality monitoring extends far beyond OS-level metrics such as CPU, memory, and disk space. Every non-trivial application has a collection of metrics which are critical to understanding how the application is behaving in production. There is no way for any off-the-shelf tool to collect these metrics, as they are internal to the application. Some examples of these metrics are:

  • Cache hits and misses for an internal, in-memory cache
  • Number of currently logged-in users
  • Authentication successes and failures

Standardizing the way applications perform instrumentation allows us to easily integrate tools such as Graphite to enable rapid data aggregation, charting, dashboarding, and eventing. For example, the below custom dashboard was built in Graphite in less than an hour:

    image

JMX for Application Instrumentation

JMX is a complicated technology which covers far more than just application instrumentation. However, this tutorial focuses on the basics of what an application developer needs to know to implement instrumentation for his application. If you would like to know more, please read the Java JMX tutorial.

To instrument an application, the developer must do the following:

  1. Write one or more Managed Beans, or MBeans, which are Java objects which contain the metrics that the application exposes
  2. Register the MBeans with the platform MBean server

A Standard MBean is defined by writing a Java interface whose name ends with MBean and a Java class which implements this interface. For example:

public interface HelloMBean
{ 
    public int getCacheSize(); 
}

public class Hello implements HelloMBean
{ 
    public int getCacheSize() { /* return cache size */ } 
}

For the MBean to be visible to management applications, it must be registered with the platform MBean server. This is typically done with code that looks like:

MBeanServer mbs = ManagementFactory.getPlatformMBeanServer(); 
ObjectName name = new ObjectName("..."); 
Hello mbean = new Hello(); 
mbs.registerMBean(mbean, name);

MBeans may also generate notifications to signal state changes, detected events, or problems. This can be used to move from a polling to a push-based update mechanism.  This is done by having the MBean class inherit from NotificationBroadcasterSupport and calling notify().

JmxCalcService – A Simple Web Application Instrumented with JMX

We will start with a simple web service called JmxCalcService which supports two URLs, add and subtract.  These URLs are implemented by two separate servlets, com.morningstar.jmxcalcservice.servlet.AddServlet and com.morningstar.jmxcalcservice.servlet.SubtractServlet.

Here is the implementation of AddServlet (SubtractServlet is virtually identical):

package com.morningstar.jmxcalcservice.servlet;

import java.io.IOException;
import java.io.PrintWriter;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

public class AddServlet extends HttpServlet
{
    public void doGet(HttpServletRequest request,
                      HttpServletResponse response)
        throws ServletException, IOException {

        int val1 = Integer.parseInt(request.getParameter("val1"));
        int val2 = Integer.parseInt(request.getParameter("val2"));
        int result = val1+val2;

        PrintWriter out = response.getWriter();
        out.println(result);
        out.flush();
        out.close();
    }
}

These servlets are bound to URLs using the WEB-INF/web.xml file:

<!DOCTYPE web-app PUBLIC
 "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
 "http://java.sun.com/dtd/web-app_2_3.dtd" >

<web-app>
  <display-name>jmxcalcservice - A simple calculation web service
    instrumented via JMX</display-name>

  <servlet>
    <servlet-name>add</servlet-name>
    <servlet-class>com.morningstar.jmxcalcservice.servlet.AddServlet</servlet-class>
  </servlet>
  <servlet>
    <servlet-name>subtract
    <servlet-class>com.morningstar.jmxcalcservice.servlet.SubtractServlet</servlet-class>
  </servlet>
  <servlet-mapping>
    <servlet-name>add</servlet-name>
    <url-pattern>/add</url-pattern>
  </servlet-mapping>
  <servlet-mapping>
    <servlet-name>subtract</servlet-name>
    <url-pattern>/subtract</url-pattern>
  </servlet-mapping>
</web-app>

JmxCalcService may be tested as follows:

$ mvn jetty:run 
$ curl 'http://localhost:8080/add?val1=5&val2=8'
$ curl 'http://localhost:8080/subtract?val1=9&val2=4'

View JMX Instrumentation Using JConsole

The JVM is already instrumented with JMX, and the Servlet container often is (although typically the instrumentation on the latter must be explicitly enabled). We can view this instrumentation using JConsole. Let’s start up the web service and then view its instrumentation. Be sure to start the web service first so that JConsole can find the process:

$ mvn jetty:run 
$ jconsole

Find the Jetty process and click “Connect”:

image

JConsole opens up a window which is updated in real-time with the state of the application:

image

You can view individual MBeans by clicking on the MBeans tab.  For example, we can view the value of the java.lang.ClassLoading.LoadedClassCount attribute as follows:

image

Adding Metrics to JmxCalcService

We will add two metrics to JmxCalcService: the number of times that add is called, and the number of times that subtract is called. These metrics will be implemented in a MBean whose interface is called com.morningstar.jmxcalcservice.mbean.JmxCalcServiceMBean:

package com.morningstar.jmxcalcservice.mbean;

public interface JmxCalcServiceMBean
{
    public int getNumAdds();
    public int getNumSubtracts();
}

The implementation of the MBean is in the class com.morningstar.jmxcalcservice.mbean.JmxCalcService This class also includes two additional functions, incrementNumAdds() and incrementNumSubtracts() These functions are called by the servlets each time their methods are called. These functions are not exposed through the MBean interface and are not available to JMX management applications.

JmxCalcService is implemented as a Singleton with thread-safe counters, as we only want one instance of the data and increments can happen on any thread. It’s implementation as below:

package com.morningstar.jmxcalcservice.mbean;

import java.util.concurrent.atomic.AtomicInteger;

public class JmxCalcService
    implements JmxCalcServiceMBean
{
    private static JmxCalcService s_instance  = new JmxCalcService();

    public static JmxCalcService getInstance()
    {
        return s_instance;
    }

    private AtomicInteger numAdds = new AtomicInteger(0);
    private AtomicInteger numSubtracts = new AtomicInteger(0);
    private JmxCalcService() { }

    public int getNumAdds() { return this.numAdds.get(); }
    public int getNumSubtracts() { return this.numSubtracts.get(); }
    public int incrementNumAdds() { return
        this.numAdds.incrementAndGet(); }
    public int incrementNumSubtracts() { return
        this.numSubtracts.incrementAndGet(); }
}

The JmxCalcService MBean must be registered with the platform MBean server. The proper way to register MBeans varies based on application type or servlet container. For Jetty, the following technique works well:

First, create a ServletContextListener which registers the MBean at application startup time and deregisters it at application shutdown time:

package com.morningstar.jmxcalcservice;

import java.lang.management.ManagementFactory;
import javax.management.MBeanServer;
import javax.management.ObjectName;
import javax.servlet.ServletContext;
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import com.morningstar.jmxcalcservice.mbean.JmxCalcService;

public class JmxCalcServiceContextListener
    implements ServletContextListener
{
    private static final String mbeanName =
        "com.morningstar.jmxcalcservice:type=JmxCalcService";

    public void contextInitialized(ServletContextEvent event) {
        try {
            MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();
            ObjectName objName = new ObjectName(this.mbeanName);
            JmxCalcService mbean = JmxCalcService.getInstance();
            mbs.registerMBean(mbean, objName);
        } catch (Exception ex) {
            System.err.println("Error registering JmxCalcService MBean: " +
                ex.toString());
        }
    }

    public void contextDestroyed(ServletContextEvent event) {
        try {
            MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();
            ObjectName objName = new ObjectName(this.mbeanName);
            mbs.unregisterMBean(objName);
        } catch (Exception ex) {
            // Ignore unregistration failures
        }
    }
}

Second, add JmxCalcServiceContextListener to the web.xml file:

...
<web-app>
  <display-name>jmxcalcservice - A simple calculation web service
    instrumented via JMX</display-name>

  <listener>
    <listener-class>com.morningstar.jmxcalcservice.JmxCalcServiceContextListener</listener-class>
  </listener>

  <servlet>
    <servlet-name>add</servlet-name>
    <servlet-class>com.morningstar.jmxcalcservice.servlet.AddServlet</servlet-class>

We must also change AddServlet and SubtractServlet to call incrementNumAdds() or incrementNumSubtracts() at the end of the function:

public class AddServlet extends HttpServlet {
    public void doGet(HttpServletRequest request,
                      HttpServletResponse response)
        throws ServletException, IOException {

        int val1 = Integer.parseInt(request.getParameter("val1"));        
        int val2 = Integer.parseInt(request.getParameter("val2"));
        int result = val1+val2;

        PrintWriter out = response.getWriter();
        out.println(result);
        out.flush();
        out.close();

        JmxCalcService.getInstance().incrementNumAdds();
    }
}

After all this, we can start the application, hit the URLs a few times, load JConsole and see the values:

image

What’s Next?

After you instrument your application with JMX, consider hooking up your application to your centralized monitoring system or using jmxtrans to collect and send metrics to Graphite so that you can chart and trend data over time.  Of course, if Graphite is your ultimate goal, statsd might be a better choice.

Creating a Virtual Java RPM

javaSome RPMs (e.g. jpackage.org’s tomcat7-7.0.39-1.jpp6.noarch.rpm) express their dependency upon Java by requiring a RPM that provides capability java (as opposed to, for example, depending on the existence of a file /usr/bin/java). On CentOS, this capability is normally provided by the java-*-openjdk RPM. Therefore, if you execute # yum install tomcat7 on a clean install of CentOS, yum will install OpenJDK in addition to Tomcat 7.

Some people prefer to run the Oracle JRE/JDK instead of OpenJDK. Oracle provides RPMs named jre-version-linux-x64.rpm and jdk-version-linux-x64.rpm to make installing them easier. Unfortunately, these RPMs do not provide the capability java. This means that if you already have the Oracle JRE installed, and you install a RPM which requires the capability java, the OpenJDK will be unnecessarily installed (and might even become the default!).

I solved this dilemma by creating a ‘virtual’ RPM package which provides the capability java by depending on the Oracle JRE. I named this package virtual-java.

Creating this package is quite easy. First I created a Makefile to make building the RPM easier and deal with rpmbuild’s nonsense regarding _topdir:

VERSION=1.7
RELEASE=1
RPMNAME=virtual-java-$(VERSION)-$(RELEASE).noarch.rpm

.PHONY: all
all: dist/$(RPMNAME)

.PHONY: clean
clean:
	rm -rf work
	rm -rf dist

dist/$(RPMNAME): work/RPMS/noarch/$(RPMNAME) dist
	cp work/RPMS/noarch/$(RPMNAME) dist/$(RPMNAME)

work/RPMS/noarch/$(RPMNAME): work/BUILD work/RPMS/noarch work/SPECS/virtual-java.spec
	rpmbuild -bb --define="_topdir ${PWD}/work" work/SPECS/virtual-java.spec

work/SPECS/virtual-java.spec: work/SPECS virtual-java.spec
	cat virtual-java.spec | sed -e s/%VERSION%/$(VERSION)/g | sed -e s/%RELEASE%/$(RELEASE)/g > work/SPECS/virtual-java.spec

dist:
	if [ ! -d dist ]; then mkdir -p dist; fi
	touch dist
work/BUILD:
	if [ ! -d work/BUILD ]; then mkdir -p work/BUILD; fi
	touch work/BUILD
work/RPMS/noarch:
	if [ ! -d work/RPMS/noarch ]; then mkdir -p work/RPMS/noarch; fi
	touch work/RPMS/noarch
work/SPECS:
	if [ ! -d work/SPECS ]; then mkdir -p work/SPECS; fi
	touch work/SPECS

Note how the Makefile defines VERSION and RELEASE, and provides them to the .spec file. This is done because the Makefile needs to know the name of the generated RPM file, which depends on VERSION and RELEASE.

Here is virtual-java.spec:

Name: virtual-java
Version: %VERSION%
Release: %RELEASE%
Group: ENTER GROUP HERE
Summary: Virtual package which provides java but uses the Sun/Oracle JRE
License: None
BuildArch: noarch
Provides: java
Requires: jre > %VERSION%

%description
Virtual package which provides java but uses the Sun/Oracle JRE

%prep

%build

%pre

%post

%install

%files

%changelog