Setting user jar classpath precedence in Hadoop MapReduce jobs

I encountered the following error when executing a Hadoop MapReduce job today:

Error running child : java.lang.NoSuchMethodError:
org.apache.commons.codec.binary.Base64.encodeAsString([B)Ljava/lang/String;

The problem was caused by clashing entries on my classpath. My jar had a dependency on Commons Codec 1.6, yet the Hadoop lib directory contained version 1.4 of the same jar (used by the Hadoop framework). This Hadoop library jar was taking precedence at runtime, hence the NoSuchMethodError.

The solution, as it turned out, is pretty simple — there’s a property you can set to make user classes take precedence: mapreduce.task.classpath.user.precedence which should be set to true.

There’s even a convenience method on the Job class in Cloudera’s distribution to set this for you:

Job job = Job.getInstance(conf);
job.setUserClassesTakesPrecedence(true);
job.setJarByClass(ExampleClass.class);

Creating an ExtJS Production Build with Sencha SDK tools

I’m working currently on an application with an ExtJS based front-end, and wanted to share my experiences at using the Sencha SDK tools to create a production build. ExtJS is a fantastic framework, with a very well documented API. The same unfortunately can’t be said for the SDK tools (at the time of writing, anyway) so perhaps someone will find this useful.

First things first — why a production build? JavaScript is a dynamic programming language, and you’ve probably been running your ExtJS application just fine in a development environment without any sort of build process. For deployment into a production environment, however, it is highly recommended to create a production build of the application. The build process involves packaging together all of your own application source code along with all of the ExtJS files on which you have dependencies, to create a single file which is then minified to remove comments, whitespace, etc. The process leads to a much smaller amount of data being transferred over the network (in my case a saving of over 70%) and far fewer network requests (2 JavaScript files for the production build compared to typically several hundred source files). It’s also more efficient even without these two considerations, because running the app from the source code and relying on the Ext Loader means that files have to be dynamically loaded once dependencies are identified, which inevitably leads to increased latency as the app waits to download and load files, only to then find out about further dependencies which it can only then start to download, etc. This has a noticeable difference on initial page load time for your users. It’s also good practice to remove comments and debug code and this can also be handled as part of the build process.

Continue reading

Deploying EJB3 Message Driven Beans in JBoss

Back in the ‘old days’ if you wanted to configure a message driven bean you’d need to write reams of accompanying XML configuration. Thankfully, EJB3 came along, and with it the wonderful world of annotations. Now all you need to do is annotate your implementation of the MessageListener interface, pass it over to JBoss and have the application server take care of things for you.

Continue reading

CSS3 pulsate animations

On a recent project, I created a webpage in which a value could be dynamically updated based on a user’s action. Now, it should be obvious to the user that the value had changed, but I wanted to add a small visual cue to add to the aesthetics so I decided to look into CSS3 animations. Support, at time of writing, isn’t great it has to be said – but most modern browsers do support it, with the notable exception of Internet Explorer. The webkit-based browsers (Safari and Chrome) have offered support for a while, whilst Firefox has had support since v5, released back in June.

Continue reading

No setter method for a webservice list

A few people have asked me this question recently, so I thought this might be a tip worth sharing more widely than just my office.

If you generate a Java client for a web service using a tool like wsimport or wsconsume you may be surprised to note that no setters are generated for any Collection properties.

This is actually by design, and is due to the implementation of JAXB. I quote from the JAXB2.1 specification:

Design Note – There is no setter method for a List property. The getter returns the List by reference. An item can be added to the List returned by the getter method using an appropriate method defined on java.util.List. Rationale for this design in JAXB 1.0 was to enable the implementation to wrapper the list and be able to perform checks as content was added or removed from the List.

So now you know!

Ignoring whitespace in ClearCase

A colleague recently (and unknowingly) had their IDE set to convert line endings to Windows (cf Unix) format when saving a Java file. Now, that’s since been corrected but it’s left us in a position where if we compare (using ClearCase) the latest version of a number of files, with their predecessors, every line is showing as having changed.

The solution turns out to be a single click away – the ClearCase comparison tool includes a nifty button which will hide any changes that solely affect whitespace. And here it is, highlighted yellow in the following screenshot:

ClearCase

It’s function is not immediately obvious, probably hence why I’d not spotted it sooner. I believe the b stands for blank_ignore – the name of the corresponding flag in the cleardiff command-line version of the tool.

Setting Java system properties in JBoss

JVM system properties can be a convenient and sensible way to manage certain elements of your application’s configuration; certainly a good alternative to hard-coding certain values. Accessing such properties with a call to System.getProperty("propertyName") is easy and unobtrusive, and since system properties are given global JVM scope you needn’t worry about problems with multiple classloaders.

Sometimes you may come across a need to set system property values within JBoss (just as the System.setProperty("propertyName", "propertyValue") method and the VM command line arguments do). Happily, JBoss 4 includes a system properties service which helps you do just that.

To make use of the service, just add an appropriate XML file to your deploy directory. You can either specify the properties right within the file, or reference an external config file. The latter option is likely preferable, certainly if there’s more than a few properties to reference. The advantage of this approach as well, is that if you run multiple JBoss instances on the same server, you can have each reference one single “master” config file.

Continue reading