Almost a third of our codebase consists of POJOs generated from Xsd descriptors scattered around in our projects. The generation is performed during build, using a Maven plugin that performs the parsing and generation tasks.

These generated classes are no more than simple value objects descending from a common root object, that contains some well-known methods like toString (), that all methods should share. Unfortunately this particular toString does not know about it’s descendants’ inner structure, so it relies heavily on reflection to perform it’s task. This springs the idea of generating the classes with their own toString methods, so the resulting code is more efficient.

To do this to be backwards compatible, I would have to generate all known xsds, invoke their old and generated toString implementation and compare the results. Also I’d have to fill these objects with values, so I’d have data to compare.

So much for the task, but what’s the fuss about? Well I want the testing kept inside Maven, so should the generated or  inherited code change, we can see it in the build process.

There seem to exist a vast number of ways to be used for mojo testing, starting from the blunt get a mojo instance, fill it with values, and run execute (), to running a separate Maven instance and use the generated test results. Given that in this particular case I’d to start the compiler and then the surefire I (eventually ) went for the later approach.

The plugin testing harness is an example of the earlier approach, while it seems from the usage example the former. Avoid if you have to invoke more, than just your own mojo. It only reads the pom.xml you provide, but doesn’t set up the required Maven environment like builders, project, …etc. It doesn’t do the setup of the default values of your mojo (not that it could resolve the non-existing values for that) So all you end up with is using a helper method for setting up your mojos and executing them from your testcase. I went as far as generating and compiling my classes, but when it came to the point of actually testing them with a surefire mojo I gave up on it, as setting that up would really have been impossible.

I then realized that from testing point of view it is in fact an integration test, not a “mere” unit test, as the testing involves interaction with third party plugins (compiler and surefire) as well. So I converted the package from the plugin testing harness’s structure to the maven-invoker plugin’s recommended structure.  It was nothing more than moving a few directories around, but I could simply throw the complicated mojo invocation out and use a well defined and simple pom instead.  This plugin creates a local repository for itself that contains your project’s dependencies and the artifact to be tested. It’s advised to create a settings.xml that points to your local repository, to limit the unnecessary fetching from your repositories. 

So far, so good. You should keep in mind however, the invoker plugin has some disadvantages. It’s way slower than the harness. It doesn’t aggregate the test results, so all you can see on a failed build, that there were errors in a particular run. Also, since it runs the test in a separate Java environment, you cannot debug your test code, which in my case is quite complex.

So you either create a separate test module and refer to it from the test pom, and create a unit test to test your code with known data, or just be brave write well thought out code and do it properly using log messages to trace your work, just like the good old times. After all repeated failure builds character. 

I actually created a small and a full test environment, testing the particularities of the test separately. To debug the test code you’ve got to import the generated Maven project in your IDE and start the test execution with the local repository and the generated settings.xml.

What were the results? I achieved a speedup of about 90% for completely empty classes, about 70% for classes with data, when the output buffer is limited, and around 30% for classes recursively and completely filled in a depth of four iterations. Overall I could say it’s a 50% improvement on the original method, with a proven backwards compatibility. Considering that this is invoked at least 6 million times a day it might bring a favorable impact on the performance of our systems.

In the process I found 3 errors in the current code, that proved that the output of method’s not used to heavily in the live system. Since the logging it produces is mandatory for regulatory compliance it might just be a good idea to roll it out in the near future.

When I was running my company I interviewed several developers for open positions, and interestingly there was an unspeakable imbalance in the gender ratio. Namely I can only recall two ladies I interviewed, out of about a hundred subjects.

I’m trying to look into the root cause for this matter.

Gender differences ?

This is the answer that comes up easiest, but this is also the most controversial of all. I can almost hear the pro-women league boo. This however has an aspect that should not be overlooked or swept aside too easily.

I do believe that there are no differences whatsoever in the brain capacities of the genders, also I firmly believe that women are equally capable of reasoning and thinking as men. However I think that, probably due to upbringing or whatnot, women are generally better at tedious and repetitive tasks, while men are generally better at innovative jobs. Wait a second here, one would argue, doesn’t enterprise development consists mainly of tedious and repetitive tasks? Yes, I should say, that most enterprise development takes place in a framework, that uses the same pattern over and over again. So women should by all means be better at it then men. They would make less errors and would be better finding them!

So do I think the reason is for having so few of them?

Women don’ like taking risks BS. there is no risk of becoming a high-paid SW developer,

Women are not interested in careers BS. they are, as much as men,

Women won’t sacrifice their families BS. this would mean there were loads of female juniors changing careers,

but wait, lets go back in time a bit, to

Higher education

Anyone knowing me, knows how much I loathe the higher education in Hungary. I believe it’s a relic and should be dealt with the respect it deserves (ie. none) So lets walk through the steps a kid decides gets her/his first graduate job.

• The kid does the entrance exams, passes and gets admitted.
• He/she takes classes and does the exams,
• The student chooses a topic for the thesis, prepares, and submits it.
• The student passes the finals, and graduates.
• He/she attends some recruitment fairs, talks to several HR people, submits loads of CVs, and attends several interviews, to get the first job.

What could go wrong? It’s just studies, you decide you want to be in IT and attend a school for it.

Choosing the career

Let’s look at it in detail. How do you choose IT? When a boy plays about with the computer, he’s considered a wiz by the family, “He installed this and that on the computer, and removed viruses, and built his own, look at it with all the flashing LEDs and glow-in-the-dark stickers! He’s going to be a computer guy!” And so be it. The parents push the kid toward the career. As for a girl? This is not the kind of thing she should do. Is there anything remotely related to computer science in that above? It’s only an expensive LEGO, except girls get Barbies (and Apples, as those are more fashionable for a girl )

Preparing for the admission tests

If a girl is interested in CS, and wants to know more about it how could she proceed? You must pass your finals and the admission tests for the faculty. This always means, you must do Maths, and maybe even Physics on a higher level. These subjects are not considered to be “girl” subjects, and this is suggested to everyone through a lifetime. Even if she were talented and interested she would be discouraged at a point by family, friends, or the teachers themselves.

Will you actually use these subjects in the field? Definitely NO. Do you have any use of studying them? Unfortunately yes. These subjects are the only way as of now to acquire the basics of logical and methodological thinking.

Studying through university/college

When you’re in you just have to sit through several boring lectures, pass some boring exams high on caffeine and that’s it. Well as for sitting through the classes she’d have to face the same ordeal that she faced before getting there. I know of a lecturer, who told his students plainly, that no matter what he will NOT allow any girls to graduate, as they are not fit for being an engineer! (some still passed though on one of his better days)

I won’t complain about the current state of education, and the impracticality of the curriculum here, that should take it’s own post later on. But it all adds up to the discouraging atmosphere.

Thesis

Choosing a topic is easy, isn’t it? You get a list of topics you are in no respect prepared to wrestle, and in better cases you get a sponsor and and outside consultant to go with, in worse cases you get a topic and an ignorant inside consultant. (If you bring your own topic, than you are either a high-flier, who actually knows something, or a conceited wannabe, who will fail miserably.) Take someone who’s been told/suggested so many times, that she’s not for this career, and you’ll get someone who hesitates. And by the time she makes her mind up all the good subjects are gone. She can take the leftovers, with a disgruntled consultant.

Still the thesis is prepared and submitted. It has to be original, and imaginative. (Gosh, we are talking about preparing for a job that is not much more than patterns glued together with simple instructions!) Good thing they don’t mess around much on judging the thesis.

Recruitment

Now the last (?) hurdle. All she has to do is convince

• a recruiter, who probably has no idea how a computer works, or what the expectations are at the designation,
• HR, who are the counterparts of the recruiter,
• the technical team leader,

at any position she applies for, that despite she knows nothing (this is rooted in the rotten education system and is not her fault), she’s the ideal candidate for the job. She has to fight all their prejudice to even get a job, where she could actually get to learn the ropes. But wait, there are still lots of boys queuing up for the same position. So they either end up in administration, project management or (like the only female member of our study group) working behind a counter at a computer parts shop. If they insist on taking a technical job, the odds are high it won’t be at a respected company but at a web-app sweatshop, with no good prospects, as she won’t get a chance of learning the best practices.

Conclusion

There is no way to change this from one day to another. There is a need to change the general attitude towards women in technical positions. Even if it the change starts it will take many generations to level up.

On the positive side I’ve lately seen several examples of contractors providing ladies as developers, and trust me they are no worse than the guys they sent. They all suck.

Side note

OK. Once you got up to this, I have to admit a few things

• I’m prepping for an interview tomorrow, so I’m trying to get my English writing and thematic vocabulary in shape,
• I’m currently working with some aged whiz-kids who get on my nerves at times, who could easily be replaced by hard-working people, or simply working would even be an improvement.

The task is to write a method that takes a string as a parameter and returns the acronym of the string in uppercase made up of the first letters of the words in the string. The acronym must ignore the words “the”, “of” and “and”.

The usual solutions are either to sequentially step through the string (Yuck!) or split it up or use a StringTokenizer class. The people usually overlook the fact, that the input strings can be padded with whitespace, or contain multiple spaces, and they usually ignore, that the keywords that are to be omitted might be found on the begining of a valid word. Thus my test ” United   States of Andorra” string breaks most of the methods.  The ones who have time to write the answer down, usually forget to return the value from the method, or to change it to uppercase and sometimes even ignore that it should be a method to start with! This is my favourite question, as it can really show how the applicant can handle stressful situations.

I was tired after several interviews one day and tried to come up with the  shortest possible solution. Naturally it contains regular expressions.

My solution looked something like this (OK I just reproduced it for the sake of the article, using nano and javac, so it might have overlooked flaws in it):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

public class Acronym { 
 
private static String toAcronym(String str) {
        return str.toUpperCase().
                        replaceAll("(THE|OF|AND)(\\W+|$)","").
                        replaceAll("(\\w)\\w*\\W*","$1");
}
 
public static void main (String args[]) {
                if(args.length>0) {
                        System.out.println(toAcronym(args[0]));
        }
}
 
}

So instead of designing the WSDL along with the matching schema definition for the connector classes, I decided to start of by designing the interfaces and the connector classes, then just create the implementation with the corresponding annotations on the server side, that will be deployed automatically. The WSDL is generated from the annotations and the connector classes, there is no need to write them manually.

Take the following example. The web service has a simple method to query a list of items, that can throw a simple exception when the database is unreachable.

import java.util.List;

import dummy.ws.type.Item;

public interface ItemService {
    public List<Item> getItemList() throws ItemException;
}

with the implementation example like:

import java.util.List;

import dummy.ws.type.Item;

import javax.jws.WebMethod;
import javax.jws.WebService;

@WebService(serviceName = "ItemService", targetNamespace = "http://dummy/ws", portName = "ItemPortType")
public class ItemServiceImpl implements ItemService {
    @WebMethod
    public List<Item> getItemList() throws ItemException {
        // TODO implementation comes here
    }
}

can be invoked with the following simple code

...
    Service service = Service.create(new URL(URLBASE
                     + "/ItemService?wsdl"), new QName(
                     "http://dummy/ws", "ItemService"));
    ItemService itemService = service.getPort(new QName(
                             "http://dummy/ws", "itemPortType"),
                             ItemService.class);
    List<Item> itemList = itemService.getItemList();
...

There are several problems with this, even though it would be great to have it working right away. It could actually work, if there were no Collections involved on the return value. So creating a simple wrapper class for the return value like:

package dummy.ws.types;

import java.util.List;

public class ItemList {

        private List<Item> content;
        public ItemList() {
        }

        public ItemList(List<Item> content) {
               this.content = content;
       }

        public List<Item> getContent() {
                return content;
        }

        public void setContent(List<Item> content) {
                this.content = content;
        }
}

along with the modification of the interface and of course the implementation to use the newly created class should break the ice.

There is another shortcoming to this method yet. The thrown exception must contain a constructor with (String, Object) parameters, and an Object getFaultInfo method. Took me quite a time to figure it out, as there is a huge gap in the documentation when you try this. Once you’ve done that you are ready to deploy.

In the project we are using WebSphere Application Server Community Edition, which is an Apache Geronimo based quite stable and fast server. The WAS CE uses Axis2 for web service implementation. The rich client we implemented used the latest Axis2 libs.

The whole thing worked well until the middle of the project, when all of a sudden, without any major change that we could trace, the client suddenly started throwing JAXB exceptions (classname nor any of its super class is known to this context). This could only be cured by creating the jaxb.index in the ws.types package with all the connector classnames listed in it, along with an ObjectFactory that has a XXX createXXX () method for each ws.type class. Intrestingly however this class, while mandatory is never touched by the code, the class is not even loaded!

On the whole this approach provided a really quick and a bit dirty way of using the web services in a single project. However I would not dare to use it on a bigger scale. I still consider starting from the WSDL and the schema, then generating the interfaces and the connector classes from them the best approach.

Once upon a time, in a kingdom not far from here, a king summoned two of his advisors for a test. He showed them both a shiny metal box with two slots in the top, a control knob, and a lever. “What do you think this is?”

One advisor, an engineer, answered first. “It is a toaster,” he said. The king asked, “How would you design an embedded computer for it?” The engineer replied, “Using a four-bit microcontroller, I would write a simple program that reads the darkness knob and quantizes its position to one of 16 shades of darkness, from snow white to coal black. The program would use that darkness level as the index to a 16-element table of initial timer values. Then it would turn on the heating elements and start the timer with the initial value selected from the table. At the end of the time delay, it would turn off the heat and pop up the toast. Come back next week, and I’ll show you a working prototype.”

The second advisor, a Java developer, highly skilled in object oriented design, immediately recognized the danger of such short-sighted thinking. He said, “Toasters don’t just turn bread into toast, they are also used to warm frozen waffles. What you see before you is really a breakfast food cooker. As the subjects of your kingdom become more sophisticated, they will demand more capabilities. They will need a breakfast food cooker that can also cook sausage, fry bacon, and make scrambled eggs. A toaster that only makes toast will soon be obsolete. If we don’t look to the future, we will have to completely redesign the toaster in just a few years.”

“With this in mind, we can formulate a more intelligent solution to the problem. First, create a class of breakfast foods. Specialize this class into subclasses: grains, pork, and poultry. The specialization process should be repeated with grains divided into toast, muffins, pancakes, and waffles; pork divided into sausage, links, and bacon; and poultry divided into scrambled eggs, hard-boiled eggs, poached eggs, fried eggs, and various omelet classes.”

“The ham and cheese omelet class is worth special attention because it must inherit characteristics from the pork, dairy, and poultry classes. Thus, we see that the problem cannot be properly solved without multiple inheritance. At run time, the program must create the proper object and send a message to the object that says, “Cook yourself.” The semantics of this message depend, of course, on the kind of object, so they have a different meaning to a piece of toast than to scrambled eggs.”

“Reviewing the process so far, we see that the analysis phase has revealed that the primary requirement is to cook any kind of breakfast food. In the design phase, we have discovered some derived requirements. Specifically, we need an object-oriented language with multiple inheritance. Of course, users don’t want the eggs to get cold while the bacon is frying, so concurrent processing is required, too.”

“We must not forget the user interface. The lever that lowers the food lacks versatility, and the darkness knob is confusing. Users won’t buy the product unless it has a user-friendly, graphical interface. When the breakfast cooker is plugged in, users should see a cowboy boot on the screen. Users click on it, and the message “Booting Application Breakfast v1.2” appears on the screen. (Breakfast v1.2 should be out by the time the product gets to the market.) Users can pull down a menu and click on the foods they want to cook.”

“Having made the wise decision of specifying the software first in the design phase, all that remains is to pick an adequate hardware platform for the implementation phase. An Intel Pentium 1.86GHz with 1.2GB of memory, a 220GB hard disk, and a TFT monitor should be sufficient. Selecting a multitasking, object oriented language that supports multiple inheritance and has a built-in GUI, means writing the program will be a snap. (Imagine the difficulty we would have had if we had foolishly allowed a hardware-first design strategy to lock us into a four-bit microcontroller!).”

The king had the computer scientist thrown in the moat, and they all lived happily ever after.

NB. The story mentions a Java developer and thus provides a valuable insight: Java coders are lazy and provide specs they don’t have to develop when possible. This is why the multiple inheritance was suggested, instead of AOP.
Actually the story is way older than Java, and in those times developers actually knew languages and didn’t try to design everything so it can be solved in their language of choice.

As usual this was not a task that was planed in the project in advance, so we had to hack together a code that didn’t require too much programming. This is usually the case when one takes two strong coffees and switches to script writing mode.

The databases in question are DB2 8.x patched up to a version equivalent to DB2 9.x, the environment the script must run in was GNU Linux, Cygwin and possibly even AIX, I had about 4 hours to write the original version.

On the end I whipped up something like this.

After this script was done I just had to add the table specific invocations to another script and voila it was able to create the scripts we needed to create the deploy scripts.

The simplest case of invoking the function, when the tables are to be compared with all their fields.

compare_content $SRC_DB_ALIAS $SRC_DB_USER $SRC_DB_PASSWORD $DST_DB_ALIAS $DST_DB_USER $DST_DB_PASSWORD 'MY_SCHEMA' 'MY_TABLE' 'ID';

But the script is also good for cases, where you would like to ignore some fields for the comparison would like to conditionally compare the dbs (MY_FILTER IS NOT NULL), use composite alternate key (MY_ALTERNATE_KEY1,MY_ALTERNATE_KEY2,MY_ALTERNATE_KEY3) for identifying a row and want to use a synthetic key as primary key (NEXT VALUE FOR MY_COMPLEX_TABLE_ID_SEQUENCE) on inserts.

compare_content $SRC_DB_ALIAS $SRC_DB_USER $SRC_DB_PASSWORD $DST_DB_ALIAS $DST_DB_USER $DST_DB_PASSWORD
'MY_SCHEMA' 'MY_COMPLEX_TABLE' 'MY_ALTERNATE_KEY1,MY_ALTERNATE_KEY2,MY_ALTERNATE_KEY3' 'MY_FILTER IS NOT NULL' 'ID' 'NEXT VALUE FOR MY_COMPLEX_TABLE_ID_SEQUENCE';
For this the script will automatically create the required statements, where the updates will omit the field ID, the inserts will insert the next value for the sequence in ID, but all comparison will be performed among values that have the same alternate key.

I’m sure this is a script that has a lot to improve, but still this is a good example to show the power of script languages when it comes to hacking together a utility that saves plenty of time, and would be a real pain to create using conventional programming techniques.

If you replace the db2 commands with the equivalents of your favorite DBMS’s command line tool you might see that this can be easily adapted to your needs as well.

NB. There are some other aspects in the original script I didn’t include. Namely it’s able to compare the database structures of the DB’s using the meta data tables of DB2. Since these modifications are quite rare in our project and it’s hard to compare data for some cases, that part of the script is not finished.

First I suspected it was because I signed the application using SUN JDK 1.6, and it might have some compatibility issues (as if) then I suspected it was the IBM JDK 1.5′s ikeytool I used to create the key (NB. I’m lazy to learn the keytool paramers, so I prefer to use a GUI for creating keys)

It turned out I was presuming the keys use UTF internally, like most Java applications should, so when entering the locality I used “Vác” with accented characters. Neither ikeytool, neither keytool warned me about this. Once creating a new key without the accent the application started working straight away.

Since “Budapest” doesn’t have any special characters, I never noticed this, but I think it deserves to be mentioned, so others won’t run into this problem.

As the application currently uses web services to access the framework containing the business logic and this is already separated in the application it’s relatively straightforward to modify this dispatcher to access the business logic.

When online the modified application works just like the one above, and during the online period the application will synchronize its database with the server using DB2 Everyplace.

When the application is off-line the actions will perform the actions on the locally synchronized database, and send MQ messages of these actions through the local MQ Everyplace. Thus the application will see the local database as it’s own and perform the actions on it’s local database. The requirements allow for this, as the data is distributed among the users, (it’s more of a data collector, than business application) and there is only a very little chance of simultaneous updates.

When the client goes on-line again first the MQ messages will be sent to the server, the client will then wait for all the sent messages to be consumed by a message bean on the application server. After the messages are consumed the local database can be synchronized with the server so the data modified by other clients will be reflected on the client.

We’ve also considered using the features of the Lotus Expeditor platform, which works similarly, but decided against it. Our two main concerns were, that firstly we have limited resources and learning a new platform is time consuming, especially for such a new framework, the other was that Lotus Expeditor’s works offline like the original Lotus Notes platform (this is our assumption, based on a very short research) has the preconception, that noone is allowed to use the same database at the same time. If someone does the records are marked as save conflicts that must be reconciled manually.

Our way would permit the business logic to handle these updates gracefully, by either simply applying the updates on the given record in sequence, or refusing the update using either a version- or timestamp, or in workflow like cases statuses.

I’m sure this is not the only, and very likely not the best way to do this, but it’s most certainly the one that would require the smallest effort.

The images suck, I know, but I’m not in a drawing mood right now.

Hard to identify InvocationTarget exceptions
Our build environment was a Tomcat 6.0.x running on Sun JDK 1.6 on Debian Linux on x86, while the test and production environments are Websphere Application Servers 6.0.12 running the default IBM JDK 1.4.2 on AIX on Power5. Upon deploying to the test server we used the images created on the build environment using the Anthill’s build as version functions. Since the target source and target levels are Java 1.4, these options were specified in the javac ant task, so the code produced can run on the target environments.
This setup worked perfectly for months on our projects, but after a while strange exceptions started popping up on the test server. Compiling and deploying the application using the same options as the ones set in Anthill on our local Websphere servers got rid of the problems, while the one deployed on the AIX running the WAS we got the exceptions. Since there didn’t seem to be any difference between the builds we didn’t suspect Anthill for quite a time, and thought it was some strange JDK problem on AIX.
The simple code:
new BigDecimal(1);
worked perfectly on our Tomcat, on our local WAS but threw an exception on the test server’s WAS. Changing the code to
new BigDecimal("1");
worked perfectly on all environments.
A collegue worked out the reason after a few days of searching. He noticed, that these issues occur, when we use methods, that have been extended in Java 1.5. For example the code above would map to BigDecimal(int i) while it will map to BigDecimal(double d). We realized, that however the generated code will be created to be ran under Java 1.4, it will be compiled for the actual runtime of the ant ran from the Anthill. Since the JRE in WAS doesn’t contain the signatures for the given method, this will throw the InvocationTargetException, which looks strange at that point of the code.
After identifying the problem, solving it was quite easy after checking out the official documentation, changing the javac block to look something like:

destdir="${build}"
fork="yes"
executable="${java.home}/../bin/javac"
/>
and passing the option -Djava.home=/my/jdk/home/dir will enable to build the project both under Anthill and Eclipse.

Publishing under WAS
Our current project is to be deployed on WebSphere, and we wanted to use Anthill to deploy automatically on WAS. This is a real pain in the backside, since if you take FOSS servlet containers and application servers (Tomcat, Jetty, Jboss, Geronimo), they are pretty friendly when it comes to deploying an application on. You just copy the WAR/EAR in a directory and it is deployed automatically. On WAS hovewer this is not half as simple.
WAS has a very specific way of accessing the administration interface ws_ant. It first seems like a very simple shell script running ant scripts, but try as you might you’ll never be able to invoke it from inside your normal ant. Rumor has it, that IBM modified the ant for some weird reason. After several failed attempts, we used the most trivial and ugly way to invoke the tasks.
Based on the forums and even IBM’s InfoCenter it seems that invoking the ws_ant with the exec task with the required arguments. I have to mention two points here, one is that there seems to be no way to handle the return value of the exec. The other is that it’s quite annoying, that all arguments of the exec must be specified using separate args.