May 30 2012

A short and clean way of computing MD5 in Java

There are multiple articles (01, 02, 03, 04) talking about how to compute MD5 in Java and some of them have some gotchas regarding the length of the result. Here is a small code snippet that works OOTB:


Aug 8 2011

Why I give up using SmartGWT

I guess that most of you agree with me that UI is one of the most painful aspects of building Web applications (at least if you compare it with backend development). Well if you are building a Web application using GWT you may be tempted to use SmartGWT as it provides really cool widgets. Well, that’s true, their widgets are really cool and the themes are great. However, I would like to explain my experience using SmartGWT for a personal project (a first version of the app is shown below).

First version using SmartGWT

My application is pretty basic, just a couple of forms, windows, tabs and toolbars; nothing more crazy than that. The main intent of the application is to allow people to take personal notes. The development went pretty well the first 2 weeks while I was working with the basic stuff; then I realize that I need to use a simple autocomplete field and though SmartGWT doesn’t provide an Autocomplete field OOTB I tried to use a custom solution using the ComboBoxItem; a really bad idea…

First, ComboBoxItem doesn’t work as a field in Calendars so big problem; lucky I found a workaround to fix the issue. Then I tried to make the ComboBoxItem’s datasource work with a remote GWT call though SmartGWT doesn’t provide a OOTB datasource that works with the GWT paradigm. Again, a big mess digging into the javascript code where SmartGWT is implemented

Ok, so let’s try to fix the autocomplete problem using the SuggestBox that GWT provides OOTB. Well, another bad idea as SmartGWT randomly works with GWT and they clearly mentioned in their docs that it is not recommended to mix both.

Finally let’s try to extend the framework to make it work in either case. Well, here is the main issue I found with SmartGwt: the GWT UI classes that they provide delegate most of their behavior to plain javascript that is inside the JAR file. Oh my God, how do you expect developers to extend your framework?

The answer may be: either open the jar and do it yourself or move to Smart GWT Pro (license based). Open the jar or extending at the javascript level lowers the level of abstraction and that’s something I don’t want and the main reason I’m using GWT and not jQuery.

I’m my case it is ridiculous paying for a license and I must say that I’m used to the business model that other open source products have (e.g. Mule, Spring, Hibernate) that you pay for support though you are able to extend the framework easily.

A small suggestion from a disappointed engineer: you have one of the best GWT UI libraries in the market that look perfect and your widgets are really cool, don’t screw it up with this mix which is NOT easy to extend.

At the end, I tried for a couple of days to implement a workaround, the solution was not stable enough to be used in a real application. I end up implementing a basic window and toolbar in plain GWT using GWT Drag and Drop and reuse some of the SmartGWT styling. The results are shown below; now I have an extensible solution that I can fix in case of problems :) but with the overhead of having to fix it myself :( .

Custom solution with Gwt drag and drop


Jan 4 2011

Input streams aren’t lazy enough?

Motivation

While I was working in the refactoring of the PGP module of Mule, I faced the problem that input streams weren’t lazy enough for me. Let me introduce the scenario; for those of you who are not familiar with Mule, Mule is a lightweight Java-based enterprise service bus (ESB) and integration platform that allows developers to connect applications together quickly and easily, enabling them to exchange data. Data is carried in messages that contain a payload (data, e.g. the contents of a file) and can be transformed and written to an endpoint (a different directory, post the information to a URL, etc).

In many situations the payload of the messages could be huge enough and may not fit into memory. To avoid this problem, Mule allows the developer to configure the messages in streaming mode. This works fine for small streams and for big streams that don’t need to be transformed. However when we need to transform those streams without having to write a temporal copy to disk, we get an OutOfMemoryException exception as Mule tries to load the contents into memory to perform the transformation. That was my scenario, I need to encrypt big files but without having to write temporary files to disk. A big picture of the scenario is shown in Fig.1 where you can see that files in the form of messages are polled from a directory, move through a flow and occasionally encrypted and written to disk.

Scenario

Fig. 1. Scenario

Proposal

The main idea of the solution is to have an intermediate object that wraps the original input stream and will start the encryption process only when other object needs to read something from the stream. I called this object LazyTransformedInputStream which represents an input stream that is going to be lazy transformed. The first proposal was configuring this object with a transformation that will perform the encryption. The idea of having a transformation is because we would like to have lazy decryptions or any other lazy transformation (e.g. compression). Fig. 2 shows the first sketch of the classes:

First sketch of classes

Fig. 2. First sketch of classes

Everything was perfect in theory but in practice I had some problems; when I tried the solution using the Bouncy castle PGP implementation I realized that on the decryption case it was not going to work. 100 encrypted bytes may represent a fewer number of decrypted bytes (e.g. 30 bytes) and features like encryption + compression may required more bytes to be read. On the other hand, a simple transformation like increasing by 1 the byte read will produce 1 byte and therefore will be easier to implement. In general, this scenario may happened on more cases like compression; therefore I modeled both scenarios as different TransformPolicy subclasses. Each policy works different regarding how many bytes are transformed:

  • TransformPerRequestPolicy: will transform only the bytes requested.
  • TransformPerRequestInChunksPolicy: will transform only the bytes requested but in chunks. For example if we configure this policy with a chunk size of 10 bytes and some object requests 2 bytes, the policy will force to transform 8 bytes more (or until the end of the stream).
  • TransformContinuouslyPolicy: will transform all the bytes without waiting them to be requested (decryption case).

You can see a summary of the classes in Fig. 3 and you can check the behavior of the classes in LazyTransformedInputStreamTestCase test case.

Final version of the classes

Fig. 3. Final version of the classes

Technical aspects

Last but not least, we have to mention a small but important thing about how LazyTransformedInputStream works. On the one hand, there is an object the reads from the transformed stream and needs to be delayed when the information has not been transformed. On the other hand we have the transformation process that should run on a different thread. Trying to execute both processes in the same thread will probably cause deadlock.

Internally, each TransformPolicy has a thread that is the one who executes the transformation code. We reuse the existing PipedInputStream and PipedOutputStream of java.io for this task. The object who wants to read the transformed stream will read from the PipedInputStream and the transformation will output the transformed bytes into a PipedOutputStream instance. In addition, the transforming thread needs to be kept alive till the object that consumes the PipedInputStream does not want more bytes so that the pipe doesn’t break. In our implementation, we terminate the thread either when the LazyTransformedInputStream is closed or when the LazyTransformedInputStream is finalized.

We did a memory profiling of the implementation with great results in the PGP module and can be found here.

Hope you enjoy the post and looking forward to hearing your comments!!!