One of my pet peeves in programming is that few people use guard clauses. A guard clause is an almost trivial concept that greatly improves readability. Inside a method, handle your special cases right away and return immediately.

Have a look at the following example:

private int doSomething() {
    if (everythingIsGood()) {

        /*
         * Lots and lots of code here, but that's a different story.
         */

        return SOME_VALUE;
    } else {
        return ANOTHER_VALUE;  // a special case
    }
}

You can easily rewrite it using the Replace Nested Conditional with Guard Clauses refactoring from Martin Fowler’s Refactoring:

private int doSomething() {
    if (! everythingIsGood()) // <-- this is your guard clause
        return ANOTHER_VALUE;

    /*
     * Lots and lots of code here, but that's a different story.
     */

    return SOME_VALUE;
}

Once you’ve read past the conditional(s) at the beginning of the method, you know that your world is in order and you don’t have to worry about special cases anymore.

I love unit tests and use them whenever possible. But in the end, there’s no substitute for a full scale system test. Only after system testing, you can be sure that everything works as intended. When I was looking for a way to test a REST-style JSON web service, I decided to create a small testing framework to simplify the task.

The result is the JSysTest framework written in Java. Using JSysTest, you can create functional web service tests based on JUnit 4 and its Hamcrest-based assertion DSL. With the help of some custom matchers, a test can look as concise as this:

  assertThat(get("/some-resource"), has(status(404)));

Have a look at the Documentation for a few more examples.

The framework is still in its early stages of development and is likely to change. I still don’t know how to make HTTP POST and PUT methods as elegant as the GET case, but I’m sure a pattern will emerge at some point.

I think the most important lesson to take away from this little experiment is that you can simplify your tests significantly if you invest a bit of time for building a highly specialized DSL.

Exciting news for Java developers: The first 1.0 release candidate of Google Collections has been released. For almost a year I’ve been waiting for this, and now it seems the waiting will soon be over.

Basically, Google Collections is a better, more consistent version of commons-collections with generics and a few cool things that let you almost use a functional style in Java (at least as far as this is possible). I think this library will quickly become a standard dependency in my Java projects.

Configuring servers is tedious work if you operate a cluster of more than a few machines (if you’re a regular reader, you’ve heard about it). I’ve created a simple deployment framework that helps with building deployment packages based on central configuration templates. It is ant-based, so people from the Java world should have no problem to adjust it to their needs. Read on to find out how it works.

The Server Deployment Framework (SDF)

Based on a server template, a bit of configuration and your application, this little framework creates a self-contained deployment package ready for installation. The package contains both the server itself and the application, so you always start from a clean slate. There’s no cruft left between installations.

With SDF, there’s a separation of responsibilities between developers and admins. Developers provide the application in a .war file. The .war file has no environment-specific configuration. Admins are responsible for configuring the application for the target environment. The general ideas is that the same application artifact that is handed to the QA department is also installed on the production systems.

This is how it works:

1. Create a server template in the src directory. Replace important configuration values with wildcards. Rename all files containing wildcards to filename.in (think autoconf).
2. Place your web applications in the base directory’s webapps directory.
3. Create/edit the environment configuration in the conf directory, ie. ENV-NAME.properties.
4. Run ant package -Dapp.environment=ENV-NAME.
5. You’ll find your pre-configured deployment package in the target directory.

Repeat steps 2-5 any time you want to make a new release.

The Bundled Tomcat

A few words on the bundled Tomcat instance: It is for illustration only and hasn’t been configured for production use. There are a couple of interesting ideas though.

All ports settings have a @PORT_PREFIX@ wildcard. That way you can run multiple Tomcat instances on one host by changing the prefix parameter. This is useful for rollouts: You can start up and check the new revision while the old one is still running. Using loadbalancer settings, you can switch over to the new application and later shut down the old revision.

In conf/context.xml, an example context parameter has been set. This is a simple way of passing environment-specific parameters to your application without having to change anything inside the .war file.

As a general rule, it’s a good idea to run just one web application per Tomcat instance. This way, it’s easier to figure out which application is misbehaving. In large operations with high performance requirements, this is the normal case anyway. When you follow the single application rule, you can as well turn off hot deployment.

Since applications tend to crash sometimes (OutOfMemory exceptions happen to the best of us), it makes sense to restart the server automatically without admin intervention. One easy solution to do this is to run Tomcat inside a JavaServiceWrapper that makes sure your JVM is in a healthy state. Remote restarts and easy JMX access are helpful additional features.

I’ve been reading up on distributed systems again. For quite a while, my monthly copy of CACM has been my only connection to computer science topics. This time, I followed a few references and came across interesting concepts (most of them familiar from back in university).

Amazon doesn’t just sell books over the internet, they’re building fascinating systems, too. And sometimes they write articles and papers about it, like in January’s edition of CACM (see Eventually consistent). If you’re familiar with Eric Brewer’s CAP theorem, you know that in a distributed system, you can decide for only two design goals out of consistency, availability and partition resilience. In traditional systems, consistency is highly valued, but if you’re willing to trade in consistency for availability, you can build high-performance systems.

One of these systems is Dynamo, a highly available, distributed key-value store that is part of Amazon’s internal infrastructure. Dynamo is write-optimized, writes are almost always possible, even if some replicas aren’t updated. Inconsistencies are resolved during read, typically by business logic inside the application itself.

Dynamo uses Consistent Hashing, which is used in distributed hash tables (DHTs) like Chord. This way, nodes can join and leave the network without having to remap the whole key space. As a friend succinctly put it “Yes, in a non-hostile environment, DHTs really work. They’re not only useful for distributing cam-rips on the Internet”. If you’re ever going to build a cache consisting of more than one caching server, Consistent Hashing is definitely something to check out.

Damn, I miss building stuff like this! It’s a shame that only large operations really need this kind of infrastructure.

In my projects, I’ve always been the one who took care of infrastructure, standardization and quality assurance from the development perspective. The funny thing is that I’m no admin and no QA guy, so most of it wasn’t even my job. In this article, I’m going to list a few things that in my opinion as a software developer are essential to a professional software project.

What you need is no secret: If you read a few books and follow some technology blogs you know the bits and pieces. I’ll list a few things from a Java/Maven perspective, so some of this may or may not apply to other platforms.

First of all define a common coding style. I shouldn’t even have to mention this, but there will be chaos and conflict among developers if you don’t have it. Just use Sun’s Code Conventions, make some exceptions like “maximum line length is 120 characters” or “no tabs allowed” and you’re halfway done. Provide a bit of example code (one class, one page of paper) and put it up on a wall. This style guide should include rules for things like logging and exception handling strategies as well. Too few developers are even aware that you need a strategy here, so write it down! Don’t forget to define package namespaces for your project.

For Java projects provide organization-wide Maven archetypes that give you a solid basis for your project. Use existing open source ones to get you started! The effort for this pays off with each new project in reduced setup costs. But wait, each of your projects is different? Oh please, then adjust your build scripts. That’s no excuse to start from zero each time!

On a related note, invest in a proper internal Maven repository setup. Maybe use a stripped down version of Atlassian’s setup. You don’t want your builds to break just because some external repository isn’t available. Define rules who may deploy what to your repository. I’ve seen a lot of chaos here, so remind people to actually think before deploying stuff and breaking builds.

Get yourself a proper bug tracker and define how you’re going to use it. It doesn’t have to be fancy, a Trac installation will do in most cases and it will give you a wiki for your technical documentation, too. Do you just track issues or also tasks for developers? What’s the workflow? Who may open a ticket, who may close it? What amount of testing is required?

For god’s sake, define rules for working with your revision control system. How often should developers check in? How should a commit message look? How does your release process look like? When do you create a branch, how are releases tagged etc. Read up on the features of your system, branching may no longer hurt. Move on if you’re still using CVS.

A professional software project needs continuous integration. Period. Set it up (I recommend Hudson), define the proper reports and actually read them. It’s crucial to check test coverage and other metrics right from the beginning. If you add reports later, you will typically get warnings for every other line of source code. Nobody will read those reports anymore and it’s too much work to clean up the code.

Provide developers with a proper workstation setup that corresponds to the platform you’re targeting. Yes, it’s a good idea if everyone on the project used the same JVM and application server versions. If that’s what your production environment uses, even better! It might improve the chance that things actually work. Just saying.

I’m barely scratching the surface here, but it’s enough for one article already. For many of the points above you can find articles on the web (check my blog’s best practices category, for example). And still, it’s frustrating how little many software companies invest in those bare essentials.

I’m back to playing with the Django web development framework again. Since I’m close to putting something online (it’s only a matter of years, actually), I’m also looking into hosting options. Due to the framework choice, Google App Engine appeared on my radar and I had a quick look at it.

Running your web application on Google’s infrastructure is certainly an intriguing idea. The official into video didn’t tell me too much, so I watched Guido van Rossum’s talk Rapid Development with Python, Django and Google App Engine on YouTube. Based on the information given there I decided against App Engine.

App Engine uses Google’s high performance distributed database system Bigtable. That’s pretty cool, but of course it works a lot different than a relational database. I’d have to port my application to Google’s persistence layer (not that much work yet) but I’d later be tied to a single hosting provider.

With the Django ORM framework out of the way, Django’s built-in generic admin site no longer works. This means, one of the framework’s biggest assets is lost and that’s the real show stopper for me.

If I were to create a highly scalable web application, the advantages would probably outweigh the disadvantages. But in my case, it means going back to comparing classic web hosters.

Most wiki-based documentation I have worked with has sooner or later turned into a tangled mess of outdated information. Some pages are maintained actively while others aren’t updated anymore and you never know how reliable the information still is. For a while I’ve been thinking about a simple technical solution to this problem. Here it is.

My general assumption is that new information is more reliable than old information. That means, a page that has been changed yesterday has a higher chance of being relevant than a page last changed five years ago. The idea isn’t particularly new: It’s a bit similar to what Eclipse Mylyn does for Java developers.

A wiki is a graph consisting of pages connected via links. Using age information, you could highlight the subgraph that is most active in respect to changes. Links that lead to newer pages are displayed in a lighter color while links to older pages are displayed in darker colors. Page titles could be color-coded in the same way. Users always know if they’re in the active center of a wiki or in a rarely updated, possibly outdated area. You can always push pages into the center by making a trivial edit.

Taking the idea one step further, with a little bit more effort a rating system could be implemented. The system could take implicit parameters like age information or page impressions and explicit parameters like user ratings into account. For each page, a score could be calculated that is used for color-coding of links and page titles.

There are many ways to store session data in web applications. They all differ in scalability, failover capabilities, and complexity. I’ll give you a quick rundown on the major themes.

Session Data on the Client

You can often implement simple personalization features or workflows by storing state on the client. From a scalability point of view, it doesn’t get any better than that. Not having to keep state on your servers saves you from a lot of trouble.

Cookies are a well-understood mechanism that can even be used in a client-only manner using JavaScript. They are useful for small portions of data that aren’t security relevant. Informal polls or simple preferences like language selections are often implemented via cookies. Cookies also play a major supporting role in user tracking and identification, but that’s a different story.

Some frameworks save workflow state in hidden form fields (see Apache MyFaces). That data can be encrypted by the server to prevent tampering.

Session Data on Your (Web-)Servers

For many developers, saving sessions on the front server (which is often a web server) is the most natural choice. That’s what Java Servlet implementations usually do.

As soon as you need more than one front server, things can become quite tricky. Your environment may support some kind of clustering that provides session replication and failover. No matter which front server gets a request, the session data is already there or will be requested on demand. Depending on the implementation, a failing front server doesn’t necessarily mean a loss of session data. Apache Tomcat has this feature, as do commercial products. As powerful as this may sound, the approach comes with a rather large price in complexity.

Fortunately, there’s an easier way: sticky sessions. Your loadbalancer (a HTTP reverse proxy) assigns a cookie to each client on its first request. The cookie determines which front server subsequent client requests are directed to. That means, each client is always directed to the same front server, so session data for a client is only needed on one front server. As a result, you don’t need replication anymore.

The sticky sessions approach is easy to use but it has two disadvantages: First, if one server crashes, you will lose part of your session data. And second, load balancing may no longer be optimal because it is done on a session basis and no longer on a request basis.

In the Java EE world, session state could also be stored inside the application server (as opposed to the Servlet container). For example, the Seam framework stored conversational data using stateful session beans at some point. I don’t know if this is still common practice.

Session Data Inside Your Database

Saving session state inside the database is common in lightweight web frameworks like Django. That way you can add as many front servers as you like without having to worry about session replication and other difficult stuff. You don’t tie yourself to a certain web server and you get persistence and all other features databases provide for free. As far as I can tell, this works rather nicely for small to medium size websites.

The problem is the usual: The database server may become your bottleneck. In that case your best bet may be to take a suitcase full of money to Oracle or IBM and buy yourself a database cluster.

Using a Dedicated Session Store

Sometimes, especially for high traffic web sites, it makes sense to store session data on a dedicated server or cluster. This takes complexity out of your web servers at the cost of an increased overall system complexity. No matter what the SOA guys say, distributing your data usually won’t make things any easier.

If, for example, you’re currently storing sessions inside your application’s database, you could move the session store part to its own database. That’s about as simple as it gets.

I’m not aware of any off the shelf products, but since storing session data is typically not the most difficult problem, you could try to roll your own based on a caching product (memcached comes to mind, as do at least a dozen Java caching solutions).

If you’re particularly ambitious (traffic-wise), using a data grid like Oracle Coherence is a solution to consider. They come with everything you need to implement a high-performance distributed session store. And then some.

Conclusion

Use cookies for small pieces of possibly long-term data where security is not an issue. Cookies can also complement other approaches.

Otherwise, use whatever your web development framework offers. Sticky sessions are a powerful but simple solution when you’re starting to scale out your system. I wouldn’t turn to session replication if I could get away without it.

If things get rough, a dedicated session server or cluster may be your last chance. But then you shouldn’t be needing my advice anyway.

For further information see Martin Fowler’s Patterns of Enterprise Application Architecture. It has a short section on session state patterns that covers client, server, and database session state.

2008 is done and over with. Before moving on I’ll follow David’s example and post my most popular articles in terms of page views.

Here they are:

1. The State of Java Build Systems
2. Generating DDL Scripts with Maven
3. Generating DDL Scripts from JPA Annotations
4. Parsing Command Line Options in Shell Scripts
5. Controlling Rhythmbox using D-Bus

What’s interesting here is that most of them are HOWTO style articles that got their traffic via search engines and from a few high profile referring sites. The top article is an exception: it’s a rant about build systems for Java that obviously struck a nerve. People submitted it to Reddit and DZone multiple times. It’s a bit sad though that none of my own favorites made it into top 5.

Anyway, this concludes my second year as a blogger. It’s been fun and there’s still a lot of topics I’d like to cover. Thank you for reading and have a happy new year!