Five Java IDEs tested

We test and compare five of the most popular Java Integrated Development Environments (IDEs).

In order to successfully develop applications using OO languages such as C# and Java, one must use an IDE to compliment their skills, arsenal of books, bookmarked Web sites, and mailing list references. The diehard may turn up his nose at IDEs in favour of text editors such as Vi or Emacs but not even the father of Java, James Gosling (who also developed a version of Emacs), uses a text editor to cut Java code.

IDEs, when used correctly, can save a great deal of money, time and ultimately reduce technology risks. They can make the life of a developer far more pleasant by way of features such as refactoring engines, code optimisers, code support, powerful code navigation features, integrated debuggers, and more.

For this article we’ll examine Borland JBuilder 2005, IBM WebSphere Studio Application Developer 5.1.2 (WSAD), Netbeans 4.0, Eclipse 3.0.1, and Sun’s Studio Creator 4.26.

We tested the IDEs from the perspective of someone at the coal face of application development as well as from the perspective of application architecture or delivery management.

It should noted that there are other IDE platforms available. We’ve chosen these five IDEs for their popularity as well as overall capability but it’s not to say that other IDEs on the market are not worthy. IDEs not tested include IntelliJ IDEA and the sound BEA Weblogic Workshop. While we could not fit every Java IDE into this article we strongly encourage readers to test these products to judge the best fit tool for you and your organisation.

Review Methodology
It’s worth noting two important factors that need consideration beyond this review. They are, existing skills or access to IDE training and how well your existing technology mix interoperates and integrates with your chosen IDE to your Java/J2EE platform.

The six factors used for evaluating an IDE for its capability and robustness on a software project are shown in the table below.

The test workstation that was used to perform the comparisons is a notebook with a Pentium 4 3.06GHz processor, 1GB of memory and 5400RPM disk drive.

Here's a list of the review criteria:

Ease of use To start, we evaluated how easy each tool was to learn and use, including an evaluation of the design environment itself, placement of commonly used items, and the general overall impression of the tool.
Features and functionality How many valuable bells and whistles does the IDE provide? These include profiler support, debugger support, code completion, reflection engines, etc.
Performance We tested how fast it was to move around each IDE, and how long compilation took. Whilst this can be driven by the hardware, there are elements of an IDE which can slow this down. Poor IDE performance is not only frustrating but can cost money.
IDE Plug-in support Plug-in support is helpful if you want to build custom components for your software development project environment, or make use of third-party plug-ins available.
Licence and support costs The bottom line is obviously important, especially when a project consists of tens or hundreds of developers.
Standards support And finally, we look at the IDE in terms of incorporating Java standards. With a growing number of third-party application products on the market being distributed as JAR and EAR files, there is a need to ensure that all standards and specifications line up within your IDE.
Table 1: Evaluation criteria

Netbeans 4.0
Netbeans is the primary competitor in the open source arena to the Eclipse movement. Netbeans is a Sun initiative and has similar base features and look to the Sun Studio Creator tool. Sun released Netbeans, named after the company it was originally bought from in mid-2000.

Netbeans’ latest incarnation, version 4.0, is a great improvement over the previous 3.5 series. Whilst 3.5 was good and effective for use in Tomcat and JBoss application server development, it had several unusual UI or navigation nuances that made it difficult to get one’s head around.

Netbeans now has a Microsoft Visual Studio .NET feel to it, which many will applaud, and version 4 has overcome some of the quirky 3.5 navigation.

Netbeans is an impressive open source project with a big arsenal of features and capabilities. In fact, it offers more base Java development features than its commercial derivative, Sun Studio Creator.

The NetBeans IDE is fast. Code completion is often a benchmark for the responsiveness of an IDE and NetBeans has it. Whilst this is a tuneable attribute, (such as how long Netbeans takes to display the suggestion box) the test workstation’s hard disk doesn’t crunch away whilst preparing lookups.

Previous versions of Netbeans suffered from this. We also found build preparation, file management, configuration and menu drop downs very fast.

The NetBeans IDE has a clean and intuitive interface--and not overbearing with dialogue boxes.

One of the nice features of Netbeans is the easy access to resources, such as server configurations, database resource management and Web server settings. This is all available from the Resources tab on the main dialogue box.

Plug-in support for Netbeans exists but it’s hidden fairly deep within the Option menu. As we’ll see with some of the other IDEs, whilst Netbeans is capable of plug-ins, management of them is quite different to the likes of Eclipse and IBM’s WebSphere.

The licence costs of an open source tool such as Netbeans simply can’t be beaten as it is free. This does not take into account any aspect of the licence arrangement other than the per seat cost for the IDE.

Like the other open source IDE’s in this round up, community-based projects tend to be up with emerging standards more so than their commercial constituents. This can be both a good and bad thing, depending on what sort of project environment you are in.

For developers looking to use Netbeans, this shouldn’t be a major problem given the decision to use an open source or non commercially aligned application has been chosen already.

Overall, Netbeans scores a 25.5 out of 30. Eclipse 3.0.1 with MyEclipse
Eclipse is another great story of open source inertia and not surprising, WebSphere is to Eclipse as Netbeans is to Sun.

Eclipse is directed, but not exclusively run by IBM, with another 10 or so other major players who involve themselves with plug-ins and the architectural direction of the platform.

What makes Eclipse different to all the other IDEs is that Eclipse is not only an IDE for Java. It boasts platform derivatives for COBOL, C/C++, and Perl. Further to this, it has a 16 other sub-projects that cover all sorts of other development facets such as UML, GUI building, and the Eclipse Modelling Framework.

The Eclipse IDE has a completely different look and feel to the Netbeans and Sun Studio Creator platforms. This makes for some adjusting if you’re more familiar with Microsoft development environments (such as developing C# or J# with Visual Studio .NET) or the NetBeans/Sun Studio Creator tools.

The Eclipse version reviewed in this roundup makes use of a third-party plug-in called MyEclipse. The key reason for this is that Eclipse on its own does not natively support full J2EE development, specifically for Web application development.

MyEclipse provides Eclipse with a raft of features that understand XML, JSPs, Javascript and other Web-like languages available as generic support or as Eclipse perspectives.

Eclipse packs a punch when it comes to features. Eclipse, predominately through its plug-in support, provides a very integrated ability to enhance the core IDE with powerful features.

Eclipse is by far the leader of all the IDEs in terms of ability to add features. Eclipse also has the advantage with the MyEclipse layer which provides all the advanced editor features that many of the other commercial IDEs offer by default.

Eclipse is a performant application. Navigation, menu usage, code completion, refactoring, and building is fast. We found build preparation, file management, configuration and menu drop downs fast.

The interface is uncluttered. Confusion from the developer perspective can arise with the way Eclipse “virtualises” the project layout.

The IDE allows you to view both the file system view and the packaged view. So long as you can understand and appreciate the way in which the package and project is laid out then there should be no problems.

Eclipse is probably the leader in terms of plug-in support. It has the backing of a huge community and many Web sites exist which provide pages of plug-ins.

The installation of Eclipse in the test lab contains a number of plug-ins installed. These plug-ins in Eclipse are deployed by literally unpacking the plug-in’s zip file into the Eclipse “plug-ins” directory and restarting Eclipse.

What’s more, Eclipse has a plug-in developer framework which allows developers to easily code their our own plug-ins for Eclipse.

Like Netbeans, Eclipse is open source and does not require any licensing costs. That said, the instance of Eclipse used in this round up is deployed using the MyEclipse IDE plug-in to provide true J2EE features.

MyEclipse costs US$29.95 per annum which gives access to support, updates, tutorials and a lot more. To any organisation, this is a bargain.

Eclipse’s open source foundation means that developers are cutting code to new standards and Java sets on a daily basis.

The industry leaders who are helping to shape the direction of the Eclipse engine do help ensure that standards and quality are infact maintained. The core Eclipse IDE is robust and based on standards, and if you want to get access to more yet-to-be-proven Java standards and specifications, you can deploy plug-ins, such as MyEclipse or Lomboz, which allow you to pick and choose your specification version of choice.

The overall rating for Eclipse was 26.5 out of 30. WebSphere Studio Application Developer 5.1.2
WebSphere Studio Application Developer (WSAD) is the latest of IBM’s IDE platform offerings under the WebSphere branding. As this article went to print the next version, dubbed Rational Application Developer (RAD) 6 is the next platform version to be officially released.

WSAD 5.1.2 is a very powerful IDE. It provides a similar development experience of Eclipse with the MyEclipse IDE, incorporating all the facilities to develop J2EE applications.

What’s more, unlike the native Eclipse platform, WSAD 5.1.2 supports direct WebSphere deployment and WSAD comes with an internal WebSphere test server and direct support for Tomcat 4 and 5 Web application servers. This is the biggest advantage over Eclipse (or other IDEs) for WebSphere application server based development.

It also supports Apache Tomcat in a slightly more integrated way than Eclipse does. Whilst most development teams will use Ant to bind and package all their Java components together for integration and production development, nothing beats having a localised test environment which aids in rapid application development.

Overall, WSAD performs well. Earlier versions of WSAD (version 4) were slightly sluggish and developers could notice the extra horsepower requirements when using it on lower spec workstations. Build preparation, file management, configuration and menu drop downs were fast. Code completion, whilst fully tuneable in terms of responsiveness, is very fast. WSAD continuously is building and pre-packaging in the background which makes it suitable for test and deployment cycles.

WSAD is very similar in feel and operation to Eclipse. In fact, if you were to find yourself a copy of Eclipse 2.1, you’d not know the difference other than a few minor cosmetic changes.

Like its Eclipse constituent, WSAD supports plug-ins well. Alas, it only officially supports Eclipse plug-ins designed for Eclipse 2.1, but it still follows the same model of plug-in architecture to Eclipse, meaning you can write your own or find plug-ins based on Eclipse 2.1.

This can be a slight disadvantage to some trying to access some of the more recently developed plug-ins for Eclipse 3.0.

In the options menu within WSAD, a plethora of plug-ins are provided as standard to the IDE, with most areas covered by default.

As WSAD is a commercial product, there is are a big difference to that of the open source constituents. Pricing and support varies depending on the customer (eg: size, IBM partner level, number of seats/licences, etc), so we can’t give hard figures.

It is however a significant price increase to that of the zero cost open source equivalents, but with the accountability and support from IBM, as well as the knowledge that it’s a commercially tested platform. For business or mission critical projects conformance is an important aspect and WSAD follows standards closely. Having the knowledge that everything is based on standards and there are no quirky issues that may arise from early release specification components in your environment is a good thing.

The overall rating for WSAD 5.1.2 is 24.5 out of 30. Sun Studio Creator 4.26
Sun Studio Creator is essentially a commercialised flavour of Netbeans with some additional and proprietary features such as native support for Sun Java System Application Server version 8.

Out of the box, Sun Studio Creator has very good support for Web Services, and one can’t go past the fact that Studio Creator comes from the stables of the Java creators, Sun Microsystems.

Studio Creator boasts some very powerful drag and drop features based on JavaServer Faces (JFS) it boasts some impressive wizard capabilities. Literally within 30 seconds you can interface the output of a HTML table with the contents of a database table.

While Microsoft has had this functionality for a while, this is Java. Akin to Unix and Windows, Java has typically been more low level and server orientated than Visual Basic and other .NET languages. It’s a good thing that Java is finally bridging this capability gap.

Whilst the IDE doesn’t offer as many nice Java-coding features as some of the other IDEs, Sun’s approach is to provide powerful drag and drop, WYSIWYG-like development capabilities.

The tool is very much geared around client development. Unlike most of the other IDEs we’ve looked at, Sun Studio Creator places a lot of emphasis on the ability to develop Client side (eg: Desktop) as well as server side applications.

If you’re looking to get up and running with a Web application fast and JSF suits your environment, this tool is quite powerful.

Overall the IDE is fast. Build times and access to features through drop down menus are also fast. Much of the IDE hangs together via modules, not unlike Netbeans. These are all loaded at start-up time which makes it fast to use and quick to respond.

The user interface layer of the tool, whilst loosely based on Netbeans, has a number of other usability features based around palettes. For developers who use tools such as Visio and graphics applications you will be familiar with this sort of UI approach.

On the left-hand side of the tool you can see the large buttons that, when pressed, expand and open up to show other options. In this part of the UI, you are given example code snippets, and drag and drop functions.

It’s fast, intuitive and Sun has really tried to make it easier for developers to get up and running quickly with lots of pre-written, drag-and-drop example code elements included. For new or inexperienced Java developers, this IDE is a great introduction to Java.

Plug-ins, or modules, within Sun Studio Creator are available yet there is no large community movement behind them as yet.

There is the capacity to add and remove them, but predominately, the tools and features you’d typically want from an IDE are 90-95 percent there already.

Whilst the plug-in support for this IDE is lower than that of the others, Sun Studio Creator has a lot of good modules already included into the core functions of the IDE.

Like WSAD, pricing for Sun Studio Creator is dependant on your organisation’s relationship with the vendor. You may be eligible for discounts or enterprise licensing arrangements.

Being written and developed by Sun means that this tool is absolutely standards based. This is a great thing for critical development engagements, similar to that of the IBM WSAD standards support. Overall, if you’re looking to deploy to Sun based Web and J2EE application servers, then this tool is a solid fit.

Overall the rating for Sun Studio Creator is 24.5 out of 30. Borland JBuilder 2005
Borland has a long history of developing solid developer tools and JBuilder follows in good stead as a quality and robust platform for Java development.

Although Borland offers the Borland Enterprise Server as an alternative J2EE Application Server platform, JBuilder comes readied with internal Tomcat Web Application Servers. This is similar to the Netbeans and WSAD IDE platforms which offer out-of-the-box Tomcat support.

This is good news for small, medium, and the growing number of large/enterprise development shops that are using Tomcat for business-critical application platforms.

JBuilder is the odd man out in this round up. It’s a completely independent IDE; one of the reasons we decided to include it.

Aside for that, it’s also very powerful. Not being aligned to Eclipse or Netbeans movements, or IBM and Sun for that matter, means that its evolution has taken place independently and devoid of any outside influence.

The whole UI experience is different and we found it very intuitive. Like other tools in the same market as JBuilder, Borland has had to think hard about how it could make its IDE better; how it could differentiate itself from the rest. Following in the legacy footsteps of tools such as Borland Turbo C, et al, JBuilder is a powerful and feature rich IDE.

Overall the IDE is quite fast. With most IDEs, the general movement around the UI into menus and option fields is fast due to the advancements of the new JDKs. Sun JDK 1.4 and the new Tiger release has come along way since 1.3 and lots of performance improvements make IDEs such as JBuilder fast. Like most IDEs, JBuilder has different window frames which allows you to setup your workspace how you see best.

This approach is also great for those lucky enough to develop using multi-head workstations where outputs warnings, and dialog boxes will appear on one monitor, leaving the primary monitor for full screen code manipulation or modelling.

The interface looks to be a fairly standard layout. The power of JBuilder is when you start to access the modelling or designer features that the tool has inbuilt.

JBuilder comes with most, if not all, of the features you’d want from a tool. JBuilder does support plug-ins but typically the additional components you would want to include are the advanced capabilities from Borland’s other product offerings like Round Trip Engineering, distributed paired-programming capabilities.

Like the other commercial IDEs, pricing is dependent on your organisation’s relationship with Borland and your specific needs and situation. Borland offers packaging arrangements where you can access discounts through purchasing other products such as Collaborator and so forth.

JBuilder ensures that you are developing with official Java standards. It’s partly due to the commercial nature of the tools developer, and partly due to the heritage of Borland. For years now Borland has developed some of the best development platforms and toolsets available on the market. Their reputation goes without saying and as such, you can be assured that the quality of the product is second to none.

Overall the rating for Borland JBuilder is 25.5 out of 30. Conclusions
The final ratings for each of the IDE tools is in the table below. Whilst these are definitely interesting ratings (note, ratings not rankings), we can say that from our tests we found each of them powerful and can definitely do the job in the right development situation.

The selection of IDEs however, should help developers choose between IDEs based on their preferences for things such as plug-ins, or development/coding features, licensing costs and potentially performance.

What’s not shown here is the positive impact that the Sun and IBM commercial IDEs can have on a development environment where Sun Java Enterprise application server and IBM WebSphere application servers respectively, are deployed.

Choosing WSAD in an IBM WebSphere environment over Eclipse 3.01 is a sound choice given that WSAD will reduce technology risks and speed development time greatly through its tighter integration with the WebSphere application server platform.

Whilst not tested, this is also the case for BEA’s Weblogic application server with the Weblogic Workshop IDE.

IDEs have a lot to do with personal preference. There are so many levers and variables in this selection process yet, hopefully this round up has given some background and a better understanding of alternative IDEs that are available.

IDE Overall Rating
Netbeans 4.0 25.5
Eclipse 3.01 with MyEclipseIDE 26.5
IBM WebSphere Studio Application Developer 5.1.2 24.5
Sun Studio Creator 4.26 24.5
Borland JBuilder 2005 25.5
Table 2: Final ratings