New Relic, RedMonk and surveys

Vendors and research firms often collaborate to survey the vendors' customers in the hopes of learning more about their requirements and purchasing behavior. Although the result is often based upon a biased or limited sample, the resulting study findings are trumpeted to the world as if they were broadly applicable. In truth, the findings are really only useful in understanding portions of, or perhaps, all of that single vendor's customers. New Relic and RedMonk released just a study. What's really refreshing is that New Relic and RedMonk took pains to present where and how this data is useful.

Here's what New Relic had to say about the study

New Relic and RedMonk evaluated technologies used in association with Java, Ruby, PHP and Python. Key findings included:

• Open source dominates Java, PHP and Python: over 80 percent of Java developers, 99 percent of PHP developers, and 99 percent of Python developers use Linux OS.
• 75 percent of Java developers deploy open source web application servers, with Apache Tomcat leading in market share.
• PostgreSQL is the most common database among Ruby and Python developers, while MySQL dominates in PHP with 83 more popularity than Redis.
• In Ruby application frameworks, Rails is 84 percent more popular than Sinatra. In PHP application frameworks, Drupal is 86 percent more popular than WordPress.

Since April 2009, New Relic has published a summary of the most commonly used versions of Ruby and other programming languages. To view the results of the study, visit http://blog.newrelic.com/?p=11079.

Snapshot analysis

New Relic provides its customers with extensive capabilities to monitor users, applications, and, to some extent,  the servers supporting Ruby, Java, PHP and Phython-based workloads. The data coming from this real-world workload monitoring is the foundation for the RedMonk report.

I was very impressed that Stephen O'Grady, RedMonk analyst, demonstrated the best of research principles by pointing out how the data was collected, analyzed and reported. He went on to point out the limits to the applicability of the research in question. Although the sample is quite large, it is focused solely and only on users of New Relic.  Here's what Stephen had to say about the data set RedMonk analyzed:

"RedMonk is not asserting that New Relic’s data is statistically representative of the market as a whole, or even of the specific communities surveyed. It reflects strictly New Relic users. That said, because the New Relic community represents tens of thousands of nodes monitored which collectively generate billions of transactions daily, we do believe the dataset represents a rare opportunity for direct observation of adoption and usage patterns within one very sizable community.

The results are very interesting and would be useful reading for those using Ruby, Java, PHP and Phython.