Sunday, July 14, 2013

Java 7 vs Groovy 2.1 Performance Comparison

I haven't used Groovy for 2 years, since my last touch with Grails. I get stuck in (hard)core Enterprise Java, with some performance aspects in background. I've almost missed a chance to learn Spock, but fortunately Warsaw Java User Group helped me to snap out of some legacy systems and back to normal self-development. In fact I hope that frameworks such as Spock or Geb will change approach to writing tests, by making them easier and more effective. Both frameworks use Groovy, as well as the new king in build tools - Gradle. Seeing pace how Groovy impacts our daily routines I decided to look closer at its performance, and compare it to Java 7.

My test environment is based on Java 1.7.0_25 and Groovy 2.1.6. As always in such comparisons I used Caliper in version 1.0-beta-1 (almost stable) and prepared a number of (I hope) representative microbenchmarks.

First benchmark based on Fork/Join framework should be most similar in both languages, because it uses some native mechanisms. My test initialize array with some random int data, and then use framework to find biggest element in array. In Groovy my compute functions looks like below:
@Override
Integer compute() {
  def size = end - start
  if (size == 1) {
    Math.max(array[start], array[end])
  } else {
    int diff = size / 2
    MaxValueSeeker left = 
      new MaxValueSeeker(array, start, start + diff)
    left.fork()
    MaxValueSeeker right = 
      new MaxValueSeeker(array, start + diff, end)
    Math.max(right.compute(), left.join())
  }
}
Java version is of course very similar. After dozen minutes of measuring I get very promising result: Groovy is slower only... 8 times :)

Now it's time to check some more realistic in everyday development. I choosed simple POJO/POGO (yeah) with few simple operations just to be sure, that JIT won't eliminate my code (and belive me he loves doing such jokes). My pseudo "business logic" method in groovy:
def int proceed(int reps) {
  List<GroovyPojo> list = new ArrayList<>()
  int sum = 0;
  reps.times {
    // first param is int and second is String
    list.add(new GroovyPojo(value: it, stringValue: it))
  }
  list.each {
    if (Integer.parseInt(it.stringValue) == it.value) {
      sum += it.value
    }
  }
  sum
}
Java version differs mostly by getters and manual String boxing in POJO constructor. One more time dozen minutes spent on reading news and... this time Groovy is slower only 7 times

The last test should be stressful and check both languages in more complex computations. I made up my mind and chose quicksort algorithm. Few loops, few if statements should work. I'm not going to copy-paste it here, because it's well known solution.  What is worth to mention is of course timing result, outdistancing Groovy almost 5 times! But I've done some googling and noticed that Groovy 2.0 introduced @CompileStatic annotation, which should give us some additional performance boost. So lets check... Yes, with static compilation Java advantage fell to 220%.

In the table below you can find detailed results. To sum up - I'm not sure that using Groovy in mission critical functions is a good idea, but definitely it's great solution for implementing tests, prototyping, etc. Just let me highlight, that writing Caliper's results parser taken about 6 lines in Groovy (parse json, iterate over measurements and count average)

MethodJava [ns]Groovy [ns]Factor
Fork/Join22.132181.0188.18
Pojos117.914856.3377.26
Quicksort68.728330.1594.80
Quicksort with @CompileStatic67.752147.7922.18
Performance comparison

7 comments:

Vince P. said...

Here's the thing - yes Groovy is several times slower than Java as you have noted. However, if you opt to use Python or Ruby, the difference is 20-50x greater depending on which one of those you choose, the version, and the framework you choose. And, let's face it, those get used in high traffic production situations every day. Obviously they can work regardless of that fact.

So, Groovy, or any other non-Java JVM dynamic language for that matter (e.g. Scala) are not going to be the worst choice by far. In fact, they're not bad at all.

As always, we have to choose carefully based on the factors most important to us, but I have yet to see an environment where performance was the most important factor and they didn't just use C or assembler. Why mess around with a VM at all then? Premature optimization isn't compatible with a choice such as Java, because if performance is your first concern, then a VM based language is never the answer.

I would like to suggest that a better reason to choose Groovy over plain Java is community acceptance. The ability to maintain code years after it has been written is far more important than today's speed benchmarks. The jury is still out, but Groovy is promising in that respect; even though it's not my first choice personally.

emszpak said...

Just to be sure - you haven't mentioned it clearly. Did you setup your build tool to enable indy optimization (in additional to use the -indy.jar)?

Jakub Kubryński said...

No, I haven't used -indy. It makes a lot problems for JIT and my experience says it's even few times slower than normal compilation.

Eleftheria K. said...

Hello Mr Kubrynski,

Nice blog! I would like to discuss a partnership opportunity between our sites. Is there an email address I can contact you in private?

Thanks,
Eleftheria Kiourtzoglou

Head of Editorial Team
Java Code Geeks
email: [dot][at]javacodegeeks[dot]com

Corvin said...

There is always trade off, dynamic groovy creates additional opportunities to create software. And I don't think if target of groovy is be quicker at some algorithms. It beats Java at several different fields. btw Have you compared it with @CompileStatic annotation?

Sean Mitchell said...

Interesting results, thanks!


"I'm not sure that using Groovy in mission critical functions is a good idea"


I think would be more appropriate to say "performance critical". No reason not to use Groovy in mission critical functions, so long as the performance you get meets your requirements.

In fact, the focus on raw performance tends to neglect the fact that it's often cheaper to scale out on slower but more productive technologies.

Cheers,

Sean

Jakub Kubryński said...

Right Sean - performance critical is more appropriate term :) In fact time2market with Groovy is much faster when we compare it to Java, so in fact it could be more suitable for mission critical tasks