Butterfly Container Performance

Jakob Jenkov
Last update: 2016-05-20

Performance is often less relevant when wiring components together, since the wiring often only happen once, at startup time, or infrequently, like at the start of the processing of an HTTP request. Even when you use the DI container to wire up complex components repeatedly at later runtime, the instantiation and wiring time of those components is often only a fraction of the time these components take to execute their business logic.

Even if object instantiation is not often a hot spot in applications, it can still be useful to have an idea of the overhead DI adds compared to instantiation with 'new'. Especially when deciding whether or not to use DI inside code you know will be an application hot spot.

We have put together a small performance test to shed some light on the overhead added to instantiation by Butterfly Container. We even included Google Guice (another DI framework) just to have another similar product to compare with. Only you yourself can determine if the overheads of Butterfly (or Guice) are acceptable in your project. Hopefully these numbers can help you. The results are listed below, ordered after time:

NOTE: Over time this test suite will be expanded to include more instantiation scenarios like more complex object graphs, singletons etc. For now it only benchmarks simple no-arg constructor instantiation.

This test was performed using Java 5, Butterfly Container 1.9.11-beta, and Guice 1.0.

The test performs 10.000.000 instantiations of a TestProduct class using a no-arg constructor, for each DI mechanism. All tests were repeated 10 times to allow the JVM to warm up. We have also tried to change the order of the tests when run to see if the sequence changed the result, but it didn't seem to make any noticable differences. Here are the numbers of the first and last of the 10 test runs:

-- NEW BASED INSTANTIATION ----------------------
Java new           :  100%,   687
Butterfly Java     :  245%,  1688
Butterfly Java 2   :  136%,   937
Guice Provider     :  866%,  5953
Guice Provider 2   :  420%,  2891
-- REFLECTION BASED INSTANTIATION ----------------
Java Reflection    :  386%,  2656
Butterfly Script   :  891%,  6125
Butterfly Script 2 :  707%,  4860
Guice Reflection   : 1846%, 12687
Guice Reflection 2 : 1394%,  9578
=================================================

-- NEW BASED INSTANTIATION ----------------------
Java new           :  100%,   750
Butterfly Java     :  229%,  1719
Butterfly Java 2   :  135%,  1016
Guice Provider     :  806%,  6047
Guice Provider 2   :  391%,  2937
-- REFLECTION BASED INSTANTIATION ----------------
Java Reflection    :  362%,  2719
Butterfly Script   :  845%,  6343
Butterfly Script 2 :  672%,  5047
Guice Reflection   : 1706%, 12797
Guice Reflection 2 : 1310%,  9828
=================================================

The numbers pretty much speak for themselves, but here are a few comments anyways:

There seems to be a small relative speedup in all mechanisms except 'new' from the first to the last run, but the speedup is not revolutionary. The observant reader may have noticed that this relative speedup seems to have come from 'new' being slower in the last run. Not the other mechanisms having sped up. In fact they have slowed down too. Just not as much as 'new'. We have seen this relative speedup in many other test runs, where 'new' didn't actually get slower, but where the other mechanisms got a little faster. Still, the speedups where small.

The first five tests use instantiation mechanisms based on Java's new, meaning they do not use reflection. The last three tests use reflection based instantiation mechanisms. Therefore the first five tests are of course faster than the last three.

If you check the test code (below) you will notice that two different instantiation methods were used: The first method asks the Butterfly Container / Guice Injector directly for an instance of some type. This requires a lookup inside the container / injector to find the factory/provider matching the requested type. The second method obtains the factory / provider from the container / injector once, then call that directly for all instantiations during the test. Not surprisingly it is faster to use the factory / provider directly than go through the container. That might be worth remembering if using DI in a hot spot.

It seems Butterfly Container has a factor 2 to 4 performance advantage over Guice. According to Bob Lee (Guice main developer) this is caused by some checks Guice does internally, for instance checking for circular dependencies. Butterfly solves these problems in a different way, so it doesn't have these checks at instantiation time.

The tests were run singlethreadedly. When we get around to designing a multithreaded test, we will publish the results here to.

The full test code is included below. To run this test yourself copy the class below. Add jenkov-butterfly-container-1.9.10-beta.jar and guice-1.0.jar to the classpath. Then call main(...).

import com.google.inject.*;
import com.jenkov.container.TestProduct;
import com.jenkov.container.Container;
import com.jenkov.container.IContainer;
import com.jenkov.container.itf.factory.IGlobalFactory;
import com.jenkov.container.script.ScriptFactoryBuilder;
import com.jenkov.container.java.JavaFactoryBuilder;
import com.jenkov.container.java.JavaFactory;

import java.lang.reflect.Constructor;
import java.lang.reflect.InvocationTargetException;

/**

 */
public class PerformanceTestGuice {

  public static int iterationsPerTest    = 10 * 1000 * 1000;
  public static int iterationsOfAllTests = 10;

  public static void main(String[] args)
  throws NoSuchMethodException, InvocationTargetException,
       InstantiationException, IllegalAccessException {

    /* NEW INSTANCE TEST */
    for(int i=0; i<iterationsOfAllTests; i++){
      long javaNewTime         = javaNewTest();
      long javaReflectionTime  = javaReflectionTest();
      long butterflyScriptTime = butterflyScriptTest();
      long butterflyScript2Time= butterflyScriptTest2();
      long butterflyJavaTime   = butterflyJavaTest();
      long butterflyJava2Time  = butterflyJava2Test();
      long guiceTime           = guiceTest();
      long guice2Time          = guiceTest2();
      long guiceProviderTime   = guiceProviderTest();
      long guiceProvider2Time  = guiceProvider2Test();

      long javaReflectionTimeRatio  =
          (javaReflectionTime  * 100) / javaNewTime;
      long butterflyJavaTimeRatio   =
          (butterflyJavaTime   * 100) / javaNewTime;
      long butterflyJava2TimeRatio  =
          (butterflyJava2Time  * 100) / javaNewTime;
      long butterflyScript2TimeRatio =
          (butterflyScript2Time * 100) / javaNewTime;
      long butterflyScriptTimeRatio =
          (butterflyScriptTime * 100) / javaNewTime;
      long guiceTimeRatio           =
          (guiceTime           * 100) / javaNewTime;
      long guiceTime2Ratio          =
          (guice2Time          * 100) / javaNewTime;
      long guiceProviderTimeRatio   =
          (guiceProviderTime   * 100) / javaNewTime;
      long guiceProvider2TimeRatio  =
          (guiceProvider2Time  * 100) / javaNewTime;


      System.out.println("-- NEW BASED INSTANTIATION --------------");
      System.out.println("Java new           :  100%,   "
          + javaNewTime);
      System.out.println("Butterfly Java     :  "
          + butterflyJavaTimeRatio + "%,  " + butterflyJavaTime);
      System.out.println("Butterfly Java 2   :  "
          + butterflyJava2TimeRatio + "%,  " + butterflyJava2Time);
      System.out.println("Guice Provider     : "
          + guiceProviderTimeRatio + "%, " + guiceProviderTime);
      System.out.println("Guice Provider 2   : "
          + guiceProvider2TimeRatio  + "%, " + guiceProvider2Time);
      System.out.println("-- REFLECTION BASED INSTANTIATION ------");
      System.out.println("Java Reflection    : "
          + javaReflectionTimeRatio + "%,  " + javaReflectionTime);
      System.out.println("Butterfly Script   : "
          + butterflyScriptTimeRatio + "%, " + butterflyScriptTime);
      System.out.println("Butterfly Script 2 : "
          + butterflyScript2TimeRatio + "%, " + butterflyScript2Time);
      System.out.println("Guice Reflection   : "
          + guiceTimeRatio + "%, " + guiceTime);
      System.out.println("Guice Reflection 2 : "
          + guiceTime2Ratio + "%, " + guice2Time);
      System.out.println("========================================");
    }
  }


  protected static long javaNewTest(){
      long newStart = System.currentTimeMillis();
      for(int i=0; i< iterationsPerTest; i++){
          new TestProduct();
      }
      long newEnd = System.currentTimeMillis();
      return newEnd - newStart;
  }

  protected static long javaReflectionTest()
  throws NoSuchMethodException, InvocationTargetException,
         IllegalAccessException, InstantiationException {
       Constructor constructor = TestProduct.class.getConstructor(null);
      long newStart = System.currentTimeMillis();
      for(int i=0; i< iterationsPerTest; i++){
          constructor.newInstance(null);
      }
      long newEnd = System.currentTimeMillis();
      return newEnd - newStart;
  }

  protected static long butterflyScriptTest(){
    IContainer container = new Container();
    ScriptFactoryBuilder builder = new ScriptFactoryBuilder(container);
    builder.addFactory("script = * com.jenkov.container.TestProduct();");

    long butterflyScriptStart = System.currentTimeMillis();
    for(int i=0; i< iterationsPerTest; i++){
        container.instance("script");
    }
    long butterflyScriptEnd = System.currentTimeMillis();
    return butterflyScriptEnd - butterflyScriptStart;
  }

  protected static long butterflyScriptTest2(){
    IContainer container = new Container();
    ScriptFactoryBuilder builder = new ScriptFactoryBuilder(container);
    builder.addFactory("script = * com.jenkov.container.TestProduct();");

    IGlobalFactory<TestProduct> factory =
        container.getFactory("script");

    long butterflyScriptStart = System.currentTimeMillis();
    for(int i=0; i< iterationsPerTest; i++){
        factory.instance();
    }
    long butterflyScriptEnd = System.currentTimeMillis();
    return butterflyScriptEnd - butterflyScriptStart;
  }

  protected static long butterflyJavaTest(){
      IContainer container = new Container();
      JavaFactoryBuilder javaBuilder =
            new JavaFactoryBuilder(container);

      javaBuilder.addFactory("java", TestProduct.class,
        new JavaFactory(){
          public Object instance(Object... parameters) {
              return new TestProduct();
          }
      });

      long butterflyJavaStart = System.currentTimeMillis();
      for(int i=0; i< iterationsPerTest; i++){
          container.instance("java");
      }
      long butterflyJavaEnd = System.currentTimeMillis();
      return butterflyJavaEnd - butterflyJavaStart;
  }

  protected static long butterflyJava2Test(){
      IContainer container = new Container();
      JavaFactoryBuilder javaBuilder =
            new JavaFactoryBuilder(container);

      javaBuilder.addFactory("java", TestProduct.class,
        new JavaFactory(){
          public Object instance(Object... parameters) {
              return new TestProduct();
          }
      });

      IGlobalFactory<TestProduct> javaFactory =
          container.getFactory("java");


      long butterflyJavaStart = System.currentTimeMillis();
      for(int i=0; i< iterationsPerTest; i++){
          javaFactory.instance();
      }
      long butterflyJavaEnd = System.currentTimeMillis();
      return butterflyJavaEnd - butterflyJavaStart;
  }


  protected static long guiceTest(){
      /* GUICE TEST */
      Injector injector =  Guice.createInjector(new Module(){
          public void configure(Binder binder) {
              binder.bind(TestProduct.class);
          }
      }) ;

      long guiceStart = System.currentTimeMillis();
      for(int i=0; i< iterationsPerTest; i++){
          injector.getInstance(TestProduct.class);
      }
      long guiceEnd = System.currentTimeMillis();

      return guiceEnd - guiceStart;
  }

  protected static long guiceTest2(){
      /* GUICE TEST */
      Injector injector =  Guice.createInjector(new Module(){
          public void configure(Binder binder) {
              binder.bind(TestProduct.class);
          }
      }) ;

      Provider<TestProduct> provider =
          injector.getProvider(TestProduct.class);

      long guiceStart = System.currentTimeMillis();
      for(int i=0; i< iterationsPerTest; i++){
          provider.get();
      }
      long guiceEnd = System.currentTimeMillis();

      return guiceEnd - guiceStart;
  }

  protected static long guiceProviderTest(){

      final Provider<TestProduct> provider =
          new Provider<TestProduct>(){
              public TestProduct get() {
                  return new TestProduct();
              }
          };

      /* GUICE TEST */
      Injector injector =  Guice.createInjector(new Module(){
          public void configure(Binder binder) {
              binder.bind(TestProduct.class).toProvider(provider);
          }
      }) ;

      long guiceStart = System.currentTimeMillis();
      for(int i=0; i< iterationsPerTest; i++){
          injector.getInstance(TestProduct.class);
      }
      long guiceEnd = System.currentTimeMillis();

      return guiceEnd - guiceStart;
  }


  protected static long guiceProvider2Test(){
       Injector injector =  Guice.createInjector(new Module(){
           public void configure(Binder binder) {
               binder.bind(TestProduct.class).toProvider(new
                   Provider<TestProduct>() {
                       public TestProduct get() {
                           return new TestProduct();
                       }
               });
           }
       }) ;

       Provider<TestProduct> provider =
          injector.getProvider(TestProduct.class);

       long guiceStart = System.currentTimeMillis();
       for(int i=0; i< iterationsPerTest; i++){
           provider.get();
       }
       long guiceEnd = System.currentTimeMillis();

       return guiceEnd - guiceStart;
   }

Jakob Jenkov

Featured Videos

Java ConcurrentMap + ConcurrentHashMap

Java Generics

Java ForkJoinPool

P2P Networks Introduction

















Close TOC
All Tutorial Trails
All Trails
Table of contents (TOC) for this tutorial trail
Trail TOC
Table of contents (TOC) for this tutorial
Page TOC
Previous tutorial in this tutorial trail
Previous
Next tutorial in this tutorial trail
Next