On Wed, Mar 18, 2009 at 10:25 PM, Ceki Gulcu <ceki@qos.ch> wrote:

Are you using the standard corpus with 50'000? If so, 12MB for 50'000 elements translates to about 240 bytes per message which is pretty good. The CPU times, e.g. 1883 ms, are fantastic.

yep,

ILoggingEvent[] eventArray = Corpus.makeStandardCorpus();

    for (int i=0; i<10; i++) {
      ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream("/tmp/logback-java.ser." + i));
      OutputStream os = new BufferedOutputStream(new FileOutputStream("/tmp/logback-protobuf.ser" + i));
      GZIPOutputStream gos = new GZIPOutputStream(new FileOutputStream("/tmp/logback-proto." + i + ".gz"), 8000);

      DeflaterOutputStream dos = new DeflaterOutputStream(
          new FileOutputStream("/tmp/logback-protobuf.def." + i),
          new Deflater(Deflater.BEST_SPEED));

      serialize(eventArray, oos);
      serializeWithProtobuf("normal", eventArray, os);
      serializeWithProtobuf("gzip ", eventArray, gos);
      serializeWithProtobuf("def   ", eventArray, dos);
      oos.close();
      os.close();
      gos.close();
    }

Still have to create a multi-threaded test, but I suspect it won't make much difference.

Maarten

Maarten Bosteels wrote:

Hi Ceki,

Keep in mind that I still have to double check my LoggingEvent-to-protobuf converter.

Serializing the corpus with different formats:
java serialization => 16 MB (16108602) protobuf => 33 MB (34385867 bytes)
protobuf + gzip => 9.9 MB (10354646 bytes)
protbuf + deflater => 12 MB (12006206 bytes)

speed:

java serialization: 4330 ms
protobuf 928 ms
protobuf + gzip 3146 ms protobuf + deflater 1883 ms

So I would choose for protobuf + deflater : 25% smaller than java serilaization and twice as fast.

Deflater means using java.util.zip.Deflater.BEST_SPEED

regards,
Maarten

On Wed, Mar 18, 2009 at 9:49 PM, Ceki Gulcu <ceki@qos.ch> wrote:

Hello Maarten,

It's interesting to learn that protobuf is faster than plain old
java serialization. Thanks.

How about the size of the stored data? How many bytes does a logging
event use on average?

Maarten Bosteels wrote:

Hi,

No objections at all.

Looking at my .proto file it's indeed pretty obvious that
CallerData and StackTraceElement are quite similar :-)
http://code.google.com/p/firewood/source/browse/trunk/compare-formats/src/main/java/com/googlecode/firewood/protobuf/logging.proto

I've done some benchmarking with protobuf (using the Corpus),
and it seems to be at least 4 times faster than Java serialization.
Will soon post the results and the code.

regards,
Maarten

-- Ceki Gülcü
Logback: The reliable, generic, fast and flexible logging framework
for Java.
http://logback.qos.ch
_______________________________________________
logback-dev mailing list
logback-dev@qos.ch
http://qos.ch/mailman/listinfo/logback-dev

------------------------------------------------------------------------

_______________________________________________
logback-dev mailing list
logback-dev@qos.ch
http://qos.ch/mailman/listinfo/logback-dev

--
Ceki Gülcü
Logback: The reliable, generic, fast and flexible logging framework for Java.
http://logback.qos.ch
_______________________________________________
logback-dev mailing list
logback-dev@qos.ch
http://qos.ch/mailman/listinfo/logback-dev