Hello, I would like to replace the CallerData class with StackTraceElement. The two classes contain the same fields. StackTraceElement is part of the JDK. I can think of no valid reason to duplicate a class found in the JDK. Comments/objections? -- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch
The only concern I would have is that it would break an existing user's code to remove it. I realize the version is still 0.9.something, but at this point Logback is being used in production, probably in lots of places. What I would suggest is to change all the Logback code to use StackTraceElement but deprecate CallerData and the getCallerData method and then have CallerData extend StackTraceElement as a wrapper. Make sure you add getStackTrace to the LoggingEvent. getCallerData could then create the CallerData array from the StackTrace array. Ralph On Mar 18, 2009, at 4:12 AM, Ceki Gulcu wrote:
Hello,
I would like to replace the CallerData class with StackTraceElement. The two classes contain the same fields. StackTraceElement is part of the JDK. I can think of no valid reason to duplicate a class found in the JDK.
Comments/objections?
-- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch _______________________________________________ logback-dev mailing list logback-dev@qos.ch http://qos.ch/mailman/listinfo/logback-dev
On 18.03.2009, at 14:52, Ralph Goers wrote:
The only concern I would have is that it would break an existing user's code to remove it. I realize the version is still 0.9.something, but at this point Logback is being used in production, probably in lots of places.
What I would suggest is to change all the Logback code to use StackTraceElement but deprecate CallerData and the getCallerData method and then have CallerData extend StackTraceElement as a wrapper. Make sure you add getStackTrace to the LoggingEvent. getCallerData could then create the CallerData array from the StackTrace array.
Extending StackTraceElement won't work because STE is a final class.
On Mar 18, 2009, at 4:12 AM, Ceki Gulcu wrote:
I would like to replace the CallerData class with StackTraceElement. The two classes contain the same fields. StackTraceElement is part of the JDK. I can think of no valid reason to duplicate a class found in the JDK.
Comments/objections?
No real objection. Why don't you simply use StackTraceElementProxy instead of STE? That way, it could - optionally - also contain ClassPackagingData. Joern.
Joern Huxhorn wrote:
Extending StackTraceElement won't work because STE is a final class.
Good point.
Why don't you simply use StackTraceElementProxy instead of STE? That way, it could - optionally - also contain ClassPackagingData.
As it stands, CallerData is almost identical to the StackTraceElement class. I don't expect CallerData to ever contain ClassPackagingData because collecting class packaging data is an extremely expensive operation, barely tolerable for exceptions but certainly not for every log statement. I would be quite surprised if we ever wanted to replace CallerData/STE with STEProxy.
Joern.
-- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch
On 18.03.2009, at 17:35, Ceki Gulcu wrote:
Joern Huxhorn wrote:
Extending StackTraceElement won't work because STE is a final class.
Good point.
Why don't you simply use StackTraceElementProxy instead of STE? That way, it could - optionally - also contain ClassPackagingData.
As it stands, CallerData is almost identical to the StackTraceElement class. I don't expect CallerData to ever contain ClassPackagingData because collecting class packaging data is an extremely expensive operation, barely tolerable for exceptions but certainly not for every log statement. I would be quite surprised if we ever wanted to replace CallerData/STE with STEProxy.
Well, I agree with your performance argument but the CPD field of STEProxy doesn't need to be used in the common case. I'd use STEProxy simply to keep the possibility to activate such info on demand, i.e. while debugging but not in production. It could certainly come in quite handy under some circumstances, e.g. while walking the path of jar hell. Joern.
I don't see packaging data being part of caller data. If it were, we would need configuration code to support this case, which would unnecessarily bloat the code. Just as importantly, the user would have to know how to enable/disable this feature. It's just not worth the trouble. If you really needed the packaging data, you would write logger.debug("I am interested in package data", new Throwable(""); Joern Huxhorn wrote:
Well, I agree with your performance argument but the CPD field of STEProxy doesn't need to be used in the common case.
I'd use STEProxy simply to keep the possibility to activate such info on demand, i.e. while debugging but not in production. It could certainly come in quite handy under some circumstances, e.g. while walking the path of jar hell.
Joern.
-- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch
On Wed, Mar 18, 2009 at 9:38 PM, Ceki Gulcu <ceki@qos.ch> wrote:
I don't see packaging data being part of caller data. If it were, we would need configuration code to support this case, which would unnecessarily bloat the code. Just as importantly, the user would have to know how to enable/disable this feature. It's just not worth the trouble.
I agree. I've never been a fan of packaging data, not even in stacktraces. Maarten
If you really needed the packaging data, you would write
logger.debug("I am interested in package data", new Throwable("");
Joern Huxhorn wrote:
Well, I agree with your performance argument but the CPD field of STEProxy doesn't need to be used in the common case.
I'd use STEProxy simply to keep the possibility to activate such info on demand, i.e. while debugging but not in production. It could certainly come in quite handy under some circumstances, e.g. while walking the path of jar hell.
Joern.
-- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch _______________________________________________ logback-dev mailing list logback-dev@qos.ch http://qos.ch/mailman/listinfo/logback-dev
On 18.03.2009, at 21:54, Maarten Bosteels wrote:
I agree. I've never been a fan of packaging data, not even in stacktraces.
It actually helped us in our current project so I could be considered a fan ;) It was a case of jar-hell that was caused by a changed group- and artifactId which ultimately resulted in having two versions of the "same" jar in the classpath at the same time. I guess it saved us a bit of time..... Joern.
I was also wondering why Maarten did not find packaging data useful. Joern Huxhorn wrote:
On 18.03.2009, at 21:54, Maarten Bosteels wrote:
I agree. I've never been a fan of packaging data, not even in stacktraces.
It actually helped us in our current project so I could be considered a fan ;)
It was a case of jar-hell that was caused by a changed group- and artifactId which ultimately resulted in having two versions of the "same" jar in the classpath at the same time. I guess it saved us a bit of time.....
Joern.
-- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch
Hello Ralph, The LoggingEvent class changed to a large extent in SVN trunk. Replacing CallerData with StackTraceElement is yet another incompatible change. Worrying about backward compatibility for CallerData and not for the rest would be pretty pointless. Ralph Goers wrote:
The only concern I would have is that it would break an existing user's code to remove it. I realize the version is still 0.9.something, but at this point Logback is being used in production, probably in lots of places.
What I would suggest is to change all the Logback code to use StackTraceElement but deprecate CallerData and the getCallerData method and then have CallerData extend StackTraceElement as a wrapper. Make sure you add getStackTrace to the LoggingEvent. getCallerData could then create the CallerData array from the StackTrace array.
Ralph
-- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch
Hi, No objections at all. Looking at my .proto file it's indeed pretty obvious that CallerData and StackTraceElement are quite similar :-) http://code.google.com/p/firewood/source/browse/trunk/compare-formats/src/ma... I've done some benchmarking with protobuf (using the Corpus), and it seems to be at least 4 times faster than Java serialization. Will soon post the results and the code. regards, Maarten On Wed, Mar 18, 2009 at 12:12 PM, Ceki Gulcu <ceki@qos.ch> wrote:
Hello,
I would like to replace the CallerData class with StackTraceElement. The two classes contain the same fields. StackTraceElement is part of the JDK. I can think of no valid reason to duplicate a class found in the JDK.
Comments/objections?
-- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch _______________________________________________ logback-dev mailing list logback-dev@qos.ch http://qos.ch/mailman/listinfo/logback-dev
Hello Maarten, It's interesting to learn that protobuf is faster than plain old java serialization. Thanks. How about the size of the stored data? How many bytes does a logging event use on average? Maarten Bosteels wrote:
Hi,
No objections at all.
Looking at my .proto file it's indeed pretty obvious that CallerData and StackTraceElement are quite similar :-) http://code.google.com/p/firewood/source/browse/trunk/compare-formats/src/ma...
I've done some benchmarking with protobuf (using the Corpus), and it seems to be at least 4 times faster than Java serialization. Will soon post the results and the code.
regards, Maarten
-- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch
I just read the project page for firewood. What type of GUI you have in mind? Is it only going to be an IDEA plug-in, or do you have other platforms in mind? Ceki Gulcu wrote:
Hello Maarten,
It's interesting to learn that protobuf is faster than plain old java serialization. Thanks.
How about the size of the stored data? How many bytes does a logging event use on average?
-- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch
On Wed, Mar 18, 2009 at 10:00 PM, Ceki Gulcu <ceki@qos.ch> wrote:
I just read the project page for firewood. What type of GUI you have in mind? Is it only going to be an IDEA plug-in, or do you have other platforms in mind?
The GUI would be in Swing. But it's a long term project :-) The aim is to make a simple plugin for both Eclipse and IDEA (and perhaps Netbeans) with almost no GUI, to keep it low-maintenance Unfortunately Jetbrains like to break their plugin API for every version :-( And then have a rich stand-alone Swing GUI that everyone can combine with their favorite IDE. The IDEA plugin works under IDEA 7.x and is already integrated with Vigilog. But I still have to fix it for IDEA 8. I looked at Lilith when it was still called Lumberjack, and it seems to have a slightly different goal. Anyway, I should have another look at it, and steal some of its ideas :-) Maarten Ceki Gulcu wrote:
Hello Maarten,
It's interesting to learn that protobuf is faster than plain old java serialization. Thanks.
How about the size of the stored data? How many bytes does a logging event use on average?
--
Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch _______________________________________________ logback-dev mailing list logback-dev@qos.ch http://qos.ch/mailman/listinfo/logback-dev
Hi Ceki, Keep in mind that I still have to double check my LoggingEvent-to-protobuf converter. Serializing the corpus with different formats: java serialization => 16 MB (16108602) protobuf => 33 MB (34385867 bytes) protobuf + gzip => 9.9 MB (10354646 bytes) protbuf + deflater => 12 MB (12006206 bytes) speed: java serialization: 4330 ms protobuf 928 ms protobuf + gzip 3146 ms protobuf + deflater 1883 ms So I would choose for protobuf + deflater : 25% smaller than java serilaization and twice as fast. Deflater means using java.util.zip.Deflater.BEST_SPEED regards, Maarten On Wed, Mar 18, 2009 at 9:49 PM, Ceki Gulcu <ceki@qos.ch> wrote:
Hello Maarten,
It's interesting to learn that protobuf is faster than plain old java serialization. Thanks.
How about the size of the stored data? How many bytes does a logging event use on average?
Maarten Bosteels wrote:
Hi,
No objections at all.
Looking at my .proto file it's indeed pretty obvious that CallerData and StackTraceElement are quite similar :-)
http://code.google.com/p/firewood/source/browse/trunk/compare-formats/src/ma...
I've done some benchmarking with protobuf (using the Corpus), and it seems to be at least 4 times faster than Java serialization. Will soon post the results and the code.
regards, Maarten
-- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch _______________________________________________ logback-dev mailing list logback-dev@qos.ch http://qos.ch/mailman/listinfo/logback-dev
Are you using the standard corpus with 50'000? If so, 12MB for 50'000 elements translates to about 240 bytes per message which is pretty good. The CPU times, e.g. 1883 ms, are fantastic. Maarten Bosteels wrote:
Hi Ceki,
Keep in mind that I still have to double check my LoggingEvent-to-protobuf converter.
Serializing the corpus with different formats: java serialization => 16 MB (16108602) protobuf => 33 MB (34385867 bytes) protobuf + gzip => 9.9 MB (10354646 bytes) protbuf + deflater => 12 MB (12006206 bytes)
speed:
java serialization: 4330 ms protobuf 928 ms protobuf + gzip 3146 ms protobuf + deflater 1883 ms
So I would choose for protobuf + deflater : 25% smaller than java serilaization and twice as fast.
Deflater means using java.util.zip.Deflater.BEST_SPEED
regards, Maarten
On Wed, Mar 18, 2009 at 9:49 PM, Ceki Gulcu <ceki@qos.ch> wrote:
Hello Maarten,
It's interesting to learn that protobuf is faster than plain old java serialization. Thanks.
How about the size of the stored data? How many bytes does a logging event use on average?
Maarten Bosteels wrote:
Hi,
No objections at all.
Looking at my .proto file it's indeed pretty obvious that CallerData and StackTraceElement are quite similar :-) http://code.google.com/p/firewood/source/browse/trunk/compare-formats/src/ma...
I've done some benchmarking with protobuf (using the Corpus), and it seems to be at least 4 times faster than Java serialization. Will soon post the results and the code.
regards, Maarten
-- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch _______________________________________________ logback-dev mailing list logback-dev@qos.ch http://qos.ch/mailman/listinfo/logback-dev
------------------------------------------------------------------------
_______________________________________________ logback-dev mailing list logback-dev@qos.ch http://qos.ch/mailman/listinfo/logback-dev
-- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch
On Wed, Mar 18, 2009 at 10:25 PM, Ceki Gulcu <ceki@qos.ch> wrote:
Are you using the standard corpus with 50'000? If so, 12MB for 50'000 elements translates to about 240 bytes per message which is pretty good. The CPU times, e.g. 1883 ms, are fantastic.
yep, ILoggingEvent[] eventArray = Corpus.makeStandardCorpus(); for (int i=0; i<10; i++) { ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream("/tmp/logback-java.ser." + i)); OutputStream os = new BufferedOutputStream(new FileOutputStream("/tmp/logback-protobuf.ser" + i)); GZIPOutputStream gos = new GZIPOutputStream(new FileOutputStream("/tmp/logback-proto." + i + ".gz"), 8000); DeflaterOutputStream dos = new DeflaterOutputStream( new FileOutputStream("/tmp/logback-protobuf.def." + i), new Deflater(Deflater.BEST_SPEED)); serialize(eventArray, oos); serializeWithProtobuf("normal", eventArray, os); serializeWithProtobuf("gzip ", eventArray, gos); serializeWithProtobuf("def ", eventArray, dos); oos.close(); os.close(); gos.close(); } Still have to create a multi-threaded test, but I suspect it won't make much difference. Maarten
Maarten Bosteels wrote:
Hi Ceki,
Keep in mind that I still have to double check my LoggingEvent-to-protobuf converter.
Serializing the corpus with different formats: java serialization => 16 MB (16108602) protobuf => 33 MB (34385867 bytes) protobuf + gzip => 9.9 MB (10354646 bytes) protbuf + deflater => 12 MB (12006206 bytes)
speed:
java serialization: 4330 ms protobuf 928 ms protobuf + gzip 3146 ms protobuf + deflater 1883 ms
So I would choose for protobuf + deflater : 25% smaller than java serilaization and twice as fast.
Deflater means using java.util.zip.Deflater.BEST_SPEED
regards, Maarten
On Wed, Mar 18, 2009 at 9:49 PM, Ceki Gulcu <ceki@qos.ch> wrote:
Hello Maarten,
It's interesting to learn that protobuf is faster than plain old java serialization. Thanks.
How about the size of the stored data? How many bytes does a logging event use on average?
Maarten Bosteels wrote:
Hi,
No objections at all.
Looking at my .proto file it's indeed pretty obvious that CallerData and StackTraceElement are quite similar :-)
http://code.google.com/p/firewood/source/browse/trunk/compare-formats/src/ma...
I've done some benchmarking with protobuf (using the Corpus), and it seems to be at least 4 times faster than Java serialization. Will soon post the results and the code.
regards, Maarten
-- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch _______________________________________________ logback-dev mailing list logback-dev@qos.ch http://qos.ch/mailman/listinfo/logback-dev
------------------------------------------------------------------------
_______________________________________________ logback-dev mailing list logback-dev@qos.ch http://qos.ch/mailman/listinfo/logback-dev
-- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch _______________________________________________ logback-dev mailing list logback-dev@qos.ch http://qos.ch/mailman/listinfo/logback-dev
Maarten Bosteels skrev:
Hi Ceki,
Keep in mind that I still have to double check my LoggingEvent-to-protobuf converter.
Serializing the corpus with different formats: java serialization => 16 MB (16108602) protobuf => 33 MB (34385867 bytes) protobuf + gzip => 9.9 MB (10354646 bytes) protbuf + deflater => 12 MB (12006206 bytes)
speed:
java serialization: 4330 ms protobuf 928 ms protobuf + gzip 3146 ms protobuf + deflater 1883 ms
So I would choose for protobuf + deflater : 25% smaller than java serilaization and twice as fast.
Deflater means using java.util.zip.Deflater.BEST_SPEED
Very interesting. Even with 100 Mbit speeds the deflater is faster than shipping the raw bytes. Please make it optional though (perhaps making the deflater level configurable and 0 meaning leave alone). -- Thorbjørn Ravn Andersen "...plus... Tubular Bells!"
participants (5)
-
Ceki Gulcu -
Joern Huxhorn -
Maarten Bosteels -
Ralph Goers -
Thorbjoern Ravn Andersen