Hello, I would like to switch from Log4J to Logback. The old system extended Log4J FileAppender and used a method "setBufferedIO". The Logback FileAppender has no such a method. Does it no longer buffer? Regards, Lars
On Fri, 13 Aug 2010 12:57:22 +0200, Lars Fischer wrote:
I would like to switch from Log4J to Logback. The old system extended Log4J FileAppender and used a method "setBufferedIO".
The Logback FileAppender has no such a method. Does it no longer buffer?
From http://logback.qos.ch/manual/appenders.html#OutputStreamAppender:
"Given the structure of current hard drives and performance tests we have conducted, it appears that turning off immediate flushing or buffering of character to byte conversions have rather small impact on performance. As such, as of logback vesion 0.9.19, the ImmediateFlush, BufferedIO and BufferSize properties have been removed without replacement." :-/ I got curious and indeed it seems that the underlying file stream implementation does no buffering, presumably to allow for recovery. However by default it does not fsync on every event, so even though you pay the OS system call and file locking (!) overhead, the OS/drive etc. will likely still buffer anyway. In my experience buffering at least the usual 4KB (pagesize) before writing can help a lot; log event output is typically a lot shorter than that. Interestingly I remember from a few years back that *some* people with lots of logs on high-traffic web sites sometimes got better throughput from unbuffered writes (contrary to everybody's expectation), whereas most others said buffering was absolutely necessary and a great improvement. -h
Hello Holger, 2010/8/14 Holger Hoffstaette <holger.hoffstaette@googlemail.com>:
On Fri, 13 Aug 2010 12:57:22 +0200, Lars Fischer wrote:
From http://logback.qos.ch/manual/appenders.html#OutputStreamAppender:
"Given the structure of current hard drives and performance tests we have conducted, it appears that turning off immediate flushing or buffering of character to byte conversions have rather small impact on performance. As such, as of logback vesion 0.9.19, the ImmediateFlush, BufferedIO and BufferSize properties have been removed without replacement."
Sorry, I missed this part of the docu.
I got curious and indeed it seems that the underlying file stream implementation does no buffering, presumably to allow for recovery. However by default it does not fsync on every event, so even though you pay the OS system call and file locking (!) overhead, the OS/drive etc. will likely still buffer anyway. In my experience buffering at least the usual 4KB (pagesize) before writing can help a lot; log event output is typically a lot shorter than that.
Interestingly I remember from a few years back that *some* people with lots of logs on high-traffic web sites sometimes got better throughput from unbuffered writes (contrary to everybody's expectation), whereas most others said buffering was absolutely necessary and a great improvement.
Hmm, looks like I have to drop this feature, when switching to logback. Thank you for the hint! Regards, Lars
participants (2)
-
Holger Hoffstaette -
Lars Fischer