Hi, Objective: Dump a "history" of recent program operations to a log file when an Exception arises. By logging to a CyclicBufferAppender, a recent history at a fairly granular level (INFO, or possibly DEBUG) can be maintained without accumulating massive amounts of data, and with a minimum of overhead. Setting this up to log to the Cyclical Buffer went well. To get the data out of the CyclicBuffer and log it to a file, a reference to the CyclicBufferAppender object is needed, and it is not clear how to get one. It seems like log.getAppender(String name) should return an Appender, but it doesn't. How can I obtain such a reference? Also, once a reference is obtained, and an Object is returned from cyclicBufferAppender.get(i), what class will that object be? I'm hoping it is a LoggingEvent object, in which case a simple call to the fileAppender.doAppend method with the event as a parameter will log the event to a "permanent" log. Again, the code will need to obtain a reference to the appropriate FileAppender to log it to. (Appropriate might be hardcoded, or specified in a properties file.) Most of the configuration is being done in logback.xml. (File is attached.) Program code, other than wanting to dump the MEMORY appender, is limited to: static final Logger log = LoggerFactory.getLogger(Main.class); and various log.X method calls. As an aside, it would be even better to loop over all Appenders, and check to see which ones are CyclicBufferAppenders. The code would dump then contents of all of the CyclicBufferAppenders, or selected ones based on some user options. Thanks in advance for your insight. Mike
Hello Mike, Comments inline. mweeda wrote:
Hi,
Objective: Dump a "history" of recent program operations to a log file when an Exception arises. By logging to a CyclicBufferAppender, a recent history at a fairly granular level (INFO, or possibly DEBUG) can be maintained without accumulating massive amounts of data, and with a minimum of overhead.
How do you link throwing an exception with dumping log buffer? Do you add the code that link the exception with dumping the log buffer into many try/catch blocks manually? That sounds quite intrusive. Have you come up with something else?
Setting this up to log to the Cyclical Buffer went well. To get the data out of the CyclicBuffer and log it to a file, a reference to the CyclicBufferAppender object is needed, and it is not clear how to get one. It seems like log.getAppender(String name) should return an Appender, but it doesn't. How can I obtain such a reference?
In the config file you show, the appender named MEMORY is attached to the root logger. So you would need to write: import org.slf4j.LoggerFactory; import ch.qos.logback.classic.Logger; import ch.qos.logback.classic.spi.LoggingEvent; import ch.qos.logback.core.read.CyclicBufferAppender; public class Main { public static void main(String[] args) { // The cast will work only if SLF4J is bound with logback Logger root = (Logger) LoggerFactory.getLogger(Logger.ROOT_LOGGER_NAME); root.warn("x"); root.warn("y"); CyclicBufferAppender<LoggingEvent> cba = (CyclicBufferAppender<LoggingEvent>) root.getAppender("MEMORY"); System.out.println(cba.getLength()); // some form of synchronization is needed here!!! for(int i = 0; i < cba.getLength(); i++) { System.out.println(cba.get(i)); } } }
Also, once a reference is obtained, and an Object is returned from cyclicBufferAppender.get(i), what class will that object be? I'm hoping it is a LoggingEvent object,
Yep, that is the case.
in which case a simple call to the fileAppender.doAppend method with the event as a parameter will log the event to a "permanent" log. Again, the code will need to obtain a reference to the appropriate FileAppender to log it to. (Appropriate might be hardcoded, or specified in a properties file.)
You could configure a file appender and attach it to the cyclic buffer if you wish. Joran is designed to handle such use cases. Please let me know if you want to pursue this route.
Most of the configuration is being done in logback.xml. (File is attached.) Program code, other than wanting to dump the MEMORY appender, is limited to: static final Logger log = LoggerFactory.getLogger(Main.class); and various log.X method calls.
As an aside, it would be even better to loop over all Appenders, and check to see which ones are CyclicBufferAppenders. The code would dump then contents of all of the CyclicBufferAppenders, or selected ones based on some user options.
How about the following code? LoggerContext context = (LoggerContext) LoggerFactory.getILoggerFactory(); for (Logger l : context.getLoggerList()) { Iterator ite = l.iteratorForAppenders(); while(ite.hasNext()) { Object o = ite.next(); if(o instanceof CyclicBufferAppender) { // dump } } }
Mike -- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch
Thanks for your help, Ceki. The attached files will show you what I came up with. Synchronization issues are avoided by having the same Appender add events to the CyclicBuffer and dump its contents. Ease of use is pretty high, since log.error(message) triggers a buffer dump. Configuration changes would allow log.X and higher to trigger a dump. -----Original Message----- From: logback-user-bounces@qos.ch [mailto:logback-user-bounces@qos.ch] On Behalf Of Ceki Gulcu Sent: Thursday, October 02, 2008 3:13 AM To: logback users list Subject: Re: [logback-user] Dumping a CyclicBuffer Hello Mike, Comments inline. mweeda wrote:
Hi,
Objective: Dump a "history" of recent program operations to a log file when an Exception arises. By logging to a CyclicBufferAppender, a recent history at a fairly granular level (INFO, or possibly DEBUG) can be maintained without accumulating massive amounts of data, and with a minimum of overhead.
How do you link throwing an exception with dumping log buffer? Do you add the code that link the exception with dumping the log buffer into many try/catch blocks manually? That sounds quite intrusive. Have you come up with something else?
Setting this up to log to the Cyclical Buffer went well. To get the data out of the CyclicBuffer and log it to a file, a reference to the CyclicBufferAppender object is needed, and it is not clear how to get one. It seems like log.getAppender(String name) should return an Appender, but it doesn't. How can I obtain such a reference?
In the config file you show, the appender named MEMORY is attached to the root logger. So you would need to write: import org.slf4j.LoggerFactory; import ch.qos.logback.classic.Logger; import ch.qos.logback.classic.spi.LoggingEvent; import ch.qos.logback.core.read.CyclicBufferAppender; public class Main { public static void main(String[] args) { // The cast will work only if SLF4J is bound with logback Logger root = (Logger) LoggerFactory.getLogger(Logger.ROOT_LOGGER_NAME); root.warn("x"); root.warn("y"); CyclicBufferAppender<LoggingEvent> cba = (CyclicBufferAppender<LoggingEvent>) root.getAppender("MEMORY"); System.out.println(cba.getLength()); // some form of synchronization is needed here!!! for(int i = 0; i < cba.getLength(); i++) { System.out.println(cba.get(i)); } } }
Also, once a reference is obtained, and an Object is returned from cyclicBufferAppender.get(i), what class will that object be? I'm hoping it is a LoggingEvent object,
Yep, that is the case.
in which case a simple call to the fileAppender.doAppend method with the event as a parameter will log the event to a "permanent" log. Again, the code will need to obtain a reference to the appropriate FileAppender to log it to. (Appropriate might be hardcoded, or specified in a properties file.)
You could configure a file appender and attach it to the cyclic buffer if you wish. Joran is designed to handle such use cases. Please let me know if you want to pursue this route.
Most of the configuration is being done in logback.xml. (File is attached.) Program code, other than wanting to dump the MEMORY appender, is limited to: static final Logger log = LoggerFactory.getLogger(Main.class); and various log.X method calls.
As an aside, it would be even better to loop over all Appenders, and check to see which ones are CyclicBufferAppenders. The code would dump then contents of all of the CyclicBufferAppenders, or selected ones based on some user options.
How about the following code? LoggerContext context = (LoggerContext) LoggerFactory.getILoggerFactory(); for (Logger l : context.getLoggerList()) { Iterator ite = l.iteratorForAppenders(); while(ite.hasNext()) { Object o = ite.next(); if(o instanceof CyclicBufferAppender) { // dump } } }
Mike -- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch
Logback-user mailing list Logback-user@qos.ch http://qos.ch/mailman/listinfo/logback-user
Hello, Thank you for the attached files showing what you have come up with. While the dump trigger is clear (a log statement of level error or whatever the triggering threshold is), I don't see a "start accumulation" trigger. If I understand correctly, during the dump all events currently in the cyclic buffer will be dumped. If the buffer size is large, then all events up to the previous error will be dumped. If the buffer size is small, too few events may be dumped. Hence my question about the "start accumulation" trigger. Cheers, mweeda wrote:
Thanks for your help, Ceki.
The attached files will show you what I came up with. Synchronization issues are avoided by having the same Appender add events to the CyclicBuffer and dump its contents. Ease of use is pretty high, since log.error(message) triggers a buffer dump. Configuration changes would allow log.X and higher to trigger a dump.
-- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch
Hi, Messages that pass the filters for RECENT, and are lower than the DumpLevelThreshold, will *always* be logged to the buffer - there isn't an event that starts accumulation. Consider an application that is deployed to hundreds of users. A handful of system crashes occur every day, and the support team cannot reproduce the problem. While interviews or forms can be used to get some sense of what users were doing when they crashed, that can be easier said then done, especially if they aren't working at a desk. To help resolve this kind of issue, we want a history of detailed activity over a recent period whenever an error occurs. (When a previous error occurred isn't of interest - we want the activity prior to the error that has just occurred.) Sending detailed messages for hundreds of users to a file over a period of hours (days even) is not so good in terms of disk space, network traffic etc. This leads to the concept of a cyclic buffer that is dumped when an error occurs. You are right that the "default" settings might not gather an ideal period of history for every investigation. As long as the code was written with the ability to log sufficient details, it is easy to change logback.xml to capture more helpful data. Two changes in particular would be to pass more messages through the ThresholdFilter of RECENT (DEBUG or TRACE), and to increase the size of its cyclic buffer. A CyclicBufferDumpAppender could be attached to one or more class specific loggers, instead of root. That isn't quite the same as having a trigger event, but it could help capture relevant (hopefully) low level messages for a longer time in less memory. Mike -----Original Message----- From: logback-user-bounces@qos.ch [mailto:logback-user-bounces@qos.ch] On Behalf Of Ceki Gulcu Sent: Wednesday, October 15, 2008 5:18 PM To: logback users list Subject: Re: [logback-user] Dumping a CyclicBuffer Hello, Thank you for the attached files showing what you have come up with. While the dump trigger is clear (a log statement of level error or whatever the triggering threshold is), I don't see a "start accumulation" trigger. If I understand correctly, during the dump all events currently in the cyclic buffer will be dumped. If the buffer size is large, then all events up to the previous error will be dumped. If the buffer size is small, too few events may be dumped. Hence my question about the "start accumulation" trigger. Cheers, mweeda wrote:
Thanks for your help, Ceki.
The attached files will show you what I came up with. Synchronization issues are avoided by having the same Appender add events to the CyclicBuffer and dump its contents. Ease of use is pretty high, since log.error(message) triggers a buffer dump. Configuration changes would allow log.X and higher to trigger a dump.
-- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch
Logback-user mailing list Logback-user@qos.ch http://qos.ch/mailman/listinfo/logback-user
participants (2)
-
Ceki Gulcu -
mweeda