Hi Gena, Again, I agree with most of your points at a high-level and don't doubt that your solution can work. I do disagree with your comment about the main requirement of logging subsystems: it does depend on application requirements. For instance, in my application, speed is of utmost importance - I am willing to lose debug and warning messages in favor of keeping application performance at its maximum. The only messages that require full reliability for my application are warning and error events (this model is supported by AsyncAppender's current design with its discardingThreshold). All that being said, I'm not quite sure why you are resistant to a framework for implementing shutdown hooks. Is there a particular reason you object to the user being able to select which shutdown mechanism should be employed by Logback? I understand that you feel your solution covers most or all cases, but is it not possible that the end user of Logback may disagree and prefer to apply their own shutdown procedure or policy? As a side question, how long would you estimate it would take to modify logback-core (and/or the other logback components) to implement your proposed solution? Are you currently working on implementing this switchToFailSafe functionality or are you proposing that someone else from the logback community develop it? Thanks! Regards, Mike Reinhold On Thu, Mar 20, 2014 at 6:00 AM, Gena Makhomed <gmm@csdoc.com> wrote:
On 20.03.2014 2:45, Michael Reinhold wrote:
I feel like we are speaking past each other a little bit in that I am
not proposing any of the specific shutdown hooks discussed so far at a theoretical level (without being fully thought through or developed, they are of course flawed). Rather, I am proposing that my solution be implemented as a framework for Logback, contributors to Logback, and/or end users of Logback to build shutdown hooks that fulfill specific needs. Some of the theoretical hooks discussed could be implemented, knowing that they are not general solutions, but rather niche solutions that may be acceptable for some applications. Then, leveraging Logback's community of contributors (including you and me), more general solutions can be developed and included as part of Logback.
framework for logback shutdown hooks just not need at all, if only two simple solutions cover all possible use cases:
1) LoggerContext.stop for case app shutdown without JVM shutdown 2) LoggerContext.switchToFailSafe for case app shutdown == JVM shutdown
register logback shutdown hook, which switch logback from "normal" to "failsafe" mode have sense even for web-applications, because tomcat can be stopped via signal SIGTERM, and such shutdown hook will flush all AsyncAppenders and FileAppenders memory bufffers.
so, it is even not need any configuration option for enabling such logback shutdown hook, it should be enabled always for all contexts.
Based on this back and forth discussion, I don't think there is any
resistance to your proposed solution as it has the potential to be a reasonably generic shutdown hook that could be applicable to many common use cases.
can you provide any case in which my proposed solution not applicable?
Furthermore, it could be implemented as one of the pluggable
classes usable by the ShutdownHookAction.
java code is trivial: run(){ LoggerContext.switchToFailSafeMode(); }
If your implementation is as
generic and effective in practice as you have described, it could actually be the default ShutdownHook implementation. If for some reason a user found that your solution was non-optimal for their application, they could implement (or work with the Logback community to implement) an alternate ShutdownHook that does meet their needs.
right now exists only three available alternatives:
1. do nothing and lost all event in various memory buffers 2. use LoggerContext.stop and lost all events after stopping 3. use LoggerContext.switchToFailSafeMode for maximum reliability
can you provide any other alternatives for these three?
if even theoretically other possible use cases not exists here, may be not need implement framework for alternatives right now?
I hope this helps explain my position adequately - I don't see our two
solutions as competing alternatives, but rather as complimentary components of the shutdown cleanup functionality of Logback.
I am talking not about only shutdown cleanup, I propose more generic solution, just add new, fail-safe mode. Switching logback to fail-safe mode useful in logback shutdown hook to prevent messages lost, and also switching logback into fail-safe mode will be useful for low-memory conditions.
for example, we have one huge desktop application with many plugins. I wrote memory monitor, which continuously monitor java heap size, and continuously provide "live set" size of java objects in heap. also this monitor have "low memory" handler, which called in case if free heap memory (max heap memory - live set size) less then 50 MiB. ("live set" size == min used heap size for rolling window of last 5 min)
switching logback into fail-safe mode will be very useful for this handler too, for preventing logback events lost in case if application will be stopped via forthcoming OutOfMemoryError.
if AsyncAppenders not used and all FileAppenders configured with "immediateFlush = true" mode - for this special case "normal" and "fail-safe" mode will be identical. for other cases - not.
My portion
provides a framework for creating shutdown hooks that cleanup the Logback context (possibly including some simple implementations) and your portion includes a more complex, but fairly general implementation of that shutdown hook framework. This gives us the best of both worlds - the flexibility to plug different hooks in if needed as well as a solid default implementation that should cover most use cases.
There exists only one way for "cleanup the Logback context" - just call LoggerContext.stop method.
there is only one question - when do this cleanup. and only one strict answer, when LoggerContext.stop is required: this is case if application shutdown != JVM shutdown. In other cases "cleanup the Logback context" is not required at all, and switching logback into fail-safe mode is always better approach, because it prevent undesirable events lost.
if user required to stop logging before actual JVM shutdown - they always can manually call LoggerContext.stop in own code.
again, no need any special framework, LoggerContext.stop already exists.
what you really need - it is configurable timeout for stopping AsyncAppender worker thread, if AsyncAppender used as frontend for appender working with remote logging service and because now in logback not exists any way for making reliable logging to remove/cloud logging services without log events lost.
partial solution of remote logging is to play with timeout value and try to find acceptable tradeoff between application shutdown speed and application event lost rate from memory buffer on app shutdown.
optimal solution - just create reliable async appender for remote services, which buffer events on disk and provides fast application shutdown and also provides reliable events delivery to remote logging service even in cases of partial network troubles/failures/overloads.
main requirement for any logging subsystem: do not lost messages. all other wishes (speed,usability,etc) is secondary requirements.
This would seem
to fit with the methodology of design that Logback generally uses where multiple implementations are possible and generally speaking a few options are included in the library, but a default implementation that covers most scenarios is also included in the library and, unless specified otherwise, is the preferred option.
I can't imagine how multiple implementations of shutdown hook can better help prevent event lost if already exists almost perfect and safe algorithm for graceful shutdown of logback with minimal possible events lost via new "fail-safe" mode.
Once again, thank you for the good discussion!
You are welcome!
-- Best regards, Gena _______________________________________________ Logback-user mailing list Logback-user@qos.ch http://mailman.qos.ch/mailman/listinfo/logback-user