Hi Gena,

Again, I agree with most of your points at a high-level and don't doubt that your solution can work.

I do disagree with your comment about the main requirement of logging subsystems: it does depend on application requirements. For instance, in my application, speed is of utmost importance - I am willing to lose debug and warning messages in favor of keeping application performance at its maximum. The only messages that require full reliability for my application are warning and error events (this model is supported by AsyncAppender's current design with its discardingThreshold).

All that being said, I'm not quite sure why you are resistant to a framework for implementing shutdown hooks. Is there a particular reason you object to the user being able to select which shutdown mechanism should be employed by Logback? I understand that you feel your solution covers most or all cases, but is it not possible that the end user of Logback may disagree and prefer to apply their own shutdown procedure or policy? 

As a side question, how long would you estimate it would take to modify logback-core (and/or the other logback components) to implement your proposed solution? Are you currently working on implementing this switchToFailSafe functionality or are you proposing that someone else from the logback community develop it? 

Thanks!

Regards,

Mike Reinhold




On Thu, Mar 20, 2014 at 6:00 AM, Gena Makhomed <gmm@csdoc.com> wrote:
On 20.03.2014 2:45, Michael Reinhold wrote:

I feel like we are speaking past each other a little bit in that I am
not proposing any of the specific shutdown hooks discussed so far at a
theoretical level (without being fully thought through or developed,
they are of course flawed). Rather, I am proposing that my solution be
implemented as a framework for Logback, contributors to Logback, and/or
end users of Logback to build shutdown hooks that fulfill specific
needs. Some of the theoretical hooks discussed could be implemented,
knowing that they are not general solutions, but rather niche solutions
that may be acceptable for some applications. Then, leveraging Logback's
community of contributors (including you and me), more general solutions
can be developed and included as part of Logback.

framework for logback shutdown hooks just not need at all,
if only two simple solutions cover all possible use cases:

1) LoggerContext.stop for case app shutdown without JVM shutdown
2) LoggerContext.switchToFailSafe for case app shutdown == JVM shutdown

register logback shutdown hook, which switch logback from "normal"
to "failsafe" mode have sense even for web-applications, because
tomcat can be stopped via signal SIGTERM, and such shutdown hook
will flush all AsyncAppenders and FileAppenders memory bufffers.

so, it is even not need any configuration option for enabling such
logback shutdown hook, it should be enabled always for all contexts.


Based on this back and forth discussion, I don't think there is any
resistance to your proposed solution as it has the potential to be a
reasonably generic shutdown hook that could be applicable to many common
use cases.

can you provide any case in which my proposed solution not applicable?


Furthermore, it could be implemented as one of the pluggable
classes usable by the ShutdownHookAction.

java code is trivial: run(){ LoggerContext.switchToFailSafeMode(); }


If your implementation is as
generic and effective in practice as you have described, it could
actually be the default ShutdownHook implementation. If for some reason
a user found that your solution was non-optimal for their application,
they could implement (or work with the Logback community to implement)
an alternate ShutdownHook that does meet their needs.

right now exists only three available alternatives:

1. do nothing and lost all event in various memory buffers
2. use LoggerContext.stop and lost all events after stopping
3. use LoggerContext.switchToFailSafeMode for maximum reliability

can you provide any other alternatives for these three?

if even theoretically other possible use cases not exists here,
may be not need implement framework for alternatives right now?


I hope this helps explain my position adequately - I don't see our two
solutions as competing alternatives, but rather as complimentary
components of the shutdown cleanup functionality of Logback.

I am talking not about only shutdown cleanup, I propose more generic
solution, just add new, fail-safe mode. Switching logback to fail-safe
mode useful in logback shutdown hook to prevent messages lost,
and also switching logback into fail-safe mode will be useful
for low-memory conditions.

for example, we have one huge desktop application with many plugins.
I wrote memory monitor, which continuously monitor java heap size,
and continuously provide "live set" size of java objects in heap.
also this monitor have "low memory" handler, which called in case
if free heap memory (max heap memory - live set size) less then 50 MiB.
("live set" size == min used heap size for rolling window of last 5 min)

switching logback into fail-safe mode will be very useful
for this handler too, for preventing logback events lost in case
if application will be stopped via forthcoming OutOfMemoryError.

if AsyncAppenders not used and all FileAppenders configured with
"immediateFlush = true" mode - for this special case "normal"
and "fail-safe" mode will be identical. for other cases - not.


My portion
provides a framework for creating shutdown hooks that cleanup the
Logback context (possibly including some simple implementations) and
your portion includes a more complex, but fairly general implementation
of that shutdown hook framework. This gives us the best of both worlds -
the flexibility to plug different hooks in if needed as well as a solid
default implementation that should cover most use cases.

There exists only one way for
"cleanup the Logback context" - just call LoggerContext.stop method.

there is only one question - when do this cleanup. and only one strict
answer, when LoggerContext.stop is required: this is case if application
shutdown != JVM shutdown. In other cases "cleanup the Logback context"
is not required at all, and switching logback into fail-safe mode
is always better approach, because it prevent undesirable events lost.

if user required to stop logging before actual JVM shutdown
- they always can manually call LoggerContext.stop in own code.

again, no need any special framework, LoggerContext.stop already exists.

what you really need - it is configurable timeout
for stopping AsyncAppender worker thread, if AsyncAppender
used as frontend for appender working with remote logging service
and because now in logback not exists any way for making reliable
logging to remove/cloud logging services without log events lost.

partial solution of remote logging is to play with timeout value
and try to find acceptable tradeoff between application shutdown speed
and application event lost rate from memory buffer on app shutdown.

optimal solution - just create reliable async appender for remote
services, which buffer events on disk and provides fast application
shutdown and also provides reliable events delivery to remote logging
service even in cases of partial network troubles/failures/overloads.

main requirement for any logging subsystem: do not lost messages.
all other wishes (speed,usability,etc) is secondary requirements.


This would seem
to fit with the methodology of design that Logback generally uses where
multiple implementations are possible and generally speaking a few
options are included in the library, but a default implementation that
covers most scenarios is also included in the library and, unless
specified otherwise, is the preferred option.

I can't imagine how multiple implementations of shutdown hook
can better help prevent event lost if already exists almost
perfect and safe algorithm for graceful shutdown of logback
with minimal possible events lost via new "fail-safe" mode.


Once again, thank you for the good discussion!

You are welcome!


--
Best regards,
 Gena
_______________________________________________
Logback-user mailing list
Logback-user@qos.ch
http://mailman.qos.ch/mailman/listinfo/logback-user