I found an interesting problem while configuring our Amazon Kinesis Connector. After starting up our service everything looks fine, but after 15 minutes or so, all hell breaks loose. The Connector is Amazon's code, and it does not throw any exceptions that our code can catch, rather it just reports a lot of errors in the log stream. This is clearly an insane situation that we need to catch, so I was thinking of writing a custom appender, filter, or turbo-filter to monitor the log stream for insanity. For example, errors per minute over some threshold. Does this seem like the right way to address this problem? Have other people used similar techniques, or other techniques to sanity check the log stream? Cheers, Eric
Eric, That's an interesting problem. I would be tempted to wrap one of those three (custom appender, filter, or turbo-filter) around a Metrics object: https://dropwizard.github.io/metrics/3.1.0/ Being that rates are one of the things that Metrics supports out of the box. Donald On Thu, Nov 13, 2014 at 1:21 PM, Eric Kolotyluk <eric.kolotyluk@gmail.com> wrote:
I found an interesting problem while configuring our Amazon Kinesis Connector. After starting up our service everything looks fine, but after 15 minutes or so, all hell breaks loose.
The Connector is Amazon's code, and it does not throw any exceptions that our code can catch, rather it just reports a lot of errors in the log stream. This is clearly an insane situation that we need to catch, so I was thinking of writing a custom appender, filter, or turbo-filter to monitor the log stream for insanity. For example, errors per minute over some threshold.
Does this seem like the right way to address this problem? Have other people used similar techniques, or other techniques to sanity check the log stream?
Cheers, Eric _______________________________________________ Logback-user mailing list Logback-user@qos.ch http://mailman.qos.ch/mailman/listinfo/logback-user
-- Family photographs are a critical legacy for ourselves and our descendants. Protect that legacy with a digital backup and recovery plan.
Thanks for the tip on 'metrics' I am studying it now. Cheers, Eric On 2014-11-13, 10:32 AM, Donald McLean wrote:
Eric,
That's an interesting problem. I would be tempted to wrap one of those three (custom appender, filter, or turbo-filter) around a Metrics object:
https://dropwizard.github.io/metrics/3.1.0/
Being that rates are one of the things that Metrics supports out of the box.
Donald
On Thu, Nov 13, 2014 at 1:21 PM, Eric Kolotyluk <eric.kolotyluk@gmail.com <mailto:eric.kolotyluk@gmail.com>> wrote:
I found an interesting problem while configuring our Amazon Kinesis Connector. After starting up our service everything looks fine, but after 15 minutes or so, all hell breaks loose.
The Connector is Amazon's code, and it does not throw any exceptions that our code can catch, rather it just reports a lot of errors in the log stream. This is clearly an insane situation that we need to catch, so I was thinking of writing a custom appender, filter, or turbo-filter to monitor the log stream for insanity. For example, errors per minute over some threshold.
Does this seem like the right way to address this problem? Have other people used similar techniques, or other techniques to sanity check the log stream?
Cheers, Eric _______________________________________________ Logback-user mailing list Logback-user@qos.ch <mailto:Logback-user@qos.ch> http://mailman.qos.ch/mailman/listinfo/logback-user
-- Family photographs are a critical legacy for ourselves and our descendants. Protect that legacy with a digital backup and recovery plan.
_______________________________________________ Logback-user mailing list Logback-user@qos.ch http://mailman.qos.ch/mailman/listinfo/logback-user
participants (2)
-
Donald McLean -
Eric Kolotyluk