Thank you Raymond for sharing the inputs so looks like this is a bottleneck from logback especially with k8s ConfigMap. Is there any workaround (apart from restarting the java process) to address this issue ? OR I'll have to wait until it gets fixed from logback by considering checksum per below suggestion. Hello Ceki, please share your advice if you have any. Thanks, Kedar. On Fri, Aug 20, 2021 at 7:06 AM <logback-user-request@qos.ch> wrote:
Send logback-user mailing list submissions to logback-user@qos.ch
To subscribe or unsubscribe via the World Wide Web, visit http://mailman.qos.ch/mailman/listinfo/logback-user or, via email, send a message with subject or body 'help' to logback-user-request@qos.ch
You can reach the person managing the list at logback-user-owner@qos.ch
When replying, please edit your Subject line so it is more specific than "Re: Contents of logback-user digest..."
Today's Topics:
1. Re: logback autoscan is not working intermittently in k8s pod (Raymond Augé)
----------------------------------------------------------------------
Message: 1 Date: Fri, 20 Aug 2021 10:05:22 -0400 From: Raymond Augé <raymond.auge@liferay.com> To: logback users list <logback-user@qos.ch> Subject: Re: [logback-user] logback autoscan is not working intermittently in k8s pod Message-ID: < CAMm6HcC+fKzu4ux-z-WpDQK+LgWt_shwaEvB7OtR3J5dJe-UVQ@mail.gmail.com> Content-Type: text/plain; charset="utf-8"
Here's a direct link [1] to the explanation of the issue which I confirmed myself to be the actual issue I was facing.
Maybe it will help both the OP and logback :)
[1] https://blog.arkey.fr/2019/09/13/watchservice-and-bind-mount/
On Fri, Aug 20, 2021 at 10:01 AM Raymond Augé <raymond.auge@liferay.com> wrote:
Hi, a long time lurker here 🙂.
So, I suspect the issue is with k8s and it's volumes for config maps/secrets symlinks are a pain point in Java. Even inotify based WachService cannot handle them.
The issue is that logback is only using a timestamp to detect configuration file changes, which is fine as we know in 99% of Java usages.
However, this doesn't seem to work with the config map volumes supplied in a k8s env because they use symlinks which don't cause the timestamp to change, I agree it's weird and unexpected.
However, supplying configuration files via K8s config maps is going to start being a much more common use case as time passes.
Logback should probably start including a checksum to detect if the contents of the file have changed and use a logical OR with the timestamp check so that either changes causes a reload.
FYI, I had to patch a similar issue in Apache Felix Fileinstall due to the same k8s config maps/secrets issue [1]
My 2 cents, Ray
[1] https://issues.apache.org/jira/browse/FELIX-6229
On Fri., Aug. 20, 2021, 5:35 a.m. Ceki Gülcü, <ceki@qos.ch> wrote:
Hello Kedar,
Have you tried to reproduce the problem on a stand alone machine without k8?
On 20/08/2021 00:41, Kedar Sirshikar wrote:
Hello Ceki, I tried debugging by reproducing the issue and observed that after 7th attempt of log change, autoScan stops working.
Have you came across similar issue in 1.2.3 version where autoScan does not pick changes from config xml if it gets changed after a certain count ? In the meantime, I am trying to see if same issue occurs on 1.1.3 version because setMainWatchURL method inConfigurationWatchListUtil is different on both versions.
Thanks, Kedar.
On Sat, Aug 14, 2021 at 3:00 AM <logback-user-request@qos.ch <mailto:logback-user-request@qos.ch>> wrote:
Send logback-user mailing list submissions to logback-user@qos.ch <mailto:logback-user@qos.ch>
To subscribe or unsubscribe via the World Wide Web, visit http://mailman.qos.ch/mailman/listinfo/logback-user <http://mailman.qos.ch/mailman/listinfo/logback-user> or, via email, send a message with subject or body 'help' to logback-user-request@qos.ch <mailto:logback-user-request@qos.ch>
You can reach the person managing the list at logback-user-owner@qos.ch <mailto:logback-user-owner@qos.ch>
When replying, please edit your Subject line so it is more specific than "Re: Contents of logback-user digest..."
Today's Topics:
1. Re: logback autoscan is not working intermittently in k8s pod (Ceki Gülcü)
----------------------------------------------------------------------
Message: 1 Date: Fri, 13 Aug 2021 14:51:09 +0200 From: Ceki Gülcü <ceki@qos.ch <mailto:ceki@qos.ch>> To: logback-user@qos.ch <mailto:logback-user@qos.ch> Subject: Re: [logback-user] logback autoscan is not working intermittently in k8s pod Message-ID: <f39cbede-6e95-7e60-d01e-ca27599bf453@qos.ch <mailto:f39cbede-6e95-7e60-d01e-ca27599bf453@qos.ch>> Content-Type: text/plain; charset=utf-8; format=flowed
Hello Kedar,
I see that changes to the logback config file are picked up. There
are
some errors reported when processing the configuration file but
these
are unrelated to the issue at hand, i.e. auto scan not working.
I don't have further suggestions.
--Ceki
On 13/08/2021 09:59, Kedar Sirshikar wrote: > Hello team, > Just trying to check if anyone got a chance to look at this
case?
> Appreciate if any clue/hint is provided to resolve this issue. > > Thanks, > Kedar. > > > On Wed, Aug 11, 2021 at 12:57 PM Kedar Sirshikar > <kedargsirshikar@gmail.com <mailto:kedargsirshikar@gmail.com> <mailto:kedargsirshikar@gmail.com <mailto:kedargsirshikar@gmail.com>>> wrote: > > Resending the attachment by updating the subject. Please
let
me know > if any additional details are required from my side. > > Thanks, > Kedar. > > > ---------- Forwarded message --------- > From: *Kedar Sirshikar* <kedargsirshikar@gmail.com <mailto:kedargsirshikar@gmail.com> > <mailto:kedargsirshikar@gmail.com <mailto:kedargsirshikar@gmail.com>>> > Date: Wed, Aug 11, 2021 at 7:58 AM > Subject: Re: logback-user Digest, Vol 115, Issue 2 > To: <logback-user@qos.ch <mailto:logback-user@qos.ch> <mailto:logback-user@qos.ch <mailto:logback-user@qos.ch>>> > > > Hi Ceki, Thank you for the reply. Forgot to send the logs earlier so > please refer attachment. > I have also added Notes at lines 1 and 91 to describe the
test.
> > > Thanks, > Kedar. > > > On Wed, Aug 11, 2021 at 3:00 AM <
logback-user-request@qos.ch
<mailto:logback-user-request@qos.ch> > <mailto:logback-user-request@qos.ch <mailto:logback-user-request@qos.ch>>> wrote: > > Send logback-user mailing list submissions to > logback-user@qos.ch <mailto:logback-user@qos.ch> <mailto:logback-user@qos.ch <mailto:logback-user@qos.ch>> > > To subscribe or unsubscribe via the World Wide Web,
visit
> http://mailman.qos.ch/mailman/listinfo/logback-user <http://mailman.qos.ch/mailman/listinfo/logback-user> > <http://mailman.qos.ch/mailman/listinfo/logback-user <http://mailman.qos.ch/mailman/listinfo/logback-user>> > or, via email, send a message with subject or body
'help' to
> logback-user-request@qos.ch <mailto:
logback-user-request@qos.ch>
<mailto:logback-user-request@qos.ch <mailto:logback-user-request@qos.ch>> > > You can reach the person managing the list at > logback-user-owner@qos.ch <mailto:logback-user-owner@qos.ch> <mailto:logback-user-owner@qos.ch <mailto:
logback-user-owner@qos.ch
> > When replying, please edit your Subject line so it is more specific > than "Re: Contents of logback-user digest..." > > > Today's Topics: > > 1. logback autoscan is not working intermittently in k8s pod > (Kedar Sirshikar) > 2. Re: logback autoscan is not working intermittently in k8s pod > (Ceki Gülcü) > > >
----------------------------------------------------------------------
> > Message: 1 > Date: Tue, 10 Aug 2021 23:07:51 -0700 > From: Kedar Sirshikar <kedargsirshikar@gmail.com <mailto:kedargsirshikar@gmail.com> > <mailto:kedargsirshikar@gmail.com <mailto:kedargsirshikar@gmail.com>>> > To: logback-user@qos.ch <mailto:logback-user@qos.ch> <mailto:logback-user@qos.ch <mailto:logback-user@qos.ch>> > Subject: [logback-user] logback autoscan is not working > intermittently > in k8s pod > Message-ID: > > <CAO=
cJL+EMFCTe7En+7nqUmzaxQoYVEYN1GXCYwdwfkt0KbjiDQ@mail.gmail.com <mailto: cJL%2BEMFCTe7En%2B7nqUmzaxQoYVEYN1GXCYwdwfkt0KbjiDQ@mail.gmail.com>
> <mailto:
cJL%2BEMFCTe7En%2B7nqUmzaxQoYVEYN1GXCYwdwfkt0KbjiDQ@mail.gmail.com <mailto: cJL%252BEMFCTe7En%252B7nqUmzaxQoYVEYN1GXCYwdwfkt0KbjiDQ@mail.gmail.com
> Content-Type: text/plain; charset="utf-8" > > Hi team, > I have raised a detailed query here > <
https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working...
<
https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working...
> <
https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working...
<
https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working...
> about > the issue I am facing. > Can someone please share inputs or any hint for solving the > above problem? > Let me know if any details are required from my side. > > Thanks, > Kedar. >